Use a non-inventory Item for the adjustment, and set it to have a zero Tax schedule

Just saw this - this is a really simple and creative solution that we can explore. Thank you so much for sharing!