That error usually comes up because the transform call is passing through the handlingmode value from the source order, and SAVE_ONLY isn’t a valid option on a Return Authorization. In practice you need to strip out or overwrite the handlingmode field before you submit the transformed record.
In SuiteScript that means after you call record.transform on the Sales Order, remove handlingmode from the resulting record or set it to a valid value before saving. This keeps the transform from trying to carry over the unsupported flag.