Is there a way to convert a custom transaction PDF to an advanced PDF template? Maybe I'm dreaming but I did think there was a way before?

Not that I know of. Only thing I can think of that was sort of similar is when NetSuite would automatically convert email templates from the old templating style to scriptable/freemarker.