I was just having trouble with a similar problem trying to merge a Custom Transaction with a Template, with no luck. Last I knew we opened a support case on it as it simply reported "invalid record type"; not sure if there's a similar issue with custom transactions and messages