Stretching my memory, but I'm thinking there's actually a different field ID for the line item field and it's not immediately obvious. Like if your custom segment is customrecord_yourvalue, the line item value is something like customrecord_yourvalue_line.
I could be totally off on this, but a quick way to check would be to just hand-key a transaction in a Sandbox that's close-ish to what you're attempting to do via the RESTlet and fill in the line item value. Then use the
field explorer extension or something similar to see what the field ID is that's being set.