So I got off the call with Netsuite Support and tested that the best way to handle this was to just void the bill payment and create a new file with just that payment.
We tested doing reverse payments but got odd results. We created a file with 2 bills attached and when they reversed it removed the 1 payment, but then created a completely new file with the other one (which is what I want to avoid). When I did it, it removed the 1 payment, but kept the other one on the same payment file.
To be on the safe side we are just going to void the bill payment itself. It shows VOID when you look at the older payment file.