Create a Suitelet, when click the Custom Buttom make a get request to the Suitelet, Do all you need there and return a record redirect to the new or modified record...