Is the field a child field or parent field? You should be able to delete fields off a custom record no problem by going to the custom record and deleting the field. It usually only complains if you are trying to delete a record that references the custom record. In those cases, you gotta delete the field there which removes the data and once references are removed, you can delete.