Any way to create a custom field on a file or folder record?

Nope, but you can create a custom record and add a file field and custom fields. This way you'd have files linked to custom fields