I have a bunch of colors I need to add to a custom "Color" field on the item records. I don't need to associate them with items at this point, just need to load all the colors in. What's the proper way to import this CSV list?

you need to create a custom list "Color". From there you should be able to do a csv import . Import Type = Customization, Record Type = Custom List. On your csv file, there are 2 columns, 1 for the custom list name then the other for the color values.