I have a question. for cd/ci development do you usually start with the xml for creating a field or workflow in the sandbox or do you build it out and then get the xml from the sandbox for the source?

definitely the latter. I would only occasionally edit an existing XML definition for an object and push it to the test environment (such as bulk updating a field id, or changing audience, or script deployment status)