is there a way to add html to the alert messages created via a workflow?

i wouldn't think so, but I wouldn't rule it out.

Not to the pop up box that I am aware. but you can create a field that displays on record conditionally. you can use HTML in the field and make it inline so it looks like an alert on the record