you can throw a JS exception (that can be a NS error object as battk says) in a beforeSubmit.
That being said, that's your equivalent to "server side validation". If possible you should also try to do client side validation for a better UX if it's a common scenario, because the error page looks nasty and the user loses all progress of record creation/editing