for us, it tends to be new features not working as advertised, so worth hanging back a release to avoid the headaches unless you REALLY need the functionality. It's very rare that an upgrade breaks anything beyond small annoyances (like asset proposals having to be done by individual asset type. Still scratching my head on that one)
Two other things that spring to mind:
• Bank file formats (the default ones) can change without notice and they can work for some financial institutions but not others) - also they get released outside of the normal upgrade cycle
• Don't change the formula on the system-generated depreciation methods in FAM; a new release resets these to the old formulae which can stuff you up if you don't notice. If you need a custom formula, create a new depreciation method