I've had this happen a few times now. When tracking down a permission issue (a search in this instance) after switching roles from admin to the users role I'm still able to access things. If I login as a user that only has the role I'm testing then I am unable to access the search. Is this normal behavior?

I would seriously hope this is NOT normal behavior.

Yes it is. I have encountered this before

That's what I thought... I've also had this happen when tracking down an issue with a n/query usage where after switching to their role with my user it still worked, but the users reported it didn't. Sure enough it was the same thing where a new account without my admin access had a permission issue. Maybe it's silly of me for assuming a role switch would reset the permissions to that of the user role I'm switching to? (outside of things scoped to specific users/departments/etc) I guess does this mean permissions are additive and not related to the currently selected role?

If that's true, how would we verify any functionality of any role restrictions

Hair pulling, and a test account?

For saved searches in particular (and possibly N/query, I have very limited experience with it so far), I'm pretty sure that if you're the owner of the search, you can access it in any role even if that role wouldn't otherwise have access. The testing process you described sounds consistent with that behavior, though there may be other nuance depending on the permissions of the records/fields included in the search.

Good catch. That would explain what I'm seeing since I'm the implementer/tester in this scenario. Does the record owner have to have NS access? If not I should be able to stand up an internal account to hold our resources.

If you're referring to the owner of the saved search, they don't need access, but any scheduled/triggered emails may not work, and only admins will be able to edit it. If that's not an issue for your use-case, it could be a good way to allow you to confirm permissions issues on searches.

Does the record owner have to have NS access?

No, but they can't be inactive. Learned that one the hard way. Inactivating the user/employee record will cause anything that is tied to that user not to work. Really all bundles, scripts, workflows, searches, etc should be owned by a service user for that reason, but I don't think that's very common in NS, at least IME.

The other thing to remember is that if its permissions around a Custom Record, and you are the owner... you are the owner regardless of the role that you are using. So switching roles, will still enable you to see the record, search the record etc. But someone else with the same role, cannot.