@jen, sorry—I initially thought you were referring to the saved search.
Anyway, If you check the setup of your portlet, you'll find an option called "Allow Inline Editing" (as shown in the attached screenshot). Please make sure this option is disabled.