IIRC the preference "line item" disappears when you click Delete Preference, but when you return to the Review screen (and subsequently search & edit the same portlet) the preference is still there.
The Remove link works, so you can delete individual preference values... just not the whole line.
NOTE uPortal 4.1 does not seem to be affected.
I am running into this issue on Master
This looks like a possible duplicate of UP-3858. Can not reproduce in master as of 10/15/2014