With Manage Portlets UI, you can assign the groups that have SUBSCRIBE permission to a portlet. However you cannot assign BROWSE. With Customize, Marketplace, and Search respect only BROWSE permission.
This opens up the possibility you publish a portlet with groups (and possibly categories), but users don't have BROWSE permissions and you need to go to Manage Permissions to assign BROWSE.
If the users have BROWSE permission on a category the portlet is assigned to, the core issue is still present but sites may not notice the issue as much. This is a possible workaround.
Add a checkbox next to each group listed, defaulted to true, that indicates BROWSE permission is also applied (in reality it could actually change the group to BROWSE permission since BROWSE implies SUBSCRIBE).
Must track adds/removes from groups and checkboxes.
Nice to have, may not be part of this issue:
Add a link under Categories that lists the categories that have BROWSE permission and the target entity (principal) the permission applies to.
Alternate: provide a link next to each category listed that provides the list of principals that get BROWSE and list of targets that get SUBSCRIBE for that category name.