As far as possible, unless the user is an admin, users with no access to specific forums can't subscribe to this particular forum or topic and won't receive email notifications.
You can manage this by setting up the usergroup access correctly AND you have to apply the per forum permissions to a specific usergroup whether or not a particular usergroup has access to a forum.
In this case when a usergroup is overruled by the per forum permissions, the user within that usergroup with no access won't receive email notifications.
See screenshots (notice the differences between global and per forum permissions):
1) Global Usergroup "Registered" permissions
2) Per forum Usergroup "Registered" permissions (this overrules/overrides the global Usergroup setting)
I hope this helps.
Let us know how it goes.