It would appear that when both Masquerade and Devel's module "Switch User" are enabled the results get confusing: "switch back" does nto quite work anymore.

The workaround is to stop using two modules for essentially same function.

Masquerade seems to be more secure in production environment.
You do not have to get rid of Devel: just uncheck everyone's permissions to switch users and/or disable the Switch User block.