I would have to say "YES" to the physical opioid tolerance never going back to normal - based on personal experience and from reading various neurochemistry articles, sites, blah, blah.
There are these inconvenient things like receptor upregulation that happens with prolonged and escalating opioid agonist use. Once you stop, those sites go inactive, but they never disappear.
Ergo, if I quit for a year and then go back on them like trying to control my pain with 10/325 percs, I could get, for instance, a 70% response of the original pre-opioid me. And then I would get physically dependent/addicted much much faster. I.e. addicted in a couple days vs. couple weeks for opioid virgins.
This whole receptor site deactivation that have been upregulated is why people get PAWS after they quit opies. It just takes time to shut them down by the cellular mechanisms involved. So, from what I read new receptor sites (those new mu receptors, for example, on your nucleus accumbens) just get inactivated, but never go away. Which is why we can never again be opioid naive virgins.
The only hope may be the use of either low dose naltrexone or ultra/nano dose naltrexone to activate the inhibitory opioid system.
I doubt that people fully understand how this stuff works at a level sufficient to create specifically targeted drugs that dock with specific opioid receptors like our own endorphins do and not cause the damage that these fakers that come from the opium poppy do. By drugs in the previous sentence I mean new and improved-upon endorphins - i.e. polypeptides with a transport protein all in a pill form that are man made and make you feel awesome and don't cause any addiction

Wouldn't that be great. You wanna feel like you are in love back in 5th grade, here is a blue pill - feeling lasts for 3 hrs, then goes away, no side-effects.