Tell your doctor you're having breathing issues. It can be fatal and the way he's just upping and upping the dose seemingly against your desire if you die he could and very well might easily be charged with manslaughter for his gross negligence, which is possible due to a recent court case.
Tell him you're having breathing issues and want something less powerful. Your tolerance will be high due to the fentanyl so it will take time to adjust and find an equivalent. Just make sure he doesn't try to take you off opiates completely because of it, but I doubt that would happen because you had normal opiates side effects from an extremely potent opiate. If he did cut you off there would be something seriously wrong with that Dr and I'm sure would put his job at risk doing that.