I'm not 100% sure what threshold dose I would start to notice since my lowest dose ever was 225mg. But since you've had only 40mg to feel it and 120mg seems too strong to you, I think you maybe have the CYP2D6 enzyme defiency that makes people extra sensitive to DXM because your body cannot metabolize it properly.
I would advise you to extra careful when trying DXM since if you do hold the deficient enzyme, normal recreational doses for most people could be deadly or very upleasant for you.
But you could just be particularly sensitive to the effects of DXM and not have the deficient enzyme. Also if you are on any medications, they could interact with DXM by interfering with your metabolism or increasing/decreasing certain neurotransmitters in your body.
If you do want to carry on trying DXM, I recommend just doing low doses and increasing very slowly to see your reaction to it.
I think you should be particularly careful with this drug as you seem like your either very sensitive to the effects, or you carry the CYP2D6 enzyme deficiency which means you carry significant risk when trying this drug.