It depends on what you mean by "difficult to handle", but I'm going to assume you mean this to be an approximate measure of the likelihood of inducing psychological stress.
Just based purely on my own experience, the easiest to handle is Ketamine, it's really not possible to have a bad time on that, although I've met many people who would disagree. Following that would be MDMA and MDA - although I have had a difficult time before on the latter substance, they are still generally safe bets for an almost guaranteed good time... but I think purely by virtue of being stimulants, and especially in non-novice users, MDxx substances still have the potential to induce anxiety and other related stimulant-associated uneasiness. This can also be a result of unpleasant body sensations such as increased heartrate, body temperature, etc, whereas Ketamine and similar dissociatives have little to no body sensation issues by virtue of their anaesthetic properties.
LSD for me is middle of the road, leaning towards easy although I'm hesitant to actually describe it as easy in case I have to eat my words at some point. It can be a very intense experience for sure, but my memories of it are primary very visual, and I think although there is a definite non-visual component, the headspace in some ways lacks intricacy... it's almost so muddled and bewildering that it's easier to just let go and go with it. Again this was only my experience, my friend who most recently did the same 300ug dose with me said afterwards that he found it close to being overwhelming and frightening at times.
The most difficult to handle for me, that I have experienced so far, would be high dose (~70g fresh, ~7g dried) psilocybe mushrooms... it's hard to quantify exactly the reason for this but it's just such a bizarre headspace, coupled with, for me, a somewhat uncomfortable bodyload and an intense emotional aspect to the experience. While mushrooms are visual in quite a unique and magical way the visuals don't have the same sparkly quality as, say, LSD, and I think the headspace is really the star attraction and a very very intricate place, in the same way that LSD isn't. I think this is what makes them difficult to handle as when you are seeing things in front of your eyes it's easy to understand that this is the effect of a drug you have taken, so that association in your mind isn't broken... but the psychedelic headspace is a much harder thing to quantify and understand.
I will note finally that although I would say mushrooms are the "most difficult to handle", my experiences with them have overall been fairly manageable which I put down primarily to setting. The most difficult individual experience I have had on a drug would probably have to be a high dose LSA / Hawaiian Baby Woodrose trip which was 10 hours of existential terror but I have no doubt that the severe nausea I experienced on this played a big part, as well as the setting which was very far from ideal. In fact this is probably an important note to finish on, that regardless of the difficulty of the substance itself, set and setting and good prior planning are always of paramount importance in making sure you are able to handle what a substance throws at you.
So far it has always been a reassuring result.