Well, I firmly believe it's about how you internalize your experience. I believe that HPPD is a form of PTSD (Post-Traumatic Stress Disorder, like a war veteran might have. It's all about how a traumatic experience has affected you internally. Using the war example (and speaking as an American), there are people who end up with PTSD from any war. But in a war like World War 2, where people felt they were doing what they had to do to defend the world, it seems there are a lot less cases resulting from it than a war like, say, Vietnam, where so many soldiers ended up believing they were there committing atrocities on behalf of a crooked government. You end up, sadly, with a whole group of people who have flashbacks where they suddenly think they're back in that traumatic experience, and it really impacts their ability to live a happy and productive life. Sometimes people have it so bad that they're constantly at least partially in that state.
When you hear of someone getting HPPD, the symptoms include the same - suddenly, without warning, they'll re-enter into that traumatic experience, and it makes life a lot harder to live. Some people even constantly have lingering visual distortions and depersonalization/derealization, just like the PTSD war example I gave above.
The good news about psychedelics, however, is that it's always up to you how you view an experience and how you choose to internalize it. I think that sometimes people, especially young people who don't necessarily have the tools developed yet to deal with confusing and overwhelming experiences, choose, subconsciously or consciously, to view the result of such an experience as something that has damaged them. Or perhaps a person experiences ego dissolution, which shows them the true nature of their existence and the illusion of individuality and materialistic society, and they were not ready for it at all - they were happy living a life in which they knew all the answers and everything fit into a neat little box of conceptual distinctions, and the experience they had blew that apart, and they try desperately to reconstruct that box, but the experience they had cannot truly be denied. Whatever the case may be, they have chosen to view something that has the potential to be a freeing and beautiful experience as something threatening and perhaps even evil. In this case it has become a traumatic experience that has created a glaring contradiction that affects their life on a very deep, spiritual level.
It's best to live your life open to accepting new experiences and willing to allow your worldview to change. If you can do this, it's unlikely that any experience of that nature is going to give you PTSD, or as we tend to call it with psychedelics, HPPD. Personally I have tripped a great many times, and this includes some experiences that were very difficult and at the time I thought I would never be the same. And perhaps I wasn't. But I used them all as learning experiences and they have made me who I am today - and that is a happier, more well-adjusted person who is deeply comfortable with his understanding of the nature of reality.
The mind is a very powerful thing that can lead you into experiencing pretty much any state of being. Allowing a traumatic experience to get so deeply planted into your sense of self has the potential to produce some very intense depersonalization, derealization, and difficult emotions, and even perceptual changes.
That said, it's still something that a lot of people experience. Psychedelics are not 100% safe, and should always be used with respect and caution.