I can't do desert climates, so that has always discounted me from ever going to Burning Man. The other thing is the cost. For a festival that prides itself on "radical inclusion", it's pretty privileged. $400 per ticket plus having to pack in everything you need makes it unattainable for many. A couple of my friends have been to Burning Man and have shown me many awesome photos, but it still looks too harsh for me. That, and the hardcore Burners I've met are usually irritatingly aristocratic individuals from within that sub-culture.
I prefer festivals that include forest, mountain, river, or ocean.