People playing crappy music or music that doesn't necessarily match the mood of the drug I'm on. For example, when I'm rolling, I want to listen to house music or drum & bass, not A Perfect Circle or Sublime. Blech.
Other people getting too fucked up to function rationally when you're having an otherwise nice trip.
Attempting to drive a vehicle when you're close to going into a k-hole. Quite a stupid thing to do, obviously, but a buzzkill nonetheless.
And being high around people who disapprove of your use is always going to be a bad experience.