It's probably because temazepam is the most toxic benzodiazepine, and that is according to several independent studies. Alprazolam is 40x more potent than temazepam by weight (0.5 mg alprazolam = 20 mg temazepam), yet the LD50 for temazepam is 833 mg/kg and compare that to alprazolam's LD50 is 2171 mg/kg, triazolam's LD50 is 1625 mg/kg (triazolam is even more potent than alprazolam), and flunitrazepam's LD50 is 1200 mg/kg. Temazepam is TOXIC as shit!
I haven't done the research on it, but are you sure this is a valid line of reasoning? Everything has an LD50, some things higher, some lower, however just because one substance has a lower LD50 than another doesn't indicate increased negative reaction at sub-lethal doses. For example the LD50 of fentanyl is very low, however that doesn't make fentanyl in anyway more "dirty" than heroin... Take alcohol: You will end up sick and hung over after far less than the LD50. Point being, LD50 is not a good indicator of how "clean" or "dirty" a drug will be. It simply shows the relative toxicity of the drug, which could simply be due to stronger autonomic activity.