I haven't examined any formal medical literature, but I'll give you my thoughts after doing some online research and hearing various testimonials:
I haven't seen one person claiming chlorpheniramine to be extremely dangerous provide irrefutable evidence for their words. It's true that anticholinergics are bad for your brain and eyes, but occasional doses won't hurt you. Alcohol is highly toxic to the liver and can cause cancer, but I know many people who have been drinking for years with no health problems. The fact that moderation is the key can never be sufficiently stressed.
Some patients take the maximum daily dose, 32 milligrams, without a problem.
http://www.drugs.com/mtm/chlorpheniramine.html
This means that eight tablets are safe to take.
The LD50 is 306mg/kg
http://www.drugbank.ca/drugs/DB01114
So, if this site is correct, and it probably is, I need a whopping 19890mg (being 145 pounds) to kill myself. My understanding from reading trip reports is that heart failure will kill me first at a dose of around 256mg. But I've only ever taken eight pills.
I've read about people who had really bad trips or had to go to the hospital at doses of less than 20 pills. Usually, these people did not start with a lower dose and work their way up so that they could determine what dose of cpm is dangerous for them. Start with four. If you notice no significant side-effects, wait a few days and try eight. If still nothing goes wrong, wait two weeks and try 12. If that's ok, wait another whole month before upping the dose again. If you're going to take triple CCC's, the trips have to be well-spaced out. Cpm has a long half-life and takes several days to be fully excreted. I can't find solid evidence that taking a reasonably low dose once in a while is dangerous. I talked to one dude, also 145 pounds, who took 64 pills within 48 hours without a problem.
But, being paranoid, I don't think I could exceed 12 tablets. Coricidin is tempting because it incurs very little nausea, whereas I can only tolerate small amounts of cough syrup/gels without vomiting. Chlorpheniramine made me hallucinate after the dxm had warn off. I saw what looked like the pokémon haunter floating in the upper corners of my room, and a medieval torture rack at the foot of my bed. But, since anticholinergic hallucinations are photophobic, I stopped seeing shit after I turned on a lamp.
Chlorpheniramine is not that bad, you just have to use it rarely, and most people have to keep the dose low.