Feeling benign doesn't mean you couldn't cause serious harm. 3-5 times a month is way too much. You should really look on Erowid, there's an Article from William E. White about the Neurotoxicity of dissociatives, i.e. Ketamine/PCP/DXM.
I was also a bit reckless, but i think it's safer to limit use to no more than once a month, perhaps less. If you're doing it then a second time on a special occasion, it's surely better than 3-5 times or even more. I know, it's a lovely substance, something really unique, but i think there's more than meets the eye.
BTW, even if the neurotoxicity isn't that much of a prob (which i doubt), there's also the toxicity to the bladder, which is there, only question is: to which extent?