Clean your bong man, all it takes it rubbing alcohol and some regular old salt. You probably won't even need to leave the house to get the supplies and it takes only minutes to clean to perfection!
As for your question, a dirty bong can certainty be a breeding ground for bacteria. But what I find to be more relevant, its also pretty damn gross bro.
Just think about it man, no hot stoner chick is gunna wana rip your nasty-ass unwashed bong.. Am I right?
yeah dude, just pour in a bunch of salt, some water, and shake the hell out of it. simple. it will taste better when you smoke from a clean bong. there could be all kinds of nasty mold growing in there that you don't want to inhale.
I try to wash mine after every few smoke seshes... I mean, I change the water everytime the ice melts... I wish it out with hot water and soap every few days