Your not damaging yourself without running any Testosterone but you will be missing a majority of the bodybuilding benefits.
I would also like to add that the only way to put on substantial muscle with HGH is to run it with a high dose of Testosterone and run the HGH at 10+ iu's. Anything less will only contribute to fat loss, better sleep, better skin, and joint healing.
Additionally you won't see drastic results from HGH unless you have a few cycles under your belt and have worked out for 2 or more years. You need to have a proper foundation and the discipline in terms of diet and training to reap the gains effectively.