I've done that combination plenty of times, also ketamine with other benzodiazepines. Personally I think valium dulls the ketamine experience, making it somewhat less interesting, although not weaker, just not as interesting. A lot of my friends have also done this combination; for as far as we know it's rather safe. I've also combined ketamine with 5 mg of flunitrazepam (roofies), which would be around 50 mg of valium if you compare benzodiazepines, which was fine too... all anekdotal 'evidence' though; me and everyone I know are always fine combining ketamine with benzo's but that's no proof ofcourse.
If you want to be save, take a lower dose of K then you normally would and see how it goes.