Its simple to get diagnosed with a "psychiatric condition" these days. They will push those drugs on you and say "if you take these you will return to normal"...from my experience i had them pushed on me when i did not need them.
^True. I once had a Dr prescribe me mood stabilizers simply because I mentioned I had a family member who was bipolar. He said, "Well, you don't have any signs of bipolar but let's put you on this as a preventative measure just to be on the safe side". But in this case let's
assume that the OP has a reasonably good doctor and that the OP has a legit problem which is substantially interfering with his/her life. Like you I would recommend trying non-drug methods like exercise, therapy, lifestyle changes, as a first choice to cope with it, perhaps that is something they could discuss with their Dr if they have the kind of Dr who is knowledgeable about and amenable to those kinds of things and doesn't just assume that drugs are the only answer.
As for the particular drugs prescribed:
Desvenlafaxine (Pristiq) is an SNRI (serotonin norepinephrine reuptake inhibitor) antidepressant. It isn't an antipsychotic. AKAIK it actually weakly inhibits the reuptake of dopamine. There is some evidence that it might possibly even be a very mild opioid.
Olanzapine is an atypical antipsychotic. It has a number of mechanisms of action including dopamine antagonism, serotonin reuptake inhibition, antihistamine and anticholinergic effects.
So my thinking is that your doctor wanted to try you on an antidepressant but was worried that since people with a predisposition for hypomania sometimes find that the hypomania is worsened by an SNRI he wanted to add the antipsychotic at the same time to balance things out. It's not an unusual combination.
I guess my suggestion would be to talk to your doctor about your concerns about side effects and see what he suggests. If you and your doctor truly think you need to be on drugs right now for your problem he may suggest alternatives or he might just say to give it a try and to come back if you notice significant side effects though.
You may also want to see another Dr for a second opinion and to confirm or refute the diagnosis of bipolar.