I'll put it like this: if it were me, I wouldn't be terribly concerned about using Mirtazapine occasionally for sleep while taking Desvenlafaxine,
HOWEVER, I just checked and there is in fact an increased risk of serotonin syndrome associated with using both TeCAs and TCAs alongside any serotonergic. With that in mind, I think it would be inappropriate for me to say that it is absolutely safe for you to use both drugs together.
If both of these medications have been prescribed by your doctor, s/he is most likely aware of the risks, but if one of these medications hasn't been prescribed to you and you're concerned about the risks, perhaps you could input both drugs into an interaction checker like
this one and asses the risks for yourself prior to taking anything?