Antidepressants are generally long-acting medications. They will build up considerably after a few days of taking them.
If the practice your describing is always just a couple days here and a couple days there, the health impacts are essentially no different than taking it every day. The only difference is that you're operating at a lower dosage than prescribed. If you're taking 20mg daily for two days and then none for two days, you can basically put yourself at 10mg per day for those four days. The difference is, you will experience the fluctuations in dosage. They will be lowest on day 4 and highest on day 2. Whatever benefits you're intending to receive from the antidepressants will be mitigated by these troughs in dosage.
But some people need less than they are prescribed to get by. At any rate, the best decision for your stability is to make a decision and stick to it. Otherwise you're going to be pulled in some way back and forth, even if it's subtle. If you're neurotic like me, I'd be questioning everything in my life trying to understand if it was a result of a peak or trough in my medication levels. Doesn't sound very productive.