Hi Kitten
It sounds like there are several issues going on here, so I'll attempt to address them one at a time.
1. It sounds like your sexuality is actually quite complex and I wonder what part it plays in this. You said you might be asexual in a different threat but at the same time can only get off with him. You won't consider blow jobs or mutual masturbation "sex" (and many people, especially gay people, would differ and tell you this is sex) and you say you only feel fulfilled through vaginal penetration. IMO it's a lot of pressure to put on a man who has sexual issues of his own, to tell him he's the only person who can get you off and only through this one act (penetration). You might want to lay off the pressure and allow both of you to come in other ways - for example, allow him to go down on you until you come then finish himself off while you watch. Taking penetration off the menu for a while and allowing him to feel "man" enough to give you orgasms in other ways might help with his anxiety. Be a bit more creative.
2. He doesn't sound like a port addict necessarily but he sounds like he has the capacity to be dishonest and deceitful by lying to you and cheating on you, you need to figure out if you can trust him at all now or if things are truly fucked, and he needs to be willing to work on re-gaining your trust.
3. He also sounds like he has a threesome/group sex fetish which he is for whatever reason reluctant, unwilling or unable to explore with you. He is channelling his sexuality away from your relationship into porn and this won't be resolved until he can feel he can share his sexuality with you. This will mean admitting his own turn-on and kinks to himself, which he may not be mature enough to do. He needs to tell you about his fantasies and turn ons and you need to learn to use these to turn him on, for example talking about what you'd all be doing to each other if you were in a threesome while you have sex, how you'd like her to touch you, how you'd like him to touch her, how hot it'd be etc. Go gently... but insist he shares these with you. Right now he sees the sex he has with you as totally different to the sex he'd like to have in his head whether he admits it on not and it's killing your intimacy.
4. You need to go to couple/sex therapy if things don't get resolved, and if he's not willing to do this leave him. Yes you might be the third woman to leave him over this, but it might take five women dumping him to give him the kick up the ass he needs to get his ass into therapy.
5. No you can't just give up on sex and hope you can have a companion-based sexless relationship. 3 or so years ago I posted here with the same question, my boyfriend wasn't sleeping with me very often, wouldn't explore any kinks with me but (I later found) would jack off to porn when I was asleep. Even without all the other issues people on BL advised me to leave him because it wouldn't work and I resisted because I loved him so much... well they were right, we broke up nearly 2 years ago and it was one of the best decisions I've ever made. You will be miserable and lose all self-esteem if things don't get resolved; you can't lie to yourself forever, the most important thing in this life is to be true to yourself. Give it a certain time limit and if you doesn't get better walk off with your head held high.
I know how shit it can feel to be constantly sexually rejected, I wouldn't wish it on anyone, it certainly isn't normal and there are hundreds of men out there who'd consider themselves lucky to fuck the shit out of you. Don't stay when it starts to do more damage than good. Hugs xxx