I'm sorry you're going through this Ruby. He has to want to get clean, there is really nothing you can do to get him to feel the way. If he is trying because you want him to, in the end he'll end up resenting you.
You said you have frequent talks regarding his using, and yet little changes. He is not ready yet. It could happen in time, or it may not happen at all - there's no way to tell. You have to ask yourself if his using is something you can accept and live with? If you accept it are you prepared for it to become more frequent and and more intense as addiction is a progressive disease and until he is ready to get sober, his using will continue to escalate over time.
Personally, I think you already have your answer and are looking for validation. You're not giving up on him, he is choosing a lifestyle over you. His problems are much deeper than just drugs, there is a mental component driving him to use. He may know what it is, he may not, but nothing is going to change with him until he addresses it. What kind of life do you want for yourself, is this something you're will to sacrifice for, and patiently wait for, because he has let you know his stance.
He is not willing to change now, if ever. He may say he will change, but his actions indicate otherwise. It's a big gamble for a relationship that isn't equal. You say there's not much drama, but you have a talk once every two weeks. I bet the number of discussions increases as his usage increases. You say he's difficult when he's drunk, what happens when he starts drinking every other night, or every night? How often do you ruminate when he's out, or worry, and wonder what's going on?
I have been with my husband for four years now, and I can count on one hand the number of actual arguments we have had. If one of us does something the other doesn't like, it only gets mentioned once - the behavior isn't repeated. I said mentioned, because it's not even a discussion it takes so little time. Seldom does anything like that come up because we are considerate to each other, and are mindful to respect the other person. I've only worried about his whereabouts once, because he left his phone at the house to run to the store really quick and it took longer than usual. He was late because he decided to run to multiple stores when the closest one was out of what we needed.
I use my relationship as an example to show that what you're experiencing isn't normal, and you deserve someone who values you as much as you do them. If you walk away from this relationship it's not because you're giving up on him - it's because you are are not his priority, and that's not your fault or anything to feel bad about. You deserve so much more, and I hope you find it soon!