It really just sounds like you're a non-responder. Some drugs can be absolutely great for some people while giving others next to no effect at all, or even making them feel downright awful. For example take Ritalin - it's enjoyed a lot by many users (some going as far as to compare it to cocaine), but personally it gives me nothing but anxiety, panic attacks, dysphoria and a horrible body load. Different strokes, etc. - I'd just ditch it and move on if I were you.
Admittedly I've rarely taken phenibut on its own. I find it really shines as an enhancer or reinforcer for the effects of other substances you take with it.
Phenibut + alcohol: quite agreeable, although not a combination I could really recommend from a harm reduction standpoint. You need to drink less to get the same amount of pleasant effects, and thus end up with fewer of the unpleasant effects from the alcohol. The phenibut has a much longer half-life than alcohol, so I doubt you'd have to worry about any nasty rebound from the phenibut wearing off before the alcohol unless you were drinking utterly ridiculous amounts.
Phenibut + benzos: likewise, not a combo I'd outright suggest trying due to harm reduction reasons, but IME the result is needing significantly lower doses of the benzo to achieve an anxiolytic effect. I could imagine this possibly being useful for somebody coming off benzos (although phenibut itself has addictive properties for a lot of people, which is definitely something to consider).
Phenibut + modafinil: one of my favourite combos, particularly from a functional perspective. The phenibut really enhances the energising, calm focus and clear-headed feeling of the modafinil, while taking the edge off some of the more unpleasant stimulant-type effects. Even drinking mild to moderate amounts of coffee on this feels fine, and doesn't leave me with that horrible over-stimulated/scatterbrained feeling which taking modafinil + coffee alone (or mixing stimulants in general) can cause. I wouldn't call it recreational as such, but it does definitely improve your organisation and make work or studying feel much more interesting and rewarding. And once you get into something you're doing, you can easily spend several hours on it and not notice the time go by, not being at all hampered or distracted by performance anxiety, tension/trepidation, or even hunger (this can be a double-edged sword however - if you don't have at least a rough plan of what you want to achieve beforehand, you can just as easily wind up wasting those hours re-organising your iTunes library or tidying up your house/dorm all night, blissfully indifferent to following morning's deadline for that essay you were supposed the be writing). It's certainly a useful combo for students, but I'm unsure how sustainable heavy studying on it could really be in the mid to long term. I'm aware that modafinil is sometimes prescribed for night-shift workers, but even then I think it's with the assumption of at least 5-7 hours sleep; I can imagine how pulling frequent all-nighters like this could potentially end up inverting your sleep-wake cycle or throwing you into sleep deprivation psychosis (not fun in the least - trust me).