I'm 26 right now and planning to return to university for a 2nd degree within the next 2 years. I'm no longer interested in my current field, and I know I want to pursue pharmacy instead, which is completely unrelated to what I do now. So aside from having different work experience up until this age, you and I are in the same boat! I'm extremely confident that I can successfully obtain a pharmacy degree and become either a pharmacist or move onto to graduate school in pharmacology; age is not a factor for me, and it shouldn't be for you either. It's not too late.
However, I advise you to think hard about what career you want after university. It sounds like you're interested in pursuing graduate school and research, which is very admirable but also very risky. Just think of how many professors there are to grad students. Academia can be a very competitive cutthroat environment. My interest is definitely in pharmacology research, however I'm going for a pharmacy degree to ensure I have a decent backup career as a pharmacist in case I can't make it in academia. To be safe, consider an applied sciences degree such a chemical engineering, which can lead to many great career opportunities immediately in addition to providing you with a background for graduate school in pure chemistry. A pure science bachelor's degree, while still impressive, is much less valuable in the job market than an applied sciences degree. I guess this is where age does factor in a bit - you can't afford to make a mistake that sets you back several more years, whereas an 18 year old can.
Good luck!