No, they won't. That's the simple answer unless you experience some strange idiosyncratic reaction. Of course, if you still feel groggy tomorrow, you already have the answer.
The half-life of diazepam and its metabolites is indeed very long, but the actual effect of diazepam is, depending on the dose, 6-10 hours. Xanax' effects only last for some 3-4 hours.
The half lives shouldn't interfere with the trip. Diazepam has an active metabolite, nordazepam. But I have never heard that, unless you take nordazepam alone, this metabolite should have any noticeable effect the next day after you take diazepam. If you only took diazepam for one day, it's metabolites won't build up enough to have an impact on a trip. If you'd taken diazepam for weeks in a row it may be different. Still it's the main effect, not the metabolites that you really feel. I wouldn't worry much about the half lives or metabolites, just enjoy the trip instead.