Ketamine is really affecting two types of glutamate - NMDA receptors, and AMPA receptors. NMDA receptor activation leads to activation of inhibitory (GABA) interneuron, while blocking NMDA receptors with ketamine can lead to downstream release of various neurotransmitters (because an NMDA receptor blocker effectively turns off the GABA interneurons). There is also some aspect of ketamine's metabolites stimulating excitatory AMPA glutamate receptors.
Its possible that you either had a hard time with the NMDA antagonism turning off GABA interneurons or the excitatory AMPA agonism - either one tends to increase excitability in the short term.
On the other hand there are drugs that oppose glutamate - benzos, lamotrigine tends to decrease AMPA, perampanel is an AMPA antagonist.
Then there are the racetams like Piracetam, that increase AMPA. These can tend to cause anxiety and I have seen them cause the "24/7 conversation in the head" type deal