Depends what you term "use". 50 whippets every hour every night for 6 hours?
Most memory "problems" are down to bone-idleness than any drug use. If you work on your memory anyone who has ever used drugs can have a memory better than 99% of the population. A few simple techniques can help anyone remember pi to 20 places within 10 minutes.