Yeah you should do something about that. 107 is not good at all. It's bad. Try taking an ice bath or drinking lots of fluids? Maybe you just misread the thermometer?
107 is really bad and you might want to have a sober person take your temp. as to make sure your not about to die. Drink some water(not to much) and relax. Are you sure they were good pills?