This period of time could certainly be unpleasant to come off of, so I recommend to pretty much right away dropping to 1mg/ day or less, and to start skipping days in between doses. This will act as a good transition between dosing daily and not dosing daily. After you do this, just start fitting as many days in between doses as you can, and you should get out of this relatively easily before it gets to the point of needing a long, gradual taper, which would be needed with even an additional 2-3 weeks most likely. If you put this off, just remember that every day you prolong this will make the addiction you have more serious and difficult to break.
Good luck.
Edit- ...So pretty much the same advice I gave above applies, except that now, the caution is stronger: you have pretty much reached the limit of still being in the limbo stage between a binge and an addiction.