To me this suggests that you may not be completely ready to drop down from 4mg yet. Please try to bear in mind that it is not a race - there are no "winners" or "losers" there are only people who stick at it and those who don't. As with any other medication, sometimes a person needs a little more, sometimes they need a little less, sometimes they need a longer course of treatment, sometimes a shorter course suffices.
If a person had to go back to get a second course of antibiotics to fight off a chest infection, or a person who had to go back to get their hayfever meds dose raised a little, or a person who was in plaster for a bit longer than the fella next in line was when they broke a bone at the same time you wouldn't think of them as being "weak" or as having "failed" in any way would you? This is the same thing.
All that stuff about "weakness" and "failure" is from people who consider addictions to simply be a flaw in a person's moral character - the same kinda people who believe depression is cured by stiffening the upper lip and cramming down emotions even further below the surface, or who believe that AIDS (alongside whatever the most recent natural disaster happens to be) is a deity's judgement on the prmiscuous, the chemically hedonistic and/or the sexually "deviant". In other words, the opinions of the uniformed, the bigoted and the ignoramus.
Again, I would
strongly recommend speaking honestly with your keyworker and/or doctor or whoever you feel most able to open up to. Be asked to be referred to a different keyworker if there is one you get along with and trust more than the one you usually see. Whilst I totally understand the apparent need to keep some pills in reserve "just in case" (I do it too at times so am on no high horse here) this is also a sign suggesting the person is not quite yet ready to let go enough to drop down in dosage with confidence. It does get harder the lower you go because the relative amounts being cut out increase each time. This is a flaw in the system in my view and it ma be worth speaking to your keyworker about being tapered more gradually - perhaps 3mg rather than 2mg. These kinda doses can be worked out as bupe also comes in the form of Temgesic (identical to Subutex in all but name - if your keyworker believes Suboxone is anything other than a shallow marketing ploy for the uneducated they should try to develop more knowledge of their area of supposed expertise... but, yeah, good luck with that

).