If all you're using buprenorphine is for a detox, you could stop today, or ideally take a lower dose for two or three more days, and suffer little ill effects. If you're only been taking 4-6mg of buprenorphine per day for three (or up to seven) days, you won't experience acute withdrawal from the buprenorphine. You will likely still experience some of the withdrawal symptoms, a general malaise, some insomnia, perhaps GI discomfort (cramping, diarrhea, nausea), heightened anxiety/feeling of alienation, regardless of whether you stop taking the buprenorphine today, in a few days, or taper down over the course of a few more days. But those symptoms would be the result of your body finishing it's detox as a result of your prior opioid dependency.
Generally speaking it would be ideal to take buprenorphine for a full week when using it to detox from short acting full agonist opioids like those you used, and tapering down as opposed to just stopping at your current dose (but considering your habit and the very long acting nature of buprenorphine, this is far from necessary). Ideally you can access other comfort meds to deal with the other symptoms, although much reduced compared to cold turkey detoxes, you will likely experience (gabapentin, clonidine, sleeping meds, etc).
The kick is the easiest part of this process though. Given your circumstances, I doubt it will be very difficult for you, regardless of whether you stop the buprenorphine today or taper down over the course of the next three or four days. I would highly suggest trying a day without taking any, and if you do feel you want to use it for a few more days, reducing your dose to no more than 2mg. Skipping days is very effective. Unless you feel maintenance is necessary (which really should be accompanied by other treatment, ideally a good therapist and psychiatrist combo), there is really no point in taking Suboxone for more than 4-10 days (assuming the detox isn't for a longer acting opioid like methadone, which is not your issue).
Like I said though, your kick is the easiest part of this process. Addressing the issues created by your (I will assume negative) patterns of drug (mis)use/addiction, as well as issues that existed prior to your drug use that may have led you to self medicate ("underlaying issues") is really the name of the game. Finding some supportive communities and integrating yourself (or sense of self/identity) into them, to become a part of something bigger than yourself that is supportive of your authentic wants and needs - most importantly one you feel genuinely/authentically connected with them, as opposed to having to force or fake your way along - and developing passions/hobbies outside of opioid (mis)use that are healthy and supportive to general, long term recovery and stability are the harder part aspects of dealing with opioid addiction and dependency.