It probably won't do you any physical harm if the dose if kept at 50-100mg. Trouble is when tolerance increases to to point they don't work anymore, this may cause you to increase the dose or try stronger opiates.
Also if your supply is disrupted then the withdrawals are considered quite severe for such a weak opioid due to its SNRI activity.
Sadly there's not many scenarios where using an opiate daily is advisable. If you can stick to a couple of times a week, this would probably be better.