That's a pretty low dose of valium. The concerning part is that you've been dosing for a good amount of time, so I would imagine it has built up in your system.
I'd think even a low dose like that, you will still have discomfort coming off. People have nicotine WD from just smoking one ciggy a day. Horrible comparison, I know, however it shows the body has developed a dependency.
I wouldn't imagine you would have severe WD like seizure watch or anything like that... but your body will know something is missing.