Nice with the oil, brownies usually call for oil rather than butter. Be sure to use it right away, the canna oil I've had seems to go rancid very quickly. As far as using the plant material that is left over... you could compost it... but that's about the only good use for it, so long as it's been cooked for a number of hours no actives will be around. Adding it to the brownies is the worst idea... who likes eating brownies with plant material?? You've already got all the good stuff in the oil!
Bon apetite