Clomipramine is a stronger reuptake inhibitor of serotonin, but that is not the same as saying that it is a better antidepressant than imipramine. The latter has a lot of functions besides inhibiting serotonin and norepinephrine reuptake, like stimulating monoamine receptors and effects on the opioidergic systems.
Also, affinity is not the same as potency. Affinity is just a general indicator of potency, but half inhibitory maximum is a much better indicator of potency. The reason why IC50 is generally not used much is because it requires you to know the experimental conditions, like substrate concentration. So it is difficult to compare drugs using this measure. Affinity is much more universal, which is why it is used. But affinity is a poor measure of potency. For instance cocaine is nothing special when it comes to affinity to the dopamine transporter compared to, say, anfonelic acid. And yet, pretty much everyone who has used both will tell you that cocaine is much more euphoric. That is because cocaine inhibits DAT in the open forward conformation by inhibiting DATs hydrogen bond, which is a particulary effective way of stopping DAT from doing it's job. As another example, moclobemide, the RIMA used to treat depression, has an insignificant affinity for MAO of 200 Um. That is so weak that it is considered insignificant. And yet, moclobemide is effective at inhibiting MAO-A. We know that it is because moclobemide when combined with SSRIs induce severe serotonin syndrome. If it were not effective, then it wouldn't induce ST when combined with therapeutic doses of SSRIs.
There are many, many factors that determine how potent a drug is besides just affinity, such as: resistance to hydrolysis by hydrochloric acid in the stomach, absorption through the G.I tract, degree of first by-pass metabolism in the liver, whether it induces or inhibits liver enzymes and what type, level of protein plasma binding in blood, and binding profile to the receptor or enzyme that it targets. In general, orally active potent drugs tend to be small molecules of a non-polar chemical configuration, with low degree of first by-pass metabolism, relatively low protein plasma binding and low nanomolar affinity for the target. Examples would be tghe SSRI escitalopram and the MAOI, rasagiline. Being lipophilic is also important as most cells have lipid layers, and crossing them is important for a molecule to reach it's target site.