in order for a slave to get huge they would have had to feed them alot
That's a good point. Food is plentiful now, but the increased metabolic cost of being big was once a serious matter. If anything, smaller humans give you more bang for the buck under a lot of conditions in terms of general performance.
Personally, I'm a bit closer to the 'hopeful monster' side of the genetic scale. At 200 cm (6'7") and around 300 lbs (136 kilos) (I'd probably be 'ripped' at about 275 or so, carrying some extra weight) I'm not cheap to maintain as far as food costs. I probably have a base metabolic requirement of around 4,000 calories a day, ie. that's what I need to avoid wasting away even if I'm just sitting around most of the time. You could feed two or three humans of more historically traditional sizes off that, and in many labor jobs I wouldn't actually be much more productive (if at all) than a small human. My size doesn't exactly gain me anything in ability to pick cotton or such.
Even if some slave holders did engage in selective breeding of slaves for strength (which frankly, I greatly doubt; they didn't have even the basic concepts of genetics down back then) you can be quite sure that they wouldn't have been selecting for peek athletic performance. Endurance, maybe, but not things like maximum cardiovascular capacity or reaction time or coordination.
I hate to feed the trolls, but there is rather extensive and conclusive evidence that the 'black' population is on average somewhat less intelligent than the other major ethnic groups (although this may be largely due to generally poorer education, nutrition, etc. rather than genetics.) Whites are about average, while east Asians have the highest overall IQs (mostly due to significantly higher visual IQ, which doesn't actually help with most reasoning tasks but may have some correlation to mathematical ability and such.) Saying such things can just about get you lynched by the political correctness squad, but the researchers are very confident of their results.
So, if you're looking to declare some group the Superior Race, it's most likely the Chinese, Japanese, etc. rather than blue-eyed blond Europeans.

(The differences are really only significant on the large population level; you can't make any assumptions about individuals based on these modest differences in averages.)
Populations do vary. We shouldn't be afraid of admitting that as long as we stay honest about what such differences do (and don't) mean. If you think you're better or worse than somebody else just because there's some statistical population difference between your respective demographic groups, you're full of shit.
