According to a study, kids who were spanked at age one were more likely to be aggressive at two, and cognitively behind their peers at three. Verbal punishment didn't seem to produce these ill effects. [CNN]
Both yelling and corporal punishment usually tend to be the result of a parent expressing anger. They are always much less about teaching the child a lesson. I remember vividly how spankings (or the dreaded "belt") were virtually always the result of my parents just losing their shit. I can't remember a time when the physical punishment was preceded by a calm or rational discussion about what the child did wrong. And frequently the alleged infraction would be so vague that there was no "lesson" to learn. For instance, my mom would be upset for an unknown reason and refuse to eat dinner with us, and my dad would get angry, assume we'd done something, and come after us. The line between "spanking your kids" and "beating your terrified children" can be very, very fine.
But at least yelling shows some amount of restraint by the parent. Growing up in an environment where someone will hit you out of frustration and anger leads to some pretty big problems as an adult. And also a shitload of boundary issues. And also lots of concerns about having my own kids and losing my own cool.