At high HP levels, this drag becomes inconsequential, but for small engines, it's a big handicap.
That's why I don't mind A/T for the bigger-V6 engined cars. As an example, our 06 Sonata's 3.3L V6 is better mated for its A/T. Plus, it has a shiftronic to compensate even more. Our A/T 95 Contour with a smaller 2.5L V6 does exactly what niky said, searching wildly for the proper gear and sometimes not finding it. I wouldn't take it mountain driving.