Gary, if you have any familiarity with handplanes, this example may help you understand what is probably happening.
Run a handplane straight down the grain on a piece of flat lumber. If you're moving it with the grain, it'll cut nicely with no tearout or bruising. Think of that plane's iron as your gouge cutting edge. Now reverse that piece of lumber and go "up the grain", you may encounter some tearout, or it won't plane as smoothly. Next, skew that plane as to present the cutting edge of the iron at an angle to the wood, and run it straight down the workpiece. It'll shear those fibers off more cleanly rather than lift them up and tear them off. If you now move that concept over to the gouge and approach the workpiece with the cutting edge at a shearing angle, the results will be similar.
The more sharp your cutting edge, the less likely tearout will occur, but its nearly always, on good stable lumber, a result of tearing that grain out rather than cutting it cleanly. That can be caused by presentation angle, dull tools, or a combination of both. Most all lumber I've worked with can be cut cleanly with proper technique. The exception might be punky or spalted wood, and some woods are more prone to tearout than others. Butternut for example - difficult, so is willow.
Some say that faster spindle speeds are better, but I always, always, have better results with tearout issues with slow lathe speeds. My final shearing cuts are done at 100rpm or less normally.
You can also treat the tearout areas in some cases to stiffen the fibers up so they cut sharply. Sometimes an application of walnut oil will do the trick. Sometimes it takes sanding sealer. Even yellow glue will do the trick. I generally prefer a lacquer based sanding sealer as it dries quickly and doesn't get sucked into the work as deeply - which could give you troubles when applying a finish.
Hope this helps some.