The Prius doesn't have "little golf cart wheels." It has 15" wheels, which are a perfectly normal size for any sort of car built before about 2000, when wheel sizes went completely crazy. Hybrids do tend to have low ground clearance, but I have driven my Honda hybrid (with 14" wheels and snow tires) through all sorts of blizzard conditions here in Colorado without problems.

If you want to go off the road, what is wrong with walking?