Very good. I'm not a wave rider, but I've seen some Warrnambool & Pt Fairy guys going off along the East beach in some good conditions. The Loch is great mid-winter for flatwater blasting. Also only about an hour's drive to some other good spots - in both directions.
I windsurf quite a bit at Port Fairy, plenty of different locations for different styles of windsurfing and wind directions. Some nice surf spots as well
Chances are you'll love it but it depends where you're coming from?