It could be there is just an over abundance of small agressive walleye that are everywhere. That's a good thing for the future but can be a pain today. I would try fishing deeper main lake structures for larger fish. If you continue to catch small fish then I've had luck with Gulp worms and leeches hooked on a jig with a stinger hook. Agressively jig it with occasional pauses held off the bottom. The smaller fish hit it but seldom get hooked. The larger fish tend to engulf it. If it's clear water jigging spoons can entice larger fish at times.
From my experience the large fish and the small fish don't mix all that much. You might find large groups of small walleyes with an occasional eater but usually the larger fish will be concentrated else where as they are probably feeding on different prey.
Have you fished this lake in past years? Were there alot of walleye the same size last year as well?
My guess is that you are probably finding a large amount of walleye from a particular year class that had a very low mortality rate that year. Which is great news for future years, once those fish are a few years older you will be finding larger numbers of bigger fish.
I would suggest moving deeper like was suggested or just keep moving until you find bigger fish.
The other possibility is that this is just what the lake can produce. If you have seen this same size fish for several seasons now and the larger ones just aren't there the lake might just not be able to support fish of large size. Or you could still be in the wrong spot and the spots you fish just tend to hold the small guys.
Thanks for the advice I moved out 5 feet deeper and sure enough I caught a 24 inch walleye and 3 17-18 inchers. Then I moved in to where i was a few days later to see if the bigger fish started and the small fish stopped but i still caught the small ones so thanks to all of you for the advice.
My biggest walleye (9 and 1/2 lbs) was caught trolling a cheap blue and silver imitation minnow lure along Lake Wenesaga's rocky bottom about 23 feet down. I felt it hit and then it was waterskiing behind the boat (I was trolling at a pretty good clip), so I didn't get much of a fight out of it.
Not that walleyes fight very good anyway. Good eating though!