You have a point.
You can definitely do more and handle more if fit
Looking closely at kite ad videos, you will see a lot of medium and large size kites being used, especially the ones that are barely in the frame, while the rider is going fast or jumping
But in general for gear sizing, there are physics limits, like force per area of board needed to plane that determine a good board size for your weight, or how much lift it takes to loft you out of the water (or beach) with the kite overhead.
Airstyle riders sometimes flirt with the lofting risk, speed riders flirt with how much of pain it is to get a 90+kg guy going on a toothpick, versus the thrill once up to record chasing.
Within that envelope of possibility/safety, there is optimization to a task.
You need a certain amount of power to do certain things, more is wasted or problematic.
Aaron Hadlow lost in freestyle in strong winds at Leucate a couple days ago mainly by bouncing out on many of his landings (my opinion, watched some of the feed).
His kite was too big, he didn't need the extra power on landing and skipped out before he could set an edge. Still seems to be fit though.
Then there are common sense elements, like additional safety margin for changing conditions (risk of gusts, squalls, lulls, etc.), or rigging big because you know the wind will die off early.
In the end it's about fun, want a short powered jump session or a long mellower session in small waves? Want to push yourself and feel it the next day, or pace yourself and ride harder tomorrow? Choices, choices....