What I learned in my experience is that:
1) When you learn at the very beginning, you would like to use the same kite size you would use on a surfboard.
2) when you progress, you definitively want to go down 20-30%. Great free riding in apparent wind.
3) when you are able to push your speed upwind & downwind then you can go up again with kite size, but preferably not strapless.
So it really depend on your weight, riding style and ability.
For the very first "try outs" using a normal kitesize is okay, but it only lasts till you are up and riding a bit more steadily, then it becomes useless with the same size as on a surfboard IMO.
And progressing, you go down in size, but mostly when the wind is just above the marginal region - then you go down A LOT in size.
But in marginal winds and in higher winds, the difference is not as extreme.
For speed and upwind/downwind, bigger kites are better indeed, BUT, if you want to excel and master the adventure of turning tight, ride waves, and 360's and everything - you most likely WONT go up in kite size IMO.
Two very different goals - speed OR agility/low end