The problem is, the inexpensive, lightweight amplifier will lapse into distortion, at the drop of a hat. The only way it will play cleanly, is at a very low level. So, when you take the same inexpensive, lightweight amplifier, and try to listen to it at any kind of normal or moderate listening level (as is natural), it will actually be distorting (at least a little bit) pretty much constantly.
So, even when you don't hear it as full blown distortion, you may first notice it in the bass frequencies, which will sound thinner, or lighter, or less clean (because the bass frequencies contain a lot of energy).
Similarly, all the musical peaks may begin to clip, and become slightly ragged, so, like the bass frequencies, even when it's not quite enough to perceive it as outright distortion, you're still, in effect, placing a hard limit on the dynamic range you hear.
So, it kind of begs the question, to say that you will always be able to keep it below the level where the distortion begins to alter the character of the sound, because it's never that clear of a line. You may not perceive it as outright distortion, until it's already at a pretty high level, like 5% or more.
The net result is, in any real world scenario, it's unlikely you will ever be playing the inexpensive amplifier at a setting where it is truly playing as cleanly as the more powerful amplifier (because your ears are not an accurate enough way to determine that, and because it would require playing it so quietly you wouldn't fully enjoy it).
The differences you would hear, primarily, between the cheap amplifier and a better one, is that the bass frequencies sound thinner, or lighter, and that the overall dynamic range is flatter, more compressed (i.e., creating a less enjoyable/more tiring sound, a blaring quality).
Dynamics are what makes music enjoyable, and what makes music sound real. That requires power. There's no easy shortcut around that.