Well, yes, there are other things going on. But the idea behind using lossy compression is that a small loss can yield an incredible increase in compression rates. The trick is to find what bits to throw out.
Also, any waveform can be described as a (possibly infinite) series of sine waves of varying frequency - that's what the Fourier transform is supposed to do.
There are plenty of good articles and FAQs on MP3 compression (the technology is as old as first generation MPEG), I just can't seem to find any right now (without bumping into MP3 spamming sites).
IIRC, one of the first things an MP3 encoder does is send the wave through a low-pass filter. I think even regular CD players do this at 20 kHz to avoid aliasing at high frequencies, since theoretically, CD audio can reproduce frequencies up to 22050 Hz. Note that sampling rates and frequencies are not the same - as a 'rule of thumb', sampling rate needs to be at least 2 x highest frequency.
Anyway, since high frequencies are much harder to encode than lower frequencies, the low-pass filter really helps on compression. Depending on the encoder and the file to be compressed, a 128 kbs MP3 can have a high frequency cut-off as low as 12-13 kHz. Even 192 kbs MP3s cut-off at 18-19 kHz, but that's okay, since it's hard to hear anything higher than 19.5 kHz, and that's if you have the tweeters to output that high, and don't have any other sounds masking it. (The old Xing encoders automatically cut off at 16 kHz, IIRC.)
BTW, all that is only considering constant bit-rate MP3s.
Sorry if I'm being confusing, I'm just writing anything I can remember about this. This post isn't very coherent...