That's why there's the calibration.
The wonderful thing about Kinect is it knows what audio the Xbox is playing. All it needs to do is figure out what the game audio sounds like (delay, echo, transformation) from each speaker. So when the Xbox plays a sound, it knows how to cancel it out by...