The polarity on speakers and speaker terminals ensures that the speakers are in phase with one another. That is, when one speaker cone (right, for example) is moving forward, the other one (left) is moving forward. If you switch the polarity of one of the speakers (either at the recever or at the speaker), when the right cone is moving forward, the left cone is moving backward. This causes a cancellation of the sound pressure developed by the speakers, resulting in lower output volume and an overall weak sound to the speakers.
When wiring speakers, you should always maintain the correct polarity of the connections. Speaker wire always has some kind of designation between the two leads. Sometimes it's a different color jacket (red and black), or it's a different color conductor (brass or silver), or some kind of indication on the jacket (one side has ridges or some printing on it).
The speaker terminals at the receiver and the speakers should have some designation as well. This might be a + for one side and a - for the other, one is black and one is red, etc. Just make sure you always hook up the positive output terminal on the receiver to the positive input terminal on the speaker. I always make red the + color and black the - color. If the wire jacket is printed or ridged, I also designate this to be the + wire. So, you would always have the + or red terminals going through the red, brass, or printed/ridged wire, to the + or red terminal on the speaker.
That way, your speakers will always be wired in phase and you'll get the best sound from them.