Are votes disregarded?

There is a conspiracy theory out there that the producers can and do ignore the votes, and simply send home who they don't want around any longer. The VoteForTheWorst crowd has helped to popularise this assertion.

What do you think? Is there any evidence to support this? Is there any way, or any evidence, to discount this theory?