This is actually the title of a book: How Not to be Wrong: The Power of Mathematical Thinking. With a background in math, I keep my eyes open for books centred around math. There are many to choose from and they all revolve around the same general theme (which I agree with): Math is nowhere near as boring as you were led to believe in school and learning about it is good for you.
It's an unfortunate but true fact that if someone is going to deceive you (either intentionally or because they themselves are honestly mistaken) there's a very strong chance they'll be able to pull it off because your brain isn't very good at mathematical thinking. Just the inability to wrap our heads around simple statistical concepts is shocking and is responsible for so much suffering in the world.
Don't feel too bad, it seems to be a universal human trait that can usually only be overcome with formal training. I'll admit it comes more naturally to me but I've still invested a huge amount of time learning and practicing it on top of my natural aptitude. But a little education does seem to go a long way, so books like this are extremely valuable.
I was particularly tickled when the author basically gave away the secret to all magic tricks, in the middle of explaining misleading stock quotes!
[L]ike all good magic tricks, it doesn't try to fool you outright. That is, it doesn't try to sell you something false − rather, it tells you something true from which you're likely to draw incorrect conclusions.