The Problems With Statistics

>> Thursday, March 1, 2012

I think I've mentioned statistics before and why tend to be leery. Maybe on one of my other blogs. Fortunately, Relax Max provided an example on his blog post. Fun with numbers! Go there first if you want to see the problem and solve it without seeing the answer 'cause I'll be doing some math below but bear with me.

The problem:

Suppose that a barrel contains many small plastic eggs. Some eggs are painted red and some are painted blue. 40% of the eggs in the bin contain pearls, and 60% contain nothing. 30% of eggs containing pearls are painted blue, and 10% of eggs containing nothing are painted blue. What is the probability that a blue egg contains a pearl?
This is quite solvable for everything but the total number of eggs.With four variables and four equations, it's workable to solve the ratios of red(R), blue(B), pearl(P) and empty(E) eggs.

.4*(R+B)=P
.6*(R+B)=E
.3P+.1E=X
.7P+.9E=Y

Now, I could show the steps, but I'm too lazy, but, from this, we can deduce that the pearl:empty ratio of blue eggs is 2:1 and of red eggs is 14:27. The ratio of pearl:empty for the total, of course is 2:3 and the ratio or blue:red eggs is 9:41

That doesn't count the eggs BUT, unless we accept the notion that there are partial eggs involved, we can figure out a least possible number of eggs. When I did this in my head on the way home, I misdid the proportions of red and blue to 8:41 and realized we'd need a factor of 49, but I realized my mistake later and revised it to factors of 50, including 100. However, that's wrong, too. If we have to keep ourselves to whole numbers (which seems logical), the per red egg proportions have to be right, too and you can't do that with a total number of red eggs of 30 or 60.

My calculations give me a minimum number of eggs of 6150; however, if any of you find a smaller number I'm more than willing to compare notes.

So, why the math problem? Are any of you still with me? Because, with all those numbers I've tossed out like I'm knowledgeable and have said something significant and absolute, there are these important little things called caveats and I have to thank soubriquet for pointing out (on RM's blog) that, without making some pretty important assumptions, you can't deduce anything. He/she pointed out a few (I'll make those green) and I'll note all the others ones I think of absolutely necessary to reach any numerical conclusion:


  • 100% of the plastic eggs in the bin are painted red or blue (why would you paint plastic eggs?). There are no other colors of eggs in the barrel (or, if there were, they've been repainted red or blue)
  • Only a blue painted egg is counted as blue, no matter what the original plastic color
  • The eggs painted red and blue are in the bin
  • We're dealing with whole eggs, not fractions of eggs that would be counted as "empty"
If any of the first three aren't true, it can't be solved. If the last part isn't true, one cannot limit the eventual total.

So what? Right? Except we, the people, are constantly bombarded with statistics all the time, oftimes touted as "facts". What we're generally not bombarded with are the caveats and assumptions which go into just about every scientific "fact" (limitations in detection, for example) and, to an even greater degree, all those more nebulous "facts" many treat like absolutes.

"XX% of the YY populace thinks ZZ," for example.

Here are some of the assumptions necessary for that last statement to mean anything:

Whatever sample they used is representative of the whole. Let's face it, chances are this is an estimate using a "sample" of the YY populace since few organizations have the wherewithal to ask any entire population (with the possible exception of say, Polar Bear Club members or something) a question. Chances are, they chose a "representative sample" that might be representative and, just as easily, might be completely unrepresentative of the whole. Let's also note that few organizations have ever asked any sample and received 100% feedback, so it's not just the sample asked but the sample that was willing to reply. I don't know about you, but just thinking those that are willing to answer polls have the same mindset as those who won't bother seems suspect. Geography might make a difference. Social class, race, gender, sexual orientation, political affiliation, those might make differences, too. If the sample is too restrictive or weighted, you won't get a meaningful answer unless that bias is included in the YY descriptor. However, narrowing down the YY descriptor makes the answer more meaningless to everyone else as well.

The wording of the question would be accurate enough to capture the opinion stated (or the thought expressed means what was asked).  Seems obvious but I've been on the receiving end of a few questions that were as misleading and answerless as the famous "So, do you still beat your wife?" question. Asking a mixed political group "Do you think Democrats in Congress/the Senate/your State government have failed to stand up to Republicans enough?" will get you different result than asking "Are Democrats causing our political problems?" but either can be lumped as "disapprove of Democrats." Given how different the questions are and how differently the mitigations are depending which camp you're in, lumping them as the same just shows you how misleading a statistic can be. Or let's try this: "Would you want your child to marry someone of the same gender?" vs. "Do you think people should be able to marry someone of the same gender?" Think they'd get the same answer ("approve of gay marriage")? Before you get too spun up on these intolerant parents, remember that not wanting your child to be gay doesn't mean you wouldn't accept them if they were any more than not wanting your child to be autistic means you'd smother them in their sleep if they were. It's all in how the question is worded which may or may not have anything to do with how the conclusion is worded. But, for the statistic to have meaning, we have to assume they are the equivalent.

People questioned say what they mean. Any time you take a poll, you are also dependent on the respondee to tell you what they actually think in a way that represents what they'll do/think/did. A poll that asks something relatively benign ("Do you like Chinese food?") is more apt to get accuracy than a poll that asks something sensitive ("Have you had sex with someone other than your spouse?") but some people won't answer correctly anyway. Anonymity might buy back some of that, but anyone who thinks a blind poll will garner only untarnished truth when asking "Have you ever had sexual fantasies about a relative?" or "Would you ever engage in premarital sex?" is probably delusional.

 There's more, of course, especially if we move from what seems like easy-peasy ground of what actually is (and we've already seen how misleading statistics can be for that) to predicting things based on trends.

Talk about voodoo.

10 comments:

  • Project Savior
     

    I'm violating my no commenting before my first cup of coffee rule, so maybe my brain isn't up to speed yet but, "30% of eggs containing pearls are painted blue, and 10% of eggs containing nothing are painted blue. What is the probability that a blue egg contains a pearl?" Are the rest, 60%, random along the 40/60 ratio in which case it would be,54%. But if they selected the eggs to paint blue randomly first, then picked the ones to paint out of that sample it would be 40% as the action of picking the eggs later would have no impact on the outcome.

  • Stephanie Barr
     

    I read it that the remainder were 54%. I transformed what I saw the problem meaning into equations so you can readily see what I thought the problem was saying.

  • Roy
     

    Hmmmmm... You lost me on this one; as far as I can see the whole post is written in a foreign language. But my sister understands that language (she's a college math professor and her main focus is statistics), so I'll pass it on to her.

  • Anonymous
     

    Wow... you really like to muddy the waters.  I came to the correct answer very quickly but, then again, I don't bother making things far more complicated than need be.  I'm a firm believer in Occam's Razor.

    Keep in mind, though not always true in real life, I've never seen a "test" question that could not be solved from the information given without one of the possible answers (on a multiple choice test) being "It cannot be solved."  What would be the point of asking trick questions unless that is the lesson itself?

    Thus, the only assumptions that one needs to make is either: (a) "it can be solved from the facts given" or (b) "I'm supposed to explain to the questioner why it might not be solvable at all."  Since there is no evidence supporting the need to make the latter assumption, the former assumption becomes the default winner. 

    This leads to the obvious answer of 12% blue w/pearl vs. 6% blue w/nothing (= 2/1 ratio).  As for minimum quantity of eggs, maintaining whole numbers, 50 eggs should suffice (6 - blue w/pearl; 14 - ~blue/pearl; 3 - blue/nothing; 27 - ~blue/nothing).

    The other (~blue) eggs could be green, yellow, orange or red. The problem merely stated that "some" are painted red, not necessarily all the ones not painted blue are red. Yet, it doesn't matter since the "non-blue" colors play no part in the answer being requested. 

    As for painted vs "already" blue, again, that only comes into play if the questioner is testing your ability to see flaws in the question's design, not whether you know how to perform statistical analysis.  Understanding what is really being tested determines which assumptions are permitted.

    If you'd like a question that is simple to understand but has no obviously "correct" answer, you need only delve into the world of ethics and philosophy (my minor in college):
    Q: you're a train conductor operating track switching in a remote mountain office overlooking the tracks; your train, filled with 300 passengers, is heading for a junction and you need to switch it to a different track since the normal one has malfunctioned; if you do not switch tracks, the train will derail off a cliff and all passengers will die; just before you switch tracks you see that your 5-year-old child, who was visiting you, has run off and somehow got caught at the track junction; if you switch the track, your child will be painfully crushed to death by the train; there is not enough time to save your child before the train arrives and the track needs to be switched.  What action do you take?

    (Open book test and calculators are permitted.  Good luck.)

    Mike H.

  • Stephanie Barr
     

    Mike, when I first looked at this, I saw it at face value, too. I have no trouble translating the obvious intent into equations and moving forward in math tests.

    However, I didn't think Soubriquet's point was without merit because, in real life, things are often not black and white. And, when you start translating real life into math, you better not lose sight of the assumptions you made and the effect on your "facts."

    Titanic was made from some of the best iron in the world at the time, but the properties changed drastically (catastrophically)in cold temperatures (like a below freezing Atlantic), as deadly as the assumption that no more than four compartments could be breached at one time or that bulkheads that didn't reach the top would be sufficient. Aircraft accidents and space accidents are an excellent lesson book on the dangers of not understanding your assumptions, thinking hardware will work the same no matter what the temperatures, pressure, lighting, toxicology, gravity - all kinds of things.

    Math problems make it all look so easy, too easy. And the answer is frequently immutable and absolutely true. In reality, however, measurements have tolerances that can stack up, contamination can trigger ignition, clever design solutions can actually acerbate a problem by changing one variable, often a variable no one was even looking at or knew existed.

    And that's why this isn't just me being a difficult person, but pointing out that facts are only as true as the assumptions they're built on.

  • Anonymous
     

    I do not disagree with you.  I was pointing out that this was presented as a test question relating to probability analysis, not an engineering question asking whether something is possible or if the assumptions might change under differing conditions.

    You do bring up a good point in that very little of what I learned in school ever questioned the facts given on tests.  Think how much better prepared students would be for life's true tests if the students were required to analyze the questions themselves as well as performing rote calculations based on them?  We go through life believing 2+2=4 when in fact it might not in certain circumstances (base 3 or 4, for example).  Seeing beyond the obvious is what leads to the great discoveries that make life better or protect us from preventable tragedies.

    It is the grey areas in life that many are unprepared for because they weren't taught to be skeptical of what is told to them, to read between the lines.  This translates to political discourse, personal relations, religion, and any number of subject areas.

    I think you have a natural gift for questioning things but many do not and could benefit from better education, at early ages, in critical analysis. Which leads to your post's discussion of using statistics as a proxy for facts.

    Mike H.

  • Relax Max
     

    Whoa. This makes me really glad I decided to become a hunter-gatherer instead of a statistician. Much of what you say is almost certainly true. I'd say at least 90/10 probability.

  • soubriquet
     

    Stephanie, Thank you for your support. I realise, absolutely, that the question as posed, expected the reader to make a certain set of assumptions or presumptions in order to answer it in the chosen manner. My early training, whether by my grandfather, or by various school teachers and college tutors, was not to make assumptions.

    "never assume," Mr Holdsworth would intone, in third-year physics, "because to Ass-Ume is to make an Ass out of U and Me!" Hardly original, I know, but it's stuck with me ever since.
    "If you don't understand the question, then read it more closely, if you still don't understand, then ask for clarification."

    Those concepts are ingrained in me. And although I saw this question first on a blog headlined "Clarity2010", (oops... I go to http://clarity2010.blogspot.com/, but it's changed its header to 'The Mind of Max'-maybe Max is giving up his search for Clarity?), in the form in which it was posted I could see a number of flaws which, to me, mean that the question is unanswerable, unless you preface your answer with a disclaimer to the effect that you will assume the questioner meant that there are ONLY eggs painted red and eggs painted blue in the barrel.

    Oh my. I've just come up with the perfect answer, proving all the other wrong.

    Because you all made an assumption.
    Never assume, because....
    The answer is not anything that any other attempt has revealed. As Mr Holdsworth would say "Study the question".

    We have a given, that the barrel contains blue painted eggs, and that 30% of the blue painted eggs in the barrel contain a pearl.
    The question does not ask what you assumed, "What is the probability that ANY PARTICULAR blue egg contains a pearl?"
    The question asked was "What is the probability that A BLUE EGG contains a pearl?".
    It's already told us that there's at least ONE blue egg containing a pearl.

    The answer, therefore, is 100%.

  • Frank Hark
     

    Hahaha, Well soubriquet. This reminds me of a great discussion of the fallacy of "common sense" as described in the book “Apollo Root Cause Analysis” (which in truth I bought from Amazon because it was cheap and I thought it was about the Apollo program). Basically we misuse the term common sense a lot. Just from these discussions you can see how everyone has a unique perspective.
    Thanks everyone.

  • soubriquet
     

    "common sense" is indeed a misnomer.
    I think by the time you leave the realms of toddlerdom, that you're realising that what seems abundantly self evident to one person is shrouded in mists of bafflement for another. And as we grow older, it gets no easier.

    However, I'm wondering whether you think my answer is correct, or is it perceived simply as a blatant attempt to avoid the math?

    Having just read a little extract from "Apollo Root Cause Analysis", I rather think it would support my approach.

Post a Comment

Labels

Blog Makeover by LadyJava Creations