Mexico City       These days almost all of us depend on data in one way or another to figure out the wild world around us from politics, to economics, to health, and on to the smallest things about how our neighbors, countrymen, and brothers and sisters around the world are thinking about everything under the sun.  A lot of the data that drives our decisions and opinions, if we tell the truth, comes from the polls we read, especially those that we choose to believe.  That’s the way it is, but there’s trouble everywhere around us:  the numbers may suck.  We’ve seen this happening in the topsy-turvy predictions around recent elections, but given how co-dependent we’ve become on polls and data, we’ve tried to believe the pollsters protestations that they “fixed” this problem or that, and the numbers will be better, or they have aggregated them all together so we can depend on the “true facts” being somewhere in the middle.

            Josh Zumbrun, who writes the “Numbers” column for the Washington Post announced that he’s going on paternal leave, but before he checks on at the end of the year, he wanted to drop a bomb on how shaky the underlying numbers are on a lot of high-value, even governmental, data our lives depend on, because polling response rates are in the toilet.  For starters, he says,

The White House Office of Management and Budget once articulated a standard that survey response rates should be above 80%. Today, nearly no surveys remain above that standard.

Then he reports that the monthly jobs report, which has been Biden’s saving grace, has a response that is now down to 71%.  It gets worse.  Employment statistics are down to 42%.  Job Openings and Labor Turnover is now 32%.  CPI data for housing is 58% and commodities is 54%.  You get the picture.  We’re driving the economy and a lot more in the dark.

            Why?  People don’t answer the phone.  We’re all in that club.  If we don’t recognize the number or it’s an 800, we know darned well that it’s spam, so we let it die without a pickup.  Zumbrun quotes Pew Research on opinion polls, saying:

Consider opinion polling. In 2000, over 90% of national polls relied on randomly calling people on the phone, according to the Pew Research Center. As recently as 2014, a narrow majority still operated this way. By 2022, fewer than 9%, or just six out of 69 organizations, still polled this way, Pew found.

In short, they don’t bother calling anymore and instead use focus groups, supposedly made up of the average Joe’s and Jane’s, if such animals really exist anymore, who withstand a barrage of questions and then extrapolate their views to stand for everyone else.  They are pros for sure, even if some seem to have their fingers on the scale to support the bias of whoever is paying them, but it’s hardly random, which has to be worrying, when so much depends on their accuracy.

            There are other problems.  Artificial intelligence is not ready for prime-time.  Satellite images and info are telling us a lot, but these are all supplements, not replacements.  The pandemic threw a wrench in the works when pollsters and data crunchers can cherry-pick the time periods for measurements and comparisons. 

            Zumbrun does this for a living, and even though on his way to pat leave, certainly wants to keep a good gig when he comes back, so he tries to paint lipstick on this pig saying the future might be better, even if the coming year won’t prove it.  For the rest of us, there’s a big flashing yellow warning sign that we need to put on any polls or data coming our way and look under the lids very carefully, before we start believing we are on solid ground.