Battling the Tyranny of Big Data

The data scientists writing the algorithms that drive giants like Alphabet Inc. (Google) and Facebook Inc. are today's technology wizards, and companies and governments increasingly use their creations -- often in secret and with little oversight -- to do everything from hiring and firing employees to identifying likely suspects for police monitoring. But there's a dark side -- and computer scientists warn that we'll need a lot more transparency if the big-data revolution is really to work for all of us.

In her recent book "Weapons of Math Destruction," mathematician Cathy O'Neil tells the story of Sarah Wysocki, a teacher fired from her job at MacFarland Middle School in Washington, after a computer algorithm churning through numbers on student performance judged her to be a poor teacher. Both students and parents consistently ranked Wysocki as an excellent teacher, yet she couldn't fairly challenge the decision because the company that developed the algorithm claimed a right to proprietary secrecy. Her firing stood despite near certainty that the algorithm, with the limited data it analyzed, couldn't have reached any statistically meaningful conclusion.