Civis Analytics’s Dan Wagner joins our experts discussing the untapped potential of data analysis in medicine, education, and elsewhere, along with the pitfalls that may lie ahead.
You’ve said you want to solve the world’s biggest problems using big data. What’s top on that list?
We’re focused on two areas: education and health. On education we’re focused on using individual-level data to help clients identify low-income students who are applying and enrolling in colleges that are beneath their potential and helping those organizations identify means to get those kids to go to colleges that are actually equivalent to their potential.
Insurance enrollment is a target, especially in the first few months of implementation of the Affordable Care Act. We are focused on working with a number of organizations to help them identify and enroll people without health insurance. It is a very difficult exercise because there is no list of the uninsured. You have to do it only through statistical inference.
One of the most interesting questions is how insurance is evolving from group policies to individual policies, and how insurance companies learn to manage that risk. We are working with several organizations to use data to proactively identify individual risk for, say, cardiovascular disease, to have an early understanding of patient risk.
And once you’ve identified risk, you ask that person to pay more?
You can’t do that. You can only price policies based on a reduced set of variables, such as age. So you can’t price on that risk, but you need to manage it.
You help organizations deal with new ways of analyzing data. Where do they have trouble?
Apart from the technical hurdles, they need to transfer from thinking about people as these kind of bulk segments toward thinking about people as individuals with their own propensities, their own risks that need to be managed carefully with individually deployable communication to help them have better lives. Frankly, it’s a very difficult transition, both from a perspective of technology and from a cultural perspective.
How did that play out when you were working on the Obama campaign?
One of the biggest changes that we made was in media. Typically, the way to follow it is by Nielsen (NLSN) ratings. So you kind of look at demographics like Hispanics or young people or women. What we did is we defined our audience based on our individual-level persuadability scores. We matched those up to set-top box data. From there we were able to identify the top television programs that had the highest density of our persuadable viewers per dollar. With that, we were then essentially able to prioritize our spending based on a single metric. It didn’t have anything to do with demographics. It just had to do with our specific audience, which had been identified at the individual level. It was pretty difficult, but culturally our organization was fit to do it because, you know, in kind of the classic Obama fashion, people were pretty wonky about that stuff.
How does security fit into the equation?
You have to be obsessive about it. A lot of these organizations are collecting information, but I don’t think many of them are aware of the highest standard data security practices. A lot of our work is done on the Amazon (AMZN) cloud, which is nice because you get to leverage a lot of the protocols that Amazon offers, versus something that you could develop internally. What Amazon provides is essentially best in class.
What is the biggest thing that we’re getting wrong or overlooking in the hype around big data?
The unfortunate thing about the hype is that people are thinking about the process, not about the outcomes. Sometimes the hype is misplaced, in a sense that it feeds this belief in the growth of data as a solution in and of itself. The big question is, what are you going to do with that information that’s actually going to improve people’s lives?
How do you handle information overload in your personal life?
As someone who works at a company that is involved in the Internet, I spend quite a lot of time there. But I try to stay electronically efficient with all my different communication and also try and make sure that I read a lot of books.
For more Fix This/Big Data, visit: www.businessweek.com/fix-this/big-data.