U.S. Government Invites Silicon Valley Invasion to Harness Databy
Commerce recruiting about two dozen in-house engineers
Paypal, Amazon among partners for government data projects
Ian Kalin wants you -- if you’re an information junkie who enjoys working in an environment akin to a Silicon Valley technology startup.
The Commerce Department’s first chief data officer is looking to hire engineers like himself who will team up with private companies to bring the government’s collection of facts and figures into the 21st century, the agency announced Monday. The aim is to help businesses grow and allow consumers to make better spending and saving decisions.
Some $4 million in seed money, which was leftover from each of the Commerce Department’s 12 agencies, will eventually be used to hire around two dozen people, making the funding of this project feel more like a West Coast technology venture. After his own stints at companies such as Google Inc. and software provider Socrata Inc., Kalin’s no stranger to the skeptics, some of whom are the very people he’s trying to recruit.
“Some of them are like, ‘Wait, you want me to do what for government? Is that even possible? Am I going to be set up for success here?”’ Kalin said in an interview. “And that’s a perfectly fair question for anyone who’s engaged with the federal government and used to Silicon Valley.”
The department now has a handful of projects with partners such as Palantir Technologies Inc., PayPal Holdings Inc. and Amazon.com Inc. Among their top priorities: using credit-card transactions to better understand consumer-spending patterns, analyzing Internet shopping trends to direct U.S. exporters toward untapped overseas markets and making trillions of bits of weather data more accessible to the public.
Eventually, the initiatives could enhance the government’s regular reports on economic indicators such as gross domestic product, trade or retail sales.
Amazon Web Services, a unit of the world’s biggest Internet-commerce company, is making the reams of data from the National Oceanic and Atmospheric Administration, one of Commerce’s bureaus, easier for consumers to digest. It’s also undertaken similar projects, including teaming with NASA and the U.S. Geological Survey to provide global satellite imagery and mineral-deposit information.
The long process to alter how economic indicators such as GDP are tallied remains “a long ways off,” Kalin said. Revisions to GDP and another of the Department’s indicators, retail sales, show why the agency should see room for improvement.
In 10 of the 12 years ending in 2014, the Commerce Department initially overestimated annual GDP rates compared to the most-recent data, according to calculations provided by the Bureau of Economic Analysis. The latest figures are the result of several revisions as more information is available. In 2014 and 2012, the government’s first try matched their latest estimates.
Similarly, for every year from 2004 through 2013, the Department overshot annual retail sales totals in their first report compared with the current tallies, according to data compiled by Bloomberg. The first estimates over-counted purchases by between 1.2 percent and 5.2 percent during that period.
The release of massive amounts of government data, and the partnerships with private industry to aggregate it, probably will meet some public criticism. Data-privacy advocates see some red flags in the initiatives.
“It’s always a great thing to see government try to bring in commercial expertise to accelerate their processes,” but you have to look at these efforts with “a bit of layered concern,” said Timothy Yim, director of data security and privacy at Startup Policy Lab in San Francisco, a non-profit group that connects government officials with startup organizations.
When government agencies and businesses share, there’s always a risk that one might unintentionally include sensitive information, said Yim. With government surveillance a hot topic these days, companies have had to redouble efforts to convince customers that their information is safe from leaks.
Kalin is adamant about keeping those holes plugged.
“Data privacy is constantly on our minds,” Kalin said in an e-mail. “We’re currently only working with aggregate data that contains no personally identifiable information” and “almost all” of the data used is already public, he said.
The Department has started its search for the new data team, and Kalin envisions the positions as similar to fellowships, or “tours of duty in public service” that might be as short as one year -- meant to fit with the job-hopping tendencies of the tech experts being recruited.
Kalin draws on his scientific instincts in trusting that in the end, the partnership makes sense.
“Government has a lot of mass, weight, inertia and size to it, and Silicon Valley has a lot of velocity. Everything’s fast -- fast to hire, fast market,” he said. “Mathematically, if you take a high mass times a high velocity, you have a higher impact.”