CIA Puts History Out in Open With Millions of Online Records

  • Agency releases 12 million pages on wars, reconaissance
  • Officials look to artificial intelligence to review documents

The Central Intelligence Agency is putting its clandestine history out in the open with a declassified trove of 12 million pages that’s available online.

Documents covering the agency’s work from the 1940s through 1990s were previously accessible -- but only by visiting the National Archives in College Park, Maryland. The CIA is required to declassify most records that are 25 years or older.

“This is one of the things that we think improves transparency for us, and it’s a simple thing” to make information “more widely available,” Joseph Lambert, the CIA’s director of information management, said in a phone interview.

Known as the CIA Records Search Tool, or CREST, the collection includes reports on policy and intelligence operations, with topics from the Cold War and Vietnam to terrorism and global economics. The agency continues to review documents for declassification, but the number of records to be assessed is growing rapidly, beyond the realm of humans “being able to scale to that kind of volume,” Lambert said.

Lambert said the agency is looking at emerging technology, including artificial intelligence and machine-learning tools, to help process the staggering amount of documents officials must pore through. That requires judging whether releasing the content would harm national security and redacting certain sensitive names and details.

“Human beings can only do so many pages,” Lambert said. “It’s a difficult endeavor to make sure that you can put together the right technologies to assist a human being going forward to scale to hundreds of millions of pages.”

    Before it's here, it's on the Bloomberg Terminal.