Secrecy News

CIA Halts Public Access to Open Source Service

For more than half a century, the public has been able to access a wealth of information collected by U.S. intelligence from unclassified, open sources around the world.  At the end of this year, the Central Intelligence Agency will terminate that access.

The U.S. intelligence community’s Open Source Center (OSC), which is managed by the CIA, will cease to provide its information feed to the publicly accessible World News Connection as of December 31, 2013, according to an announcement from the National Technical Information Service (NTIS), which operates the World News Connection (WNC).

The WNC “is an online news service, only accessible via the World Wide Web, that offers an extensive array of translated and English-language news and information,” an NTIS brochure explains. “Particularly effective in its coverage of local media sources, WNC provides you with the power to identify what really is happening in a specific country or region. Compiled from thousands of non-U.S. media sources, the information in WNC covers significant socioeconomic, political, scientific, technical, and environmental issues and events.”

“The information is obtained from full text and summaries of newspaper articles, conference proceedings, television and radio broadcasts, periodicals, and non-classified technical reports. New information is entered into WNC every government business day. Generally, new information is available within 48-72 hours from the time of original publication or broadcast.”

“For over 60 years, analysts from OSC’s domestic and overseas bureaus have monitored timely and pertinent open-source materials, including grey literature. Uniquely, WNC allows you to take advantage of the intelligence gathering experience of OSC,” the NTIS brochure says. Soon, that will no longer be true.

The WNC public feed from the Open Source Center is a highly attenuated version of what is available to official government users.  Within government, copyright considerations are ignored, but for public distribution they must be respected, and so (with some exceptions) only information products whose creators have signed a royalty agreement with NTIS are publicly released.

Even with that significant limitation and the attendant public subscription fees, the NTIS World News Connection has remained a highly prized resource for news reporters, foreign policy analysts, students and interested members of the public.

I check it almost every day.  Recently, for example, I have been following official statements from Russian officials who allege that the U.S. is covertly developing biological weapons for use against Russia in a military laboratory in the Republic of Georgia. The claim seems bizarre, but may nevertheless be politically significant.  Detailed English-language coverage of the matter, or of many other stories of regional interest and importance, is not readily available elsewhere.  (Moreso than in the past, however, portions of the material that is publicly accessible through WNC can be obtained elsewhere, through other news services or foreign websites.)

The reasons for the decision to terminate the World News Connection are a bit obscure.  Producing it is not a drain on U.S. intelligence– the marginal costs of providing the additional feed to NTIS are close to zero.  (The total budget for open source intelligence was about $384 million in FY2012, according to classified budget records obtained by the Washington Post from Edward Snowden.)  However, the program is a headache for NTIS to manage, particularly since NTIS officials had to negotiate numerous contracts with media source providers to offer their products to the public.  But the large majority of that work has already been accomplished, and now it will be rendered useless.

Mary Webster of the Open Source Center had initially proposed to cancel the public information feed as of September 30, according to an NTIS official.  Then she was persuaded to grant a six month reprieve.  But in the end, a cut-off date of December 31, 2013 was set.

If that comes to pass, it will be a blow to researchers and proponents of public intelligence. The Federation of American Scientists had previously argued that the U.S. government should actually expand public access to open source intelligence by publishing all unclassified, uncopyrighted Open Source Center products.  (“Open Up Open Source Intelligence,” Secrecy News, August 24, 2011.)  Instead, even the current range of publications will no longer be systematically released.  (Only a small fraction of publicly unreleased OSC records ever seem to leak.)

Although the Open Source Center is managed by the Central Intelligence Agency, it is formally a component of the Office of the Director of National Intelligence.  Yet the move the terminate public access to OSC products seemed to catch the ODNI unawares.

“Obviously our attention is on a possible lapse in appropriations, but we are looking into this,” said an ODNI spokesman on September 30, just before the government shutdown.

“The information provided through NTIS makes an irreplaceable contribution to U.S. national security,” wrote Prof. Gary G. Sick of Columbia University in an October 1999 letter, in response to a previous proposal to curtail coverage in the World News Connection.

The World News Connection “informs us about other countries in ways that otherwise would be nearly impossible,” Dr. Sick wrote. “It costs virtually nothing in comparison with almost any other national security system. It is not as sexy as a bomber or a missile, but its contributions to national security can be attested to by generations of policy-makers. I was in the White House during the Iranian revolution and the hostage crisis, and my respect for the power of this information was born at that time. I often found it more helpful than the reams of classified material that came across my desk at the NSC.”

8 thoughts on “CIA Halts Public Access to Open Source Service

  1. Is that old chestnut contending the Central Intelligence Agency (CIA) works for the President merely a meme? The assertion that the President is Commander-in-Chief of the Armed Forces has turned out to be a meme, since he apparently hasn’t sufficient authority to close a military installation. “Commander-in-Chief” is sounding more like “Kentucky Colonel” all the time!

    But if the CIA really does answer to the President, can he not order it to order Mary Webster to keep the Open Source Center open? If the CIA manages that Center, then it manages Ms Webster as well. Or would a couple of telephone calls from the Oval Office be too much bother for the President? In the event, I thank Steven Aftergood for yet another timely report.

  2. Isn’t this more of the same? After all, the MSM is all but neutered as far as the news is concerned. Oh, local fender benders are still available, but the Investigating journalists are from another by-gone era. Why let the people know what’s going on in the world, keep them ignorant, just like the NSA has achieved.

  3. Thanks for publicizing this. As an American student of international relations, I would be at a loss of an important public resource. What were the official reasons given for this? Where (or who) may we reach best for official comment regarding this?

  4. The complaint here is that the CIA is not providing free access for journalists and scholars to open source materials the CIA translates as part of their intelligence work. The rationale for the CIA to stop doing it is that navigating copyright law is a pain in the ass. There is a different copyright threshold for internal government use (I use the full OSC products every day so I’m familiar with what they do and how useful they are).

    Key sentence: “However, the program is a headache for NTIS to manage, particularly since NTIS officials had to negotiate numerous contracts with media source providers to offer their products to the public.”

    This is the era of the internet and automatic translation programs. If academics want free access to translations of foreign copyrighted materials they can use Google or mess with hiring translators and dealing with copyright law themselves. These dislike the CIA anyway, now they want to complain because they don’t want to pay for the services they scam off the agency they hate? I think they might want to rethink their complaint. But that’s just me :)

    Note: Thanks for the comment. I should have made it more clear that the World News Connection is not a free service. It is offered by NTIS for subscription at a fairly hefty fee. –Steven Aftergood

  5. As a scholar who publishes regularly on Asian international politics, the FBIS Open Source Center has been my primary research source for decades. The utility to the government is that it provides additional points of view to those available inside the government and additional assessments to government analysts who frequently don’t have the time to parse open source information. To the public, its availability enhances the knowledge of the “informed public”–essential to an effective democracy. I am appalled that the WNC is being shut down for reasons of bureaucratic convenience. “Penny wise and pound foolish.”

  6. I believe this is more of a problem with NTIS, rather than the CIA, which seems to have operated as a “profit center” since the early 1980′s. Around the mid-1990′s the General Management Staff of the CIA wanted to make the electronic format of the Foreign Broadcast Inforamtion Service/Joint Publications Research Service (FBIS/JPRS) products available on Elsevier’s NEXIS database. Part of the idea, I think, was that the intelligence community would use NEXIS to access the data. I argued against this approach for a couple of reasons. One, FBIS and JPRS reports were already available to the public for free through the Government Printing Office Depository Library Program. Would that go away? Two, I didn’t like the idea of taking information collected by the government at taxpayer expense and being “given” to a private company for resale at premium prices. And, three, what about copyright and For Official Use Only (FOUO) issues?

    Ultimately, the Open Source Center was created and the Agency developed its own restricted system for the Intelligence Community and peddled off the UNCLASSIFIED/UNLIMITED to the World News Connection which I’ve seen available on DIALOG Information Services, commercially, and I suspect, directly from NTIS to academia where I tested it at the College of William and Mary (my opinion, the search engine and the result displays were poor especially compared to the restricted version). Still, if this goes away, will the GPO Library Depository Program still receive UNCLASSIFIED/UNLIMITED Open Source Center material?

  7. This is a huge mistake. I have long been dependent on the WNC for materials that I have found crucial to my research and that of my students at Johns Hopkins University. This decision must be reversed.

    Robert O. Freedman

    Johns Hopkins University

  8. As a former employee for 34 years standing of this esteemed organization, I find it very sad that the use of such hard-earned information is going to be cut off from the public. Those researchers who to some extent have always depended on the timeliness and accuracy of the products provided by OSC must be very saddened and, to some extent, I would sense aggrieved by this move.

    The information provided by OSC, formerly FBIS, has always been based on unclassified, open source media. However, with OSC moving more and more into the realms of social media to garner its information, then I am sure that the situation becomes more tricky.

    OSC must continue to allow its product to be read by the public and by researchers alike. It is vital that the truth is available to all.

Leave a Reply