It is to be expected that national intelligence services will sometimes fail to identify and discover a threat to the nation in a timely fashion.  But when intelligence warns of a threat that isn’t really there, and then nations go to war to meet the phantom threat — that is a serious, confounding and deeply disturbing problem.

But in a nutshell, that is the story of the war in Iraq, in which the U.S. and its allies attacked Saddam Hussein’s Iraq because of the supposedly imminent threat posed by Saddam’s stockpile of weapons of mass destruction — a threat that proved illusory.

A new book published in the United Kingdom called “Failing Intelligence” provides a remarkable account of the British experience of how intelligence on the Iraqi WMD program was shaped and packaged to support the decision to go to war in Iraq.  The book’s author, Brian Jones, was the chief specialist in weapons of mass destruction on the UK Defence Intelligence Staff.  He was also a skeptic of the stronger claims made about the existence of Iraqi WMD stockpiles.  The book documents his mostly unsuccessful attempts to register that skepticism, to moderate the extreme claims made by government officials, and later to hold those officials accountable for their actions.

He provides a detailed first-hand account of how his efforts were consistently deflected in the rush to war, and how intelligence declined into propaganda.  It’s a grim but instructive case study in the overlapping failure of intelligence gathering, intelligence production, and intelligence oversight.

The National Security Archive has recently published three richly informative collections of declassified U.S. and British government documents on the lead-up to the Iraq war (including several key documents cited or relied upon by Brian Jones).

“The more deeply the processes of creating the government reports on the alleged Iraqi threat are reconstructed — on both sides of the Atlantic — the more their products are revealed as explicitly aimed at building a basis for war,” wrote John Prados of the National Security Archive and journalist Christopher Ames in an analysis of the documents.

“In the light of a decision process in which no serious consideration was given to any course other than war, the question of whether American and British leaders set out to wage aggressive war has to be squarely faced,” they wrote.

The Intelligence Science Board (ISB), which was established in 2002 to provide independent scientific advice to the Director of National Intelligence, has been disbanded by the new DNI, James R. Clapper Jr., as part of a process of reorganizing and streamlining the ODNI organization.

“My understanding is that the Director will be disbanding all 20 of his advisory boards, which includes the ISB,” one participant in Intelligence Science Board studies told Secrecy News.

DNI Clapper “did a zero base review of all outside advisory boards as part of an efficiency review,” an ODNI official said.  “The new strategy is to have one Senior Advisory Group and then convene Task Forces on specific issues as needed.”  The Task Forces in turn “may have expiration dates.”  The membership of the new umbrella Advisory Group is now in formation, the official said.

The overall contribution of the Intelligence Science Board is difficult for an outsider to assess, since little of its work has been made public.  But the Board’s 2006 report on “Educing Information” (pdf), which authoritatively explained that there was no empirical justification for the use of coercive interrogation (or torture), remains a milestone in the field.  It demonstrated independent judgment as well as immediate policy relevance.

An ODNI spokesman said the move to eliminate the Board should not be seen as a rejection of science advice, but as a step toward a smaller standing bureaucracy and increased efficiency.

“One of the things I’m doing is… essentially restructuring the Office of the Director of National Intelligence” said DNI Clapper at an October 6 speech (pdf) to the Bipartisan Policy Center.

“The Soviet Army is the best prepared force in the world to conduct both offensive and defensive NBC [nuclear, biological and chemical] operations,” according to a 1984 U.S. Army manual (large pdf) that is newly available online.

The three-part manual, based on Soviet military literature and other open sources, provides a dauntingly detailed account of almost every aspect of Soviet military structure and operations.

So, for example: “The Soviets recognize three basic types of smoke screens: blinding, camouflaging, and decoy. Each type is classified as being frontal, oblique, or flank in nature, depending on the placement of the screen.”

Perhaps of equal or greater importance, the manual implicitly documents the U.S. Army’s perception of the Soviet military late in the Cold War.

“In the Soviet view, the correlation of forces has been shifting in favor of the socialist camp since the Soviet defeat of Nazi Germany in World War II. Soviet Marxist-Leninist ideology requires the correlation to shift continuously in favor of socialism. The correlation of forces may be advanced by both violent and nonviolent means. When it is advanced by violent means, the military component of the correlation is the dominant factor.”

The first volume of the manual, originally “for official Government use only,” has not previously been published online.  See “The Soviet Army: Operations and Tactics,” Field Manual 100-2-1, July 16, 1984 (203 pages, large pdf).

The second volume is “The Soviet Army: Specialized Warfare and Rear Area Support,” FM 100-2-2, July 16, 1984 (100 pages, pdf).

The third volume is “The Soviet Army: Troops, Organization, and Equipment,” FM 100-2-3, June 1991 (456 pages, large pdf).

In February 1957, the Army produced an extremely detailed “Glossary of Soviet Military and Related Abbreviations” (pdf), Army Technical Manual TM 30-546.

After a court issued a ruling last spring that a Yemeni detainee held in U.S. custody should be released, the opinion was briefly published in the case docket and then abruptly withdrawn for classification review.  When it reappeared, reporter Dafna Linzer discovered, it was not only redacted but had been significantly altered.

“The alterations are extensive,” she found. “Sentences were rewritten.  Footnotes that described disputes and discrepancies in the government’s case were deleted.  Even the date and circumstances of [the detainee’s] arrest were changed.”

Yet in what seems like an insult to the integrity of the judicial process, no indication was given that the original opinion had been modified — not just censored — as a consequence of the classification review.  ProPublica obtained both versions of the ruling and published a comparison of them, highlighting the missing or altered passages.  See “In Gitmo Opinion, Two Versions of Reality” by Dafna Linzer, ProPublica (co-published with The National Law Journal), October 8.

Two histories of the early decades of MI-6, the United Kingdom’s foreign intelligence service, have recently been published.  “MI6: The History of the Secret Intelligence Service 1909-1949” by Keith Jeffery is the authorized version, prepared with the cooperation of the Service.  “Six: A History of Britain’s Secret Intelligence Service” by Michael Smith is the unauthorized version.

Close students of intelligence history will want to read both volumes, which neatly represent the respective virtues of authorized and unauthorized history.  As the authorized historian, Jeffery enjoyed privileged access to classified Service archives that no other writer is likely to obtain for years to come.  But he was also subject to official restrictions on what he was permitted to publish. So, for example, he could not identify any agents who had not already been publicly identified nor could he tell their stories if doing so would result in their identification.

“Six,” the unauthorized history by veteran intelligence reporter Michael Smith, ranges more widely (though it ends a decade earlier in 1939), taps into foreign archives and private, non-governmental collections, and is subject to no such prior restrictions on disclosure.

The tales of the Service’s early years, now nearly a century old, are vividly told by author Smith, whose book is full of striking observations and asides.  Trainspotting in World War I and the early confrontation with Soviet intelligence, among other topics, are treated in this volume, which ends at the dawn of World War II.  “Six” has not yet been published in the U.S. but is available from Amazon.com in the UK.

Noteworthy new reports from the Congressional Research Service that have not been made readily to the public include the following (all pdf).

“North Korea: Legislative Basis for U.S. Economic Sanctions,” September 29, 2010.

“China’s Sovereign Wealth Fund: Developments and Policy Implications,” September 23, 2010.

“Defense: FY2011 Authorization and Appropriations,” September 17, 2010.

“Federal Conspiracy Law: A Brief Overview,” April 30, 2010.

A White House report to Congress (pdf) last week assessed “both positive and negative trends in the implementation of our Afghanistan and Pakistan strategy.”

The report described the progress — or lack thereof — made this year towards achieving eight specified objectives. Those objectives include enhancing stability and civilian control in Pakistan, improving Pakistan’s counterinsurgency capabilities, and reversing the Taliban’s momentum in Afghanistan, among others. (The disruption of terrorist networks in Afghanistan and Pakistan was addressed in an undisclosed classified annex.)

The report was quite candid in its judgments. “Afghan anti-corruption efforts continue to be weak.” The security situation in Pakistan is “tenuous.” The Pakistani military has demonstrated an “inability” to maintain control of areas it seized from insurgents. During the second quarter of this year, “the Pakistan military continued to avoid military engagements that would put it in direct conflict with Afghan Taliban or al-Qa’ida forces in North Waziristan. This is as much a political choice as it is a reflection of an under-resourced military prioritizing its targets.”

Paradoxically, there is something encouraging about the new report since it indicates that U.S. government officials are not actively deluding themselves or deceiving others about the difficult realities of the conflict in Afghanistan and Pakistan.

The White House report to Congress was first reported in the Wall Street Journal (“U.S. Slams Pakistani Efforts on Militants” by Adam Entous and Siobhan Gorman, October 6) but the document itself was not made readily available to the public. A copy is now posted on the Federation of American Scientists website here.

A U.S. Army regulation (pdf) issued yesterday requires Army personnel to report any incident on a newly expanded list of possible indications of “espionage, international terrorism, sabotage, subversion” as well as “leaks to the media.”

“The Army is a prime target for foreign intelligence and international terrorist elements… from within and OCONUS [outside the continental United States],” the regulation states. “The Army also faces threats from persons on the inside… who may compromise the ability of the organization to accomplish its mission through espionage, acts of terrorism, support to international terrorist organizations, or unauthorized release or disclosure of classified or sensitive information.”

The regulation presents an extensive description of suspicious behaviors that are reportable to authorities, including “attempts to expand access to classified information by repeatedly volunteering for assignments or duties beyond the normal scope of responsibilities.”

It also provides guidance on how to respond to the discovery of a clandestine surveillance device (“do not disturb the device”) or an approach by a foreign intelligence officer (“remain noncommittal, neither refusing nor agreeing to cooperate”;  also, “do not, under any circumstances, conduct your own investigation”).

The regulation includes tables listing behavior that may be exhibited by a person engaged in espionage, indicators of insider threats of terrorism, and signs of extremist activity that may pose a threat to U.S. military facilities or operations. See “Threat Awareness and Reporting Program,” U.S. Army Regulation 381-12, October 4, 2010.  (The prior edition of this regulation, formerly titled “Subversion and Espionage Directed Against the U.S. Army (SAEDA)” and dated January 15, 1993, is available here.)

The Pentagon’s heavy-handed attempt to censor the new Afghanistan war memoir “Operation Dark Heart” by Anthony Shaffer has predictably turned a volume of narrow, specialized interest into a mainstream bestseller.

It has also focused attention on just what information the government was seeking to conceal, and why.  For a review of the material that was blacked out in the second edition of the book, see “Censored book masks sensitive operations” by Sean D. Naylor, Army Times, October 4.  A side-by-side view of the book’s Index, in censored and uncensored formats, is here (pdf).

Noteworthy new reports from the Congressional Research Service include the following (all pdf).

“The Helium-3 Shortage: Supply, Demand, and Options for Congress,” September 21, 2010.

“China’s Steel Industry and Its Impact on the United States: Issues for Congress,” September 21, 2010.

“Authority of State and Local Police to Enforce Federal Immigration Law,” September 17, 2010.

“Statutory Damage Awards in Peer-to-Peer File Sharing Cases Involving Copyrighted Sound Recordings: Recent Legal Developments,” September 16, 2010.

By censoring Anthony Shaffer’s new book “Operation Dark Heart” even though uncensored review copies are already available in the public domain, the Department of Defense has produced a genuinely unique product:  a revealing snapshot of the way that the Obama Administration classifies national security information in 2010.

With both versions before them (excerpts), readers can see for themselves exactly what the Pentagon classifiers wanted to withhold, and can judge for themselves whether the secrecy they tried to impose can be justified on valid national security grounds.  In the majority of instances, the results of such an inspection seem disappointing, if not very surprising, and they tend to confirm the most skeptical view of the operation of the classification system.

The most commonly repeated “redaction” in Operation Dark Heart is the author’s cover name, “Christopher Stryker,” that he used while serving in Afghanistan.  Probably the second most common redactions are references to the National Security Agency, its heaquarters location at Fort Meade, Maryland, the familiar abbreviation SIGINT (referring to “signals intelligence”), and offhand remarks like “Guys on phones were always great sources of intel,” which is blacked out on the bottom of page 56.

Also frequently redacted are mentions of the term TAREX or “Target Exploitation,” referring to intelligence collection gathered at a sensitive site, and all references to low-profile organizations such as the Air Force Special Activities Center and the Joint Special Operations Command, as well as to foreign intelligence partners such as New Zealand.  Task Force 121 gets renamed Task Force 1099.  The code name Copper Green, referring to an “enhanced” interrogation program, is deleted.

Perhaps 10% of the redacted passages do have some conceivable security sensitivity, including the identity of the CIA chief of station in Kabul, who has been renamed “Jacob Walker” in the new version, and a physical description of the location and appearance of the CIA station itself, which has been censored.

Many other redactions are extremely tenuous.  The name of character actor Ned Beatty is not properly classified in any known universe, yet it has been blacked out on page 15 of the book.  (It still appears intact in the Index.)

In short, the book embodies the practice of national security classification as it exists in the United States today.  It does not exactly command respect.

A few selected pages from the original and the censored versions of Operation Dark Heart have been posted side-by-side for easy comparison here (pdf).

The New York Times reported on the Pentagon’s dubious handling of the book in “Secrets in Plain Sight in Censored Book’s Reprint” by Scott Shane, September 18.

The House and Senate this week approved legislation that will require the Inspector General of each executive branch agency that classifies information to evaluate the agency’s classification program and to assess its implementation of classification policies and procedures. The new measure should help to bolster the oversight of the national security classification system, which is currently the sole responsibility of the Information Security Oversight Office.

The provision was included in the “Reducing Over-classification Act” (HR 553), which was originally introduced by Rep. Jane Harman (D-CA) and amended by Sen. Joe Lieberman (I-CT) and which generally seeks to promote improved information sharing.

Despite its bold title, the legislation does not establish any new criteria for assuring appropriate classification nor does it even define the term overclassification. Yet by enlisting the Inspectors General to oversee agency compliance with current classification policies, the bill could make a significant contribution to addressing the problem of wrongful or unnecessary secrecy.

In particular, the IGs may be expected to monitor agency implementation of the Fundamental Classification Guidance Review, the Obama Administration initiative that is supposed to eliminate obsolete classification requirements in each agency (as required by executive order 13526, section 1.9). To date, there is no available evidence that agencies have made any progress in performing the Reviews, which must be completed by June 2012.