DOT&E Director, Operational Test & Evaluation  
FY98 Annual Report
FY98 Annual Report


Air Force ACAT IC Program: Prime Contractor
Total Number of Systems:One of a KindSeveral
Total Program Cost (TY$):$1770M 
Average Unit Cost (TY$):$1770M 
Full-rate production:1QFY94Service Certified Y2K Compliant
Rebaselined:4QFY94No (In progress)
Integrated Mission IOT&E:4QFY94 


The Cheyenne Mountain Upgrade (CMU) program upgrades ballistic missile, air, space, and command center elements within the Cheyenne Mountain Complex, North American Aerospace Defense (NORAD) Command, Colorado Springs, CO. CMU also upgrades and provides new capability to survivable communications and warning elements at the National Military Command Center, U.S. Strategic Command, and other forward user locations. CMU additionally provides an austere Alternate Missile Warning Center backup to Cheyenne Mountain at Offutt Air Force Base, NE.

Cheyenne Mountain is at the heart of the Integrated Tactical Warning and Attack Assessment (ITW/AA) system which provides warning and assessment of possible attack on North America, our allies, and our space assets from ballistic and cruise missiles, space weapons, and aircraft. The ITW/AA mission encompasses three related portions: missile warning, space control, and air sovereignty/defense. To accomplish this mission, information provided to warfighters through the ITW/AA system must be timely, accurate, and unambiguous. Furthermore, the ITW/AA system must be survivable, maintain strict separation of real and test data, and be highly reliable, minimizing unscheduled downtime.

Timeliness ensures that National Command Authorities, Unified Commanders-in-Chief (CINC), Canadian authorities, and other decision-makers receive tactical warning and assessment information in sufficient time to take appropriate defensive and retaliatory measures. Accuracy of information is essential to forming and executing corrective actions. Lack of ambiguity ensures there is no conflicting information (e.g., different information on the same event on two displays) that can lead to hesitation or confusion. Survivability ensures that ITW/AA information continues to be provided in peacetime contingencies and through the pre- and early trans-attack portions of wartime. Separation of real and test data minimizes the risk of false alarms, while reliability supports round-the-clock vigilance.

Cheyenne Mountain Upgrade systems upgrade or replace ITW/AA processors, displays, and communications at the NORAD Command Center, other centers within the Cheyenne Mountain Complex in Colorado Springs, CO, and at worldwide forward user locations. CMU also upgrades and provides new processors and displays at the National Military Command Center, the United States Strategic Command, unified CINC command centers, the Canadian National Defense Operations Center, and other user locations. CMU additionally provides an austere Alternate Missile Warning Center backup to Cheyenne Mountain at Offutt Air Force Base, NE.

Through these upgrades, CMU supports the Joint Chiefs of Staff's military concept of full-dimensional protection to the warfighter, enhancing the ability of our nation's civilian and military leadership to rapidly and accurately correlate ITW/AA information and prepare and execute the appropriate response through the application of information superiority.

Functionally, CMU is a system-of-systems, consisting of the following six major systems: (1) the Survivable Communications Integration System (SCIS); (2) the Command Center Processing and Display System Replacement (CCPDS-R); (3) the Alternate Missile Warning Center (A/MWC); (4) the Space Defense Operations Center 4 (SPADOC 4); (5) the Granite Sentry; and (6) the Communications System Segment Replacement (CSSR).


The national attack warning system has existed since the United States identified the need for early warning against air attack during the Second World War. In the Post-War era, this system and similar Canadian systems were brought together as the multi-national North American Air Defense Command. With the development of computers and the threat of ballistic missile attack, NORAD incorporated a series of semi-automated warning and assessment systems, culminating with the development of the 427M computer system. These systems were hosted within Cheyenne Mountain, outside of Colorado Springs, CO. The 427M system became operational in 1979.

Beginning in 1979, NORAD and the United States Air Force Air Defense Command developed a series of command level programs to resolve operational effectiveness and suitability problems with the aging 427M computer system. These problems were addressed by the creation of individual acquisition programs with limited scope and cost.

By the mid-80s, six such programs were underway with an aggregate cost of almost $2 billion. There was a lack of integration in the efforts of these projects due to their independent origins, resulting in conflict and duplications. By 1985, it was evident that the set of CMU upgrades needed to be restructured as a single major integrated upgrade program. This was reinforced by a series of GAO reports. The Cheyenne Mountain Upgrade program was formally started in 1989 under Congressional and Defense Acquisition Board direction, to consolidate the previously separate acquisitions into a single program.


OT&E of CMU systems has been ongoing since 1989 when the first phases of SPADOC and Granite Sentry were operationally tested by Air Force Space Command. CMU was placed on the DOT&E oversight list and AFOTEC was named as the operational test agency in 1990.

Several subsystems were tested between 1990 and 1993 with mixed results. DOT&E assessed the overall CMU program as marginal in both operational effectiveness and suitability, and warned that the program was at risk of breaching its baseline.

In 1994, the Air Force rebaselined the program with a three-year stretch-out from 1996 to 1999 and a $48 million increase in funding. Major changes to the CMU test concept involved a stretch-out of the operational test program to coincide with the program extension and the selective use of combined developmental tests and operational tests (DT/OT) and operational assessments to supplement dedicated IOT&E in support of user operational acceptance decisions.

An end-to-end IOT&E of the missile-warning portion of the ITW/AA mission was conducted in 1996. This was one of the most extensive tests ever attempted, internetting a worldwide network of ten radar and infrared satellite sensor sites, the Missile Warning and NORAD Command Centers within Cheyenne Mountain, the Alternate Missile Warning Center, and over 25 worldwide forward user sites, including the NMCC. Based on this IOT&E, DOT&E assessed the CMU subsystems supporting the missile-warning mission as operationally effective and suitable with some limitations. The limitations were reviewed by the operational community, who determined that the CMU subsystems had no significant operational impact due to numerous workarounds and near-term software and hardware fixes.

An operational test of Granite Sentry and an end-to-end operational assessment of the Air Defense mission (including nuclear detonation detection mission) were conducted in 1997. DOT&E assessed these subsystems as effective and suitable with some limitations regarding forward user situational awareness. These limitations have since been addressed.

A final end-to-end test of the CMU system-of-systems was conducted in 1998. This test also included the first end-to-end test of the space control portion of the ITW/AA mission interfaced to an operational CSSR. The test involved injection of simultaneous missile, air, and space scenarios at forward sensor sites and at other selected CMU communications entry points. The purpose of this test was to ensure that the overall CMU system met user requirements under a realistic wartime data load.


DOT&E finds CMU to be operationally effective, with limitations in the space control portion of the ITW/AA mission. CMU is operationally suitable for its wartime mission, but unscheduled missile-warning downtime during routine operations and the SPADOC 4's increasing lack of logistic supportability are of concern.

Tests demonstrated that CMU systems: (1) generally provide timely, accurate, and unambiguous tactical warning and assessment information for all three portions of the ITW/AA mission; (2) are survivable; and (3) maintain strict separation of real and test data. Limitations involved delays and ambiguity of some space warning messages processed by SPADOC 4, caused primarily by inaccurate SPADOC 4 threat data bases and inadequate man-machine interfaces. SPADOC 4's man-machine interface is overly rigid and caused operators to enter conflicting information and make errors in entering space attack threat information, particularly during high stress periods. The Air Force Space Command plans to incorporate new man-machine interfaces as part of the NORAD/USSPACECOM Warfighter Support System initiative, but it is not clear when these new interfaces will become available.

CMU's survivability is adequate to perform the mission in peacetime and in the pre- and trans-attack phases of the wartime threat. However, there are minor weaknesses in High-Altitude Electromagnetic Pulse (HEMP) protection of some CMU equipment at the National Military Command Center. Furthermore, until the SCIS communications interface to MILSTAR is certified and tested, survivable communications will continue to rely on the Defense Satellite Communications System-based Jam-Resistant Secure Communications System, which is to be replaced by MILSTAR over the next few years. Communications through MILSTAR was not tested because the interface between CMU systems and MILSTAR is not certified for testing.

The IOT&E simulated a high defense condition (DEFCON), during which time non-mission activities such as routine software upgrades, training, and exercises are minimized or eliminated. The CMU suitability rating is based on meeting unscheduled mission downtime requirements for all portions of the ITW/AA mission during the 60-day integrated IOT&E.

DOT&E also reviewed historical downtime data for non-wartime, low DEFCON operations over the period August 1996-August 1998. Unscheduled downtime increased for all three portions of the ITW/AA mission, but particularly for missile warning. Predictably, complications from software upgrades and associated testing, crew training, and occasional non-CMU communications equipment outages, increased the average downtime during peacetime operations.

Unscheduled missile-warning mission downtime raises concern over the adequacy of assessments of missile attacks that might occur during low DEFCON periods. While the probability of a "bolt-out-of-the-blue" attack in the post-Cold War era is low, an accidental or unauthorized launch can not be totally discounted. On a positive note, analysis shows that most failures are quickly repaired, with longer outages being a rare occurrence. This indicates that rapid transition between peacetime and wartime environments should be possible in most cases. Additionally, downtime risk is mitigated by the system's capability to provide uncorrelated missile warning sensor data directly to forward users, either by automated means or by voice, even if CMU systems are down. Nonetheless, as attack assessment will be degraded under such circumstances, downtime causes should be investigated and corrected.

Even though downtime requirements for all three portions of the ITW/AA mission were met during IOT&E, we are concerned about the increasing difficulty supporting SPADOC 4's aging IBM 3090-200J computers and peripherals. If uncorrected, this situation could lead to unacceptable downtime even in high DEFCON periods. As a near-term fix, the Air Force Material Command purchased two complete IBM 3090 systems and converted them into spares, but a long-term solution must be found to replenish spares or replace the computers. The lack of an independent disc drive back up subsystems between SPADOC 4 processors and contribute to excessive downtime during the non-IOT&E period. SPADOC 4 does not have fully redundant disc drive subsystems to provide instant, up-to-date data base access. Redundancy exists within, but not between disc subsystems. The current configuration can require several hours to recover from a disc controller failure. When this happens, the space control mission must be transferred to the backup operations center in Dahlgren, VA. The Air Force Space Command has identified a replacement disc drive architecture to correct this problem and plans to install it by June 2000.

Year 2000 (Y2K) processing was assessed as a special interest item. The System Program Office established a test sub-organization to specifically test all aspects of the ground-based ITW/AA system for Y2K compliance. A five-phased test approach is being implemented beginning with individual component testing and ending with an operational test of the system with the clocks set to four specific time periods. DOT&E's assessment is that CMU and testing will meet the mandate of Y2K compliance.

Dedicated operational testing of the CMU program is complete and no further OSD oversight is planned. U.S. Space Command declared CMU full operational capability on October 14, 1998. All legacy systems have been decommissioned and CMU is supporting current operations. This will be DOT&E's final CMU Annual Report.


Strategic command, control and communications systems such as CMU are typically one-of-a-kind, and their lengthy acquisition cycles makes it difficult to incorporate the latest computer technology. The CMU program was initiated primarily from the increasing unsupportability of the previous 427M system. Ironically, now that CMU is complete, some of its systems are again becoming unsupportable. Management initiatives fostering faster acquisition time for such programs, without reducing necessary testing, should be investigated.

The lack of an integrating contractor early in the program led to a number of interoperability problems among the various subsystems, resulting in time delays and cost increases. Problems were exacerbated by lack of a single point of contact for the larger ITW/AA system within which CMU is embedded. Future projects such as this would benefit greatly by greater use of integration offices with the broad knowledge and authority to guide the process. It is worth noting that U.S. Space Command is in the process of establishing an integrated ITW/AA office in Cheyenne Mountain.

Combined developmental and operational tests provided developers the opportunity to identify operational deficiencies early in the program, and in turn correcting deficiencies earlier in the program was inexpensive relative to the cost of performing them later. CMU demonstrated that combining developmental and operational tests early in a program not only save the program time and money in the long run, but also assists in delivering more operationally effective and suitable systems to the warfighter.

Recommendations to improve effectiveness:

Recommendations to improve suitability:

Return to Table of Contents