Risks and Rushing:
The Causes and Costs of Production Concurrency
Statement prepared for the House Committee on Government Reform, Subcommittee on National Security, Veterans Affairs, and International Relations, hearings entitled Joint Strike Fighter (JSF) Acquisition Reform: Will It Fly? May 10, 2000,
Thomas L. McNaugher
The RAND Corporation
A decade ago I published a book about military research, development, and acquisition in which I identified patterns of defense acquisition that, while often producing some of the world’s best weapon systems, nonetheless represented a very costly way to develop and field new technologies. I am honored to appear before this subcommittee to discuss one of those patterns, namely so-called "production concurrency" – the widespread practice of rushing new systems through the later stages of development, into production, and thence to operating units. Let me be clear before I begin, however, that my message here is grounded in my book; I have done very little research on the weapons acquisition process since 1990. I am speaking for myself, in other words, not the RAND Corporation.
Production concurrency was justified during the Cold War on three seemingly unassailable grounds:
Clearly the first of these justifications – what might be called the "assumption of decreasing uncertainty" – was key to the approach; the latter two justifications made sense only so long as technical challenges slowly disappeared in the latter stages of the development process. Yet research for the book led me to question and ultimately reject precisely that assumption. What I documented was a significant upswing in technical and design uncertainty as programs neared the end of development. These were largely unavoidable, and sprang from several sources.
First, "system integration" normally occurred fairly late in the development process, and presented unique developmental challenges and risks. Successfully testing individual components of a new system was no guarantee that those components would work together as a system. The component parts of the B-1B bomber’s electronic countermeasures (ECM) suite, for example, tested well individually. But when placed in position aboard the aircraft they interacted with each other in perverse ways that demanded redesign of components as well as the system as a whole. Significantly, in this case system integration occurred very late indeed; the ECM suite was not tested as a system until after the first production B-1Bs entered operational service.
System integration problems seemed to plague even relatively simple combinations of reasonably well-understood components. The Army’s DIVAD (division air defense) system mated the old M-48 tank chassis with the F-16’s APG-68 radar and a production model German gun system. Not a difficult combination, one might think, and on that assumption that Army’s program office sought a relatively rapid move to production. Indeed, the Army signed a fixed-price final development and production contract with Ford Aerospace, the winning contractor, that raised the costs of violated contractual test and production milestones. Yet the Army was forced to violate those milestones, as DIVAD encountered an array of technical and operational test difficulties that ultimately led to its cancellation in 1985. Not all of DIVAD’s problems stemmed from systems integration; getting an aircraft radar to work in the ground environment proved to be more challenging that expected. But integrating DIVAD’s three basic components contributed to the system’s problems and ultimate cancellation.
The move to production tooling inserted a second set of new technical challenges into the acquisition process, and often ushered in a subtle interaction between the original design and the tooling for it. As with DIVAD, this observation applied even to seemingly simple systems. Army contractors successfully prototyped the TOW (tube-launched, optically-tracked, wire-guided) anti-tank missile in the early 1960s, but the system did not enter production until 1970. The delay stemmed in substantial part from technically difficulties associated with mass-producing spools of very thin wire wound carefully enough to unravel quickly, as the missile flew, without breaking.
A third set of late-arising technical challenges emerged when new systems were sent to operating units. Sometimes rigorous, day-to-day operational use surfaced lingering design flaws that remained hidden during formal operational testing late in the development process. But the more pronounced design challenge normally stemmed from the extent to which real operators – pilots, soldiers, and sailors – use new equipment in ways that were not envisioned by designers or tested in formal testing. Although this was a tribute to the work of developers – their systems invited novel uses – it nonetheless could produce embarrassing and costly technical problems in their products. Some of the reliability and performance problems that plagued the US Air Force’s F-1000 engine, for example, stemmed from the engine’s remarkable power and resilience. Whereas the engine had been designed principally for speed, pilots of the new F-15, the first aircraft powered by the F-100 engine, found operational advantage in rapidly changing speed, submitting the engine to "thermal cycles" not envisioned as the engine was designed and tested. This was one of several reasons the new engine underwent nearly a decade’s worth of maturational development before reaching its full potential.
These three sets of technical challenges were, in my view, more-or-less generically associated with the development of new weapon systems. A fourth set of challenges appeared to be avoidable. This set resulted from major system redesign at the end of the early phase of development (the so-called "demonstration-validation," or "dem-val" phase) that substantially devalued the information gained during that phase. In these cases, what was then called "full-scale engineering development" (FSED, now called "engineering-manufacturing development," or EMD) confronted risks and technical challenges more appropriate to an earlier phase of development.
Redesign was sometimes prompted by the failure of technological prospects in the dem-val phase. In the development of the Air Force’s Advanced, Medium-Range Air-to-Air Missile (AMRAAM), for example, dem-val did exactly what it was supposed to do – it showed that new, solid-state components could not produce the radar power required of that missile. FSED thus commenced on the basis of a more traditional design based on standing-wave tube technology. But getting the required power from that older technology into a tube small and light enough to be carried by the F-16 tossed up enormous operational and technical problems that drove the system’s costs up and plagued early operational testing. Ultimately the Congress intervened directly in the program, setting out test, cost, and scheduling criteria of its own to control AMRAAM’s late development.
On other occasions, the services imposed significant new and additional requirements on the evolving design simply because the move from dem-val to FSED presented the opportunity to do so. The F-16’s dem-val phase was the so-called "lightweight fighter competition," held in the early 1970s, which pitted prototypes from General Dynamics and Northrop against each other in a competition focused exclusively on fighter characteristics. In taking the winning design (General Dynamics’) into FSED, the USAF asked for strike (bombing) as well as fighter capabilities. Not surprisingly, the additional requirements prompted substantial design changes in the F-16’s airframe as well as its avionics.
Production concurrency represented a rather expensive way to structure the transition from development to production. Capitalizing and hiring labor on the basis of the optimistic assumption of decreasing technical risk raised the cost of every delay and design change. Moving into production before designs matured meant sending less capable systems into the force; a B-1B lacking a working ECM suite was not capable of performing the mission it was design to perform, namely penetrating Soviet airspace. These problems could be fixed, but it was more expensive to fix fielded systems than it was to fix the pre-production design. To the extent that the services simply foreswore expensive retrofits to fielded systems, on the other hand, they reduced the overall military effectiveness of their fielded fleet of that system. In these cases, production concurrency actually reduced the overall military effectiveness of the fielded fleet, while it also raised costs.
The pervasiveness of production concurrency in Cold War weapons acquisition suggests that the nation was willing to accept these costs, presumably in the urgent pursuit of continued technological superiority over Soviet forces. Arguably with the Soviet threat gone, the nation is in a position to slow the rush to production in an effort to handle the late-arising risks and uncertainties I have just described. Significantly, this does not mean imposing a sharp break between development and production. Rather, it means organizing acquisition in a way that recognizes that the move to production is in fact part of the development process, and that early operational experience with a new system is in a very real sense part of the system’s operational testing. One alternative – I suspect there are others – would be a slower move to production, extensive operational use of initial production articles, and rapid feedback that takes the information so-generated and applies it to design "fixes" before the new system is far into production.
Such an approach could yield considerable military effectiveness, possibly at reduced cost than what was associated with Cold War programs. Examining the F-15’s development schedule, for example, RAND’s Allen D. Lee modeled an alternative approach that "minimized concurrency, extended low-rate production, and intensified testing [of initial production aircraft] and use of test information [to modify the initial design]." The result:
Costs decreased about $7 million [in comparison to actual program costs incurred]; Air Force retrofit labor decreased by 180,000 man-hours; and overall [fleet] effectiveness increased modestly [because more F-15s were outfitted with larger fuel capacity and newer electronics technology].
Although slowing the rush to production to make these fixes would have added marginally to the length of the development cycle, this would be less of a concern in the absence of the Soviet threat.
Notwithstanding these potential benefits, slowing the development process, and especially the move to production, may be very difficult to achieve. Production concurrency may have been rationalized on the basis of meeting the Soviet threat. But it served political as well as strategic purposes: it raised sunk costs and increased program "momentum" at a crucial juncture in the program’s evolution. In the sometimes highly charged political milieu through which weapon systems moved from inception to production, the latter stage of development was an especially dangerous phase. Elaborate testing inevitably yielded hard data that undermined the confidence of early – and almost always optimistic – cost and performance estimates. This was exactly what testing was intended to do, of course, and in a purely technical environment test data might have been used to "fix" the design. But in the political stew that often came to surround controversial or expensive weapons programs, bad test data could easily become the basis for political indictment, legitimate or not, killing or dramatically altering the program before it got to production. Production concurrency blunted these efforts by raising the costs of slowing down. Often production articles were already in the force before unflattering test data were available.
I say this not to disparage the motives of service program managers but to call attention to the obvious, if under-appreciated, fact that America’s weapons acquisition process is a political, as well as a military-technical, undertaking. Defense funds are public money, and "national security" is a public good. Inevitably, incredibly complex technical development projects are submitted to the rigors of the annual budget process as well as countless other committee hearings. Bureaucratic machinations in the Pentagon interact with politics on Capitol Hill. Weapons projects become the objects of the "games people play" – the normal tugging and hauling by which U.S. public policy is made.
Under these circumstances, one could at least argue that political strategies might have been essential to move new systems steadily through the political process. For much of its history the United States purchased weapons episodically if at all. In the realm of ground and air force equipment, the problem was not buying too fast, but buying anything at all! Normally U.S. weapons technology lagged rather than led that found in Europe. Against this background, the fifty years from the beginning of World War II through the end of the Cold War stand out as the only years in which the nation armed itself, more-or-less systematically, with the most technically sophisticated weapons in the world. Arguably it took a pervasive sense of threat to mobilize and focus the nation’s complex political processes sufficiently to achieve that sustained result. That result may also have required that programs be structured in ways that withstood the vicissitudes of the political process, even if doing so ignored some technological risks and raised costs.
I make no value judgment either way about the politics of weapons acquisition or the behavior of program managers or other players in the process. I merely point out that, to the extent that the acquisition process is shaped by political as well as technical and military considerations, the collapse of the Soviet Union need not automatically usher in a new era of slower, more careful, weapon development programs. We can and should rewrite our acquisition regulations to reflect an approach to development that respects late-arising risks. Hopefully an era marked by the absence of the Soviet threat yet continued U.S. engagement around the world will produce a political environment in which such regulations can affect programs more powerfully than they did during the Cold War.
But this may be asking more than the political system can deliver, in which case the acquisition process will continue to be shaped largely by political forces. Concerned officials or legislators will focus on specific projects – the JSF being a good example -- perhaps because they look risky, are expensive, exhibit excessive cost growth, or have failed certain tests. This was, I would argue, the Cold War reality. It can be arbitrary, especially in the short run. Over time, however, the political process does tend to pick out the losers, or the problem cases, and bring them under focused political control. Intervention of this sort still needs guidelines – a set of rules, or a model pattern of development, that can serve as the basis for imposed control. Thus here too it is worthwhile trying to write acquisition regulations that seek to prescribe such practices. But the extent to which these prescriptions are implemented will vary, perhaps widely, across programs.
Which brings me to the subject of this hearing. Just as the Joint Strike Fighter is seen as the first major post-Cold War weapon development project, so this hearing should be seen as helping to launch the post-Cold War acquisition debate after a decade long "procurement holiday". It and others like it will play a role in shaping the political consensus I refer to above. Nowhere on the horizon can we find a numerically superior threat seeking to match or surpass our technological prowess across nearly the full spectrum of weaponry. Yet as a nation we remain engaged globally, and we have found our technically sophisticated weaponry to be very useful in handling problems from Iraq’s invasion of Kuwait to ethnic cleansing in Kosovo. This hearing and others like will have some influence on whether we can arrive at a political consensus that can support a weapons acquisition process that is more "relaxed" than Cold War practice yet not as lethargic and episodic as was the practice before World War II.