Rodrigues May 10

Mr. Chairman and Members of the Subcommittee:

I am pleased to be here today to discuss the application of best commercial practices to Department of Defense (DOD) weapon acquisitions in general and to the Joint Strike Fighter in particular.

After having done hundreds of reviews of major weapon acquisitions over the last 20 years, we have seen many of the same problems recur—cost increases, schedule delays, and performance problems. Over the last 4 years, we have undertaken a body of work to examine weapon acquisition issues from a different, more cross-cutting perspective. Specifically, we have examined the best product development practices of leading commercial firms. Collectively, our reviews have included the practices of over 20 leading commercial firms that represent a variety of industries, including electronics, satellite communications, automotive, medical, and aircraft. Leading commercial firms are getting the kind of results that DOD seeks: more sophisticated products developed in less time and cost than their predecessors. Our work shows that DOD can learn valuable lessons from the commercial sector to get better and more predictable outcomes from weapon system development programs. A listing of the reports we have issued on best practices that can be applied to weapon acquisitions is included in the appendix.

DOD has taken steps to reflect best commercial practices in its acquisition policies. However, the real test of these policies is in how they influence individual decisions, such as the upcoming engineering and manufacturing development decision on the Joint Strike Fighter program. This program is to produce three fighter variants to meet multiservice requirements: conventional flight for the Air Force, short take-off and landing for the Marine Corps, and carrier operations for the Navy. The program will also provide aircraft to the British royal Navy and Air Force. As currently planned, the program will cost about $200 billion to develop and procure over 3,000 aircraft and related support equipment.

My testimony focuses on the best commercial practices for developing new products, the reasons why DOD does not follow these practices, and the opportunity that Joint Strike Fighter represents to strengthen—or weaken—the effect of best practices and acquisition reform on major weapons.

Leading commercial firms have adopted a knowledge-based approach to developing new products, underwritten by incentives that encourage realism, candor, and meeting product expectations. Making sure that new technology is mature—that is, demonstrating that it works—before a product development starts is the foundation for this approach. Leading commercial firms discipline the product development phase by adhering to (1) time limits for completing development and (2) high standards for demonstrating design and production knowledge. These practices have put commercial managers in an excellent position to succeed in developing better products in less time and producing them within estimated costs. To do otherwise would risk failure in the form of customer dissatisfaction.

DOD programs, with some exceptions, proceed with lower levels of knowledge about key factors of product development and allow technology development to take place during product development. DOD’s variances from best practices stem from strong incentives for starting programs too early; overpromising performance capabilities; and understating expected costs, schedules, and technical risks. While these incentives evolved over time to help build support for programs, they put program managers in a very difficult position to deliver better weapons on time and within budget. Moreover, there is little risk that the DOD customer will be dismayed by a program not being delivered as promised. DOD accepts the need to get better outcomes from its weapon system programs and accepts best commercial practices as a way to get those outcomes. In fact, it is currently incorporating such practices in a major revision of its acquisition policy. However, new policies will not produce better program outcomes unless they influence the decisions made on individual weapon systems.

DOD has designated the Joint Strike Fighter as a flagship program for acquisition reform. Funding requests are now before the Congress to support the Joint Strike Fighter’s April 2001 entry into engineering and manufacturing development. By best practices standards, none of the fighter’s critical technology areas identified by the program office are expected to be at readiness levels considered an acceptable risk for entry into engineering and manufacturing development. Delaying this phase of the program until these technologies are mature would improve the chances that the Joint Strike Fighter will be fielded as planned. However, despite not having the requisite knowledge for the eight technologies, DOD has deemed the risks manageable and proposes to proceed with the program as planned. Such a decision reinforces traditional incentives and increases the likelihood for future cost, schedule, and performance problems. DOD’s plans to move the Joint Strike Fighter into engineering and manufacturing development with immature technology illustrates a lack of commitment to following best commercial practices as part of its acquisition reform efforts.

The characteristics of best commercial practices suggest a process for developing new capabilities—whether they are commercial or defense products—that is based on knowledge. It is a process in which technology development and product development are treated differently and managed separately. The process of developing technology culminates in discovery and must, by its nature, allow room for unexpected results and delays. The process of developing a product culminates in delivery, and therefore, gives great weight to design and production. Discipline is inherent because criteria exist, tools are used, and a program does not go forward unless a strong business case on which the program was originally justified continues to hold true.

We have learned that a knowledge-based process is essential to getting better cost, schedule, and performance outcomes. This means that decision makers must have virtual certainty about critical facets of the product under development when needed. Such knowledge is the inverse of risk. Commercial and military programs do not all follow the same processes in their development cycles. However, at some point, full knowledge is attained about a completed product, regardless of the development approach taken. This knowledge can be measured at three key junctures that we refer to as knowledge points:

Knowledge point 1: when a match is made between the customer’s requirements and the available technology;
Knowledge point 2: when the product’s design is determined to be capable of meeting performance requirements; and
Knowledge point 3: when the product is determined to be producible within cost, schedule, and quality targets.

We have identified metrics that indicate these knowledge levels and can thus help forecast outcomes as a development program progresses. A best practices model for technology development and product development is depicted in figure 1.

Figure 1: Best Practices for Developing Technology and Products

Commercial firms gain more knowledge about a product’s technology, design, and producibility much earlier than DOD acquisition programs we reviewed. Two features of leading commercial products stand out for making a manageable product development. First, there is a clear break between technology development and product development. The launch of a new product development in commercial ventures is a clearly defined undertaking and before beginning, firms insist on having the technology in hand that is needed to meet customer requirements. Second, leading firms limit the length of time it takes to develop the product. This limit is key to getting the product to market and focuses attention on the design and production knowledge points. It also provides discipline to the technology development process to ensure product development will not be launched until the technology match is made.

The leading commercial firms we have visited consciously limited their product developments from 18 months to just over 4 years. They understand that this keeps product development within a time frame that can keep people focused on delivering a product. In fact, one commercial executive observed that it is unreasonable to expect people to focus on a goal like production start that is more than 4 years away. The limited time frames provide strong incentives for a commercial manager to keep immature technologies out of the product design. In fact, these time frames give product managers clout in fostering cost and performance trade-offs before the program begins, ultimately limiting a product’s requirements to what can be achieved with demonstrated technologies within the specified time period. To live within these time constraints and keep innovation alive, commercial firms have adopted an evolutionary approach; they save requirements that cannot be met with proven technologies for the next iteration of the product. Commercial firms also found that limiting product development time frames

makes it easier to hold people accountable for meeting promised cost, schedule, and performance targets;
enables a production-oriented focus throughout product development, providing incentives for identifying risks early;
makes product development costs and schedule more predictable; and
allows firms to get into production and, therefore, to the point of sale quicker.

Once a product development is under way, the firms demand—and receive—specific knowledge about the design and producibility of the product before production begins. The process of discovery—the accumulation of knowledge and elimination of unknowns—is completed well ahead of production. There is a synergy in this process, as the attainment of each successive knowledge point builds on the preceding one.

In contrast, DOD programs are started earlier and allow technology development to continue into product development and even into production. Consequently, the programs proceed with much less knowledge available—and thus more risk—about required technologies, design capability, and producibility. This approach to technology and product development is shown in figure 2.

Figure 2: Levels of Knowledge Attained in DOD Weapon System Developments

Proceeding with lower levels of knowledge available means that during product development, maturity of technology, the design, and production methods must all be pursued at the same time. The rippling effect of discovering and overcoming problems in product development explains much of the turbulence in DOD program outcomes. Metrics, such as technology readiness levels and percent of engineering drawings complete, can be used to predict these consequences. Product development times, long to begin with, stretch even further in reaction to problems. We calculate that they can take 3 to 10 times as long as commercial products.

Technology development has the ultimate objective of bringing a technology up to the point that it can be readily integrated into a new product and counted on to meet requirements. We have found that getting the match between customer requirements and mature technology to be the biggest contributor to a successful product development. As a technology is developed, it moves from a concept to a feasible invention to a component that must fit onto a product and function as expected. In between, there are increasing levels of demonstration that can be measured. In our review of best practices for including new technology in products, we applied a scale of technology readiness levels—from one to nine—pioneered by the National Aeronautics and Space Administration and adapted by the Air Force Research Laboratories.

Without going into the details of each level, a level four equates to a laboratory demonstration of a technology that is still not in a usable form. Imagine an advanced radio technology that can be demonstrated with components that take up a table top. When initial hand-built versions of all of the radio’s basic elements are hand-wired and tested together in a laboratory, the radio reaches a readiness level of five. A technology readiness level of seven is the demonstration of a technology that approximates its final form and occurs in an environment outside the laboratory. The same radio at level seven would be installed and demonstrated in a platform, such as a fighter aircraft.

The lower the level of the technology at the time it is included in a product development program, the higher the risk that it will cause problems in the product development. According to the Air Force Research Laboratory, level seven enables a technology to be included on a product development with acceptable risk. When we asked leading commercial firms to apply these standards to their own methods for assessing technology maturity, we found that most insisted on even higher levels of readiness before they allowed a new technology into product development. When we examined weapon systems that experienced cost and schedule problems, we found that they started with key technologies at levels three and four. By the time the programs reached a point DOD considers analogous to beginning product development, key technologies were still at level five or lower. Conversely, DOD programs for which key technologies were at significantly higher levels of maturity at the start, had not experienced such problems.

By the halfway point in product development, leading commercial firms achieve near certainty that their product designs would meet customer requirements and have gone a long way toward ensuring that the products can be produced. Both DOD and the commercial firms hold a critical design review to review engineering drawings, confirm the design is mature, and "freeze" the design to minimize changes in the future. The completion of engineering drawings and their release to manufacturing organizations signify that program managers are confident in their knowledge that the design performs acceptably and is mature. The drawings are critical to documenting this knowledge because they are precision schematics of the entire product and all of its component parts. They also reflect the results of testing and simulation and describe the materials and manufacturing processes to be used to make each component.

Both DOD and commercial companies consider the design to be essentially complete when about 90 percent of the engineering drawings are completed. Officials from commercial companies such as Boeing and Hughes told us that they typically had over 90 percent of these drawings available for the critical design review. The DOD programs we reviewed had less than 60 percent—one had less than one-third—of the drawings done at the time their critical design reviews were held. Thus, these programs had significantly less knowledge about their designs. The programs did not get or were not expected to get to the 90-percent level of completion on the drawings until late in development or in production. Nonetheless, at the time of the critical design reviews, the risks of proceeding with the rest of development on these programs as planned were deemed acceptable. The programs however encountered significant design problems in testing that occurred after the critical design review.

Leading commercial firms reach the point at which they know that manufacturing processes will produce a new product conforming to cost, quality, and schedule targets by the end of product development—before they begin fabricating production articles. Reaching this point means more than knowing the product could be manufactured; it means that all key processes are under control, such that the quality, volume, and cost of their output are proven acceptable. Commercial firms relied on good supplier relationships, known manufacturing processes, and statistical process control to achieve this knowledge early and, in fact, had all their key processes under statistical process control when production begins. All of the companies we visited agreed that a high level of knowledge about technology and design early in the process makes the control of processes possible.

DOD programs did not have nearly this level of knowledge at production. One weapon system program that had been in production for nearly 9 years at the time of our 1998 review still had less than 13 percent of its key manufacturing processes in control. Another program had 40 percent of its key manufacturing processes in control 2 years before production was scheduled to begin but was not scheduled to have all key processes in control until 4 years into production. Both programs experienced basic producibility problems that were not discovered until late in development or early in production. These risks went unrecognized even though DOD had established criteria for determining whether risks were acceptable and whether enough knowledge had been gained to enter the next development phase.

The most important factors in the adoption of best practices are the incentives perceived by managers of technology and product developments. The differences in the practices employed by the leading commercial firms and DOD reflect the different demands imposed on programs in their environments. The way success and failure are defined for commercial and defense product developments differs considerably and results in different incentives, evoking different behaviors from the people managing the programs. Specific practices take root and are sustained because they help a program succeed in its environment.

Leading commercial firms begin product development only when a solid business case can be made. The business case centers on the ability to produce a product that the customer will buy and that will provide an acceptable return on investment. The point of sale occurs in production after development is complete; program success is determined when the customer buys the finished product. If the firm has not made a sound business case, or has been unable to deliver on one or more of the business case factors, it faces a very real prospect of failure—the customer may walk away. Also, if one product development takes more time and money to complete than expected, it denies the firm opportunities to invest those resources in other products. Because the match between technologies and product requirements is made before the product development is launched, the cost and schedule consequences associated with discovery are minimized.

Production is a dominant concern throughout the product development process and forces discipline and trade-offs in the design process. This environment encourages realistic assessments of risks and costs; doing otherwise would threaten the business case and invite failure. For the same reasons, the environment places a high value on knowledge for making decisions. Program managers have good reasons to identify risks early, be intolerant of unknowns, and not rely on testing late in the process as the main vehicle for discovering the performance characteristics of the product. By protecting the business case as the key to success, program managers in leading commercial firms are conservative in their estimates and aggressive in risk reduction. Ultimately, preserving the business case strengthens the ability of managers to say "no" to pressures to accept high risks or unknowns. Practices such as maturing technologies to high readiness levels before inclusion in a program, having 90 percent of engineering drawings done by the critical design review, and achieving statistical process control before production are adopted because they help ensure success.

The basic management goal for a weapon system program in DOD is similar: to develop and deliver a product that meets the customer’s needs. However, the pressures of successfully competing for the funds to start and sustain a DOD program provide different incentives. Compared with commercial programs, the DOD environment encourages launching product developments that embody more technical unknowns and less knowledge about the performance and production risks they entail. A new weapon system can be more readily defended if it possesses performance features that significantly distinguish it from other systems. Consequently, aspiring DOD programs have incentives to include performance features and design characteristics that rely on immature technologies. These unknowns place a much greater focus on maturing technology during product development than we found on commercial programs.

Even though less information about a new product development is available at the time DOD programs are launched, the competition for funding forces detailed projections to be made from that information. A product development cannot be launched unless the program’s development and production cost, as well as timing, falls within available funding. Because DOD relies largely on forecasts of cost, schedule, and performance that are comparatively soft at the time, success in competing for funding encourages managers to squeeze cost and schedule estimates into profiles of available funding. Additional requirements, such as high reliability and maintainability, serve to make the fit even tighter. As competition for funding will continue throughout the program’s development, success is measured in terms of ability to secure the next installment.

The risks associated with developing new technologies together with the product—within tight estimates—are deemed acceptable. Production realities, critical to matching technological capabilities with customer requirements on commercial programs, are too far away from the DOD launch decision to have the same curbing effect on technology decisions. The environment for managing weapon system programs is particularly difficult for managing technology development. The ups and downs and resource changes associated with the technology discovery process do not mesh well with a program’s need to meet cost, schedule, and performance goals. Problems with developing technologies, which are to be expected, can actually threaten the support for a program if they become known.

These pressures and incentives explain why the behavior of weapon system managers differs from managers of commercial product developments. Problems or indications that the estimates are being breached do not help sustain funding support for the program in later years, and thus their admission is implicitly discouraged. An optimistic cost estimate makes it easier to launch a product development and sustain annual approval; admission that costs are likely to be higher could invite failure. Rewards for discovering and recognizing potential problems early in a DOD product development are few. Less available knowledge makes it harder for program managers to say "no." In contrast with leading commercial firms, not having attained knowledge—on the full performance of a key technology or the true risks facing manufacture, for example,—can be perceived as better than knowing that problems exist. For these reasons, the practices associated with managing to knowledge standards—such as for technology, design, and production maturity—are not readily adopted in DOD.

These observations about the differences between the commercial and DOD environments should not be interpreted to mean that commercial managers are somehow more skilled or knowledgeable than their DOD counterparts. DOD program managers act in response to the pressures they face. All of the numerous participants in the acquisition process play a part in creating these pressures. Commercial program managers are put in a better position to succeed; they have to worry only about product design and production within the cost, schedule, and performance demands of the business case.

Commercial practices for gaining knowledge and assessing risks can help produce better outcomes on weapon systems. Collectively, better individual outcomes will help DOD to attain modernization goals and improve funding stability for programs. For such practices to work, however, the knowledge they produce must help a DOD program succeed in its environment. Thus, the DOD environment must become conducive to such practices. At least two factors are critical to fostering such an environment. First, managers must be relieved of the need to overpromise on performance and resource estimates at the program launch decision. Separating technology development so that it does not have to be managed within the confines of a weapon system program would go a long way to relieving this pressure. Clearly, DOD has to develop technology, particularly the technology that is unique to military applications. However, by separating technology development from weapon programs, DOD could insist on higher standards for knowledge on its programs and get better results when those programs transition to production.

Second, once a program is under way, the participants in the acquisition process must make it acceptable for managers to identify unknowns as high risks so that they can be aggressively worked on earlier in development. If the Congress and DOD weighed program launch decisions and subsequent progress on weapon systems by applying a common set of knowledge standards, like those gleaned from leading commercial firms, they could create a better business case for starting a weapon system program. By developing technology separately to high readiness levels before including it in a program and by adhering to standards such as knowledge points in product development, DOD program managers can be put in a better position to succeed in the timely design and production of weapon systems. The shorter cycle times associated with these practices could make it possible to better align the tenures of program managers with the product development phase, making them more accountable for program outcomes.

The Office of the Under Secretary of Defense (Acquisition, Technology, and Logistics) supports shorter weapon system development times and more aggressive pursuit of technology outside of programs. It also supports the use of best commercial practices, such as taking an evolutionary approach to developing new products. DOD is capturing these and other practices in a substantial revision of the regulations that guide the management of weapon system programs. These regulations are currently in draft form.

The real test of the participants’ resolve to get better outcomes by applying best practices will be the decisions made on individual weapon systems, such as for launch and funding. These decisions define what success means in DOD and what practices contribute to success. Decisions made by DOD or the Congress to advance or fund programs that do not have enough knowledge to meet agreed-upon standards signal to managers that not having the necessary level of knowledge is acceptable. On the other hand, participants should support decisions to not start new programs that need technology advances to meet unforgiving requirements or to recognize early that a change in a program is necessary to attain desired knowledge levels.

The Joint Strike Fighter program, and its implications for best practices and true acquisition reform, is a good test of whether the desire for better outcomes can outweigh traditional pressures to get a program approved. In our recent review of the Joint Strike Fighter program, we employed knowledge standards consistent with best practices and DOD acquisition reforms. DOD has designated the Joint Strike Fighter program as a flagship program for acquisition reform. The Joint Strike Fighter program, the most expensive aircraft program in DOD, is at a critical juncture in its acquisition cycle. It is approaching the point where DOD must decide whether to commit to the engineering and manufacturing development phase—analogous to a commercial product launch decision. During engineering and manufacturing development, the Joint Strike Fighter will be fully developed, engineered, designed, fabricated, tested, and evaluated to demonstrate that the production aircraft will meet stated requirements. This phase is estimated to cost $20 billion, require annual funding levels as high as $4 billion, and last about 8 years.

The best practice for such a decision is to have a match between mature technologies and weapon requirements. It represents the first knowledge point. The Joint Strike Fighter does not meet this standard; several technologies that are critical to meeting requirements will not be sufficiently mature. Consequently, the Joint Strike Fighter will not enter the engineering and manufacturing development phase with low technical risk. However, DOD would like to go forward with the program anyway. Doing so would have two major consequences. First, it would put the program on a path that has yielded cost growth and schedule slippage on many previous programs. Second, as Joint Strike Fighter is the largest acquisition in the foreseeable future, it will send signals to other programs that best practices and acquisition reform need not be heeded when it comes to major weapon systems.

While we have been encouraged by the design of the Joint Strike Fighter acquisition strategy, we have become concerned about its implementation. Our biggest concern is that critical technologies for meeting affordability and performance requirements are projected to be at low levels of maturity if the engineering and manufacturing development contract begins as scheduled in April 2001. In other words, the program may not achieve a fundamental element of a knowledge-based process—the separation of technology development from product development—as it begins full-fledged product development activities. This means that the Joint Strike Fighter program manager will be responsible for developing technologies while concurrently designing, building, and testing the prototype Joint Strike Fighter aircraft. On past programs, such concurrency has caused significant cost growth and schedule delays. This is the first, and perhaps most important, knowledge point exhibited in best practices, because lack of knowledge about technologies means the program manager cannot fully focus on design and manufacturing issues. This additional risk makes product development cycle time and cost less predictable. In addition, once in a product development environment, external pressures to keep a program moving become dominant, such as preserving cost and schedule estimates to secure budget approval.

In our recently issued report, we evaluated the maturity of key technologies on the Joint Strike Fighter program. At our request, the program office identified eight technology areas that are considered critical to meeting the fighter’s cost, schedule, and performance objectives. In conjunction with the program office and the two competing contractors, we determined the readiness levels of these technologies needed to meet Joint Strike Fighter performance requirements at three points in time: when the Joint Strike Fighter program was started in 1996, when we conducted our review in December 1999, and when the program is scheduled to enter engineering and manufacturing development. Those assessments showed that when the Joint Strike Fighter program was started, most of the critical technologies were well below the readiness levels considered acceptable risk to begin a program. The technology readiness levels of the eight critical Joint Strike Fighter technology areas are shown in figure 3.

Figure 3: Joint Strike Fighter Critical Technology Readiness Levels at Program Start and as Projected for Entry into Engineering and Manufacturing Development

As figure 3 shows, all of the critical technology areas are expected to be at readiness levels lower than the level seven considered acceptable risk for entry into engineering and manufacturing development. Six of the technologies will still be below the readiness level that is considered acceptable risk for program start, which occurred over 3 years ago for the Joint Strike Fighter program. Many of these will only be demonstrated in laboratories or in ground tests when the engineering and manufacturing development phase starts. They have a considerable amount of development remaining before they are considered mature. Moreover, as a result of cost growth and schedule concerns, in May 1999, DOD delayed some technology demonstrations until after the engineering and manufacturing development phase begins.

Should any of these technologies be delayed or, worse, not available for incorporation into the final Joint Strike Fighter design, the impact on the program would be dramatic. For example, if one of the above critical technologies needed to be replaced with its planned backup, DOD could expect an increase of several billion dollars in production and operation and support costs. The backup technology would also significantly increase aircraft weight, which can negatively impact aircraft performance. This technology is expected to be at unacceptable readiness levels at the beginning of the engineering and manufacturing development phase, which indicates that substantial technology development must still occur during this phase.

As noted above, at the policy level, DOD officials have agreed that technology development should be kept separate from product development and that technology readiness levels are a valid way to assess technology maturity. However, in its response to our report on the Joint Strike Fighter—an individual program decision—DOD balked at the use of technology readiness levels and their implications for keeping technology development out of the Joint Strike Fighter’s engineering and manufacturing development phase. One of the reasons DOD cited for its unwillingness to accept the technology readiness levels assessed for the eight Joint Strike Fighter technologies was that the levels were based on integration in the Joint Strike Fighter aircraft—too high a standard. On the contrary, the technology readiness levels assessed by the program office and the contractors were based on a clear understanding that a level seven could be reached by demonstrating a technology in a relevant environment. It was further made clear that a relevant environment would include demonstrating a technology in an existing aircraft like an F-16, not a Joint Strike Fighter.

A second reason DOD disagreed with the readiness levels assessed for the eight technologies was that its own risk mitigation plans and judgment were more meaningful and that they showed the technology risk to be acceptable. Risk mitigation plans and judgment are necessary to managing any major development effort like the Joint Strike Fighter. However, without an underpinning such as technology readiness levels that allows transparency into program decisions, these methods allow significant technical unknowns to be judged acceptable risks because a plan exists for resolving the unknowns in the future. Experience on previous programs has shown that such methods have rarely assessed technical unknowns as a high or unacceptable risk; consequently, they failed to guide programs to meet promised outcomes. Technology readiness levels are based on actual demonstrations of how well technologies actually perform. Their strength lies in the fact that they characterize knowledge that exists rather than plans to gain knowledge in the future; they are thus less susceptible to optimism. A clear picture of knowledge—or its absence—may be more likely to prompt action than a favorable risk assessment.

The Joint Strike Fighter program began in 1996 and will not deliver its first operational aircraft until 2008. With the 4 years spent in concept demonstration and the 8 years to be spent in engineering and manufacturing development, the result is a 12-year development cycle. This is much longer than the development cycles of leading commercial firms and double the goal set by the Office of the Secretary of Defense. Much of this long development cycle is the result of the need to mature technologies that have not yet been demonstrated as ready to meet key cost and performance requirements. DOD has highlighted the Joint Strike Fighter program as one that will make significant cost/performance trade-offs in order to develop an affordable aircraft. However, DOD’s desire to have a low cost aircraft that must also meet demanding requirements has limited the range of technological solutions and has necessitated the pursuit for new technologies.

Traditionally, a weapon’s final performance requirements are developed early in a program, or in many cases before the program begins. In the case of Joint Strike Fighter program, requirements were finalized much later in the acquisition cycle. Program officials stated that this provided the program flexibility to conduct cost and performance trade-offs before requirement and design decisions became final. While this approach is consistent with best practices, it has not adequately taken into account the readiness of the critical technologies. Many of the trade-offs that were made involved decisions to bring technologies that were not yet demonstrated into the engineering and manufacturing phase of the program. Thus, the program does not have a baseline design based on demonstrated technologies that could be developed in cycle times commensurate with best practices.

As the Joint Strike Fighter program approaches the engineering and manufacturing development decision point, it is not in a good position to succeed, if success means delivering the aircraft on time and within budget. The design requires significant technological invention in order to satisfy all the user’s requirements. According to Joint Strike Fighter officials, an objective of the engineering and manufacturing development phase is bringing the technologies up to maturity levels that will allow them to be incorporated onto a Joint Strike Fighter. Therefore, at a time when the program should be focused on designing and building the aircraft, the Joint Strike Fighter program will have to put significant effort and resources into demonstrating that key technologies are ready for inclusion onto the product. As a result, the program has planned almost 8 years for its engineering and manufacturing development phase. The length and scope of the effort operate against the ability to estimate cost and completion schedules.

Commercial firms have established practices to limit product development cycle times, thereby increasing the possibility that program managers will remain on programs until they are complete. Holding one program manager accountable for the content of a product at the time the launch decision is made encourages that person to raise issues and problems early and not overpromise the capabilities of a new product by relying on immature technologies. This puts the manager in a good position to deliver a high quality product, on time, and on budget.

Since the next phase of the Joint Strike Fighter program is estimated to last about 8 years, program managers currently involved in key decisions about the development plan will likely not be responsible for its implementation. It has already had three program managers since its beginning about 3 ½ years ago. As a result, conditions to be accepted at engineering and manufacturing development, such as the acceptance of low technology readiness levels, will more than likely become the responsibility of another program manager.

At this point in its development, there are a number of ways for DOD to make the Joint Strike Fighter program’s environment more conducive to better cost and schedule outcomes. We believe that separating technology development from product development can still create conditions for a successful Joint Strike Fighter program. To proceed as planned—entering a phase of the program with immature technologies that should be focused on design and production—is to risk continued delays and cost growth. Instead, the program has an opportunity to mature technologies in a more risk-tolerant environment by making the right decisions now.

In our report, we recommend that the Joint Strike Fighter program continue in its current program definition and risk reduction phase, delaying the decision to move into engineering and manufacturing until technologies are demonstrated to acceptable levels. Taking the additional time to mature the technologies will then allow the program manager to focus on design and manufacturing risks during engineering and manufacturing development. It also increases the possibility of completing product development in a more timely and predictable manner. Such a delay does not necessarily lengthen the total product development cycle. In fact, the knowledge gained from time spent developing technologies in the beginning can often shorten the time it takes to get the product to market.

Similarly, a delay should not be misinterpreted as a lessening of support for the Joint Strike Fighter program. Rather, it would demonstrate decisionmakers’ willingness to make the up-front investment necessary to mature key technologies before committing the Joint Strike Fighter team to deliver a product. Such a commitment is more likely to put the program on a better footing to succeed than placing the burden on the engineering and manufacturing development phase.

Mr. Chairman, that concludes my statement. I will be happy to respond to any questions you or other Members of the Subcommittee may have.

For future questions regarding this testimony, please contact Louis Rodrigues, (202)512-4841. Individuals making key contributions to this testimony include Katherine Schinasi, Paul Francis, Michael Sullivan, Matthew Lea, and Katrina Taylor.

Defense Acquisition: Employing Best Practices Can Shape Better Weapon System Decisions (GAO/T-NSIAD-00-137, Apr. 26, 2000).

Best Practices: DOD Training Can Do More to Help Weapon System Program Implement Best Practices (GAO/NSIAD-99-206, Aug. 16, 1999).

Best Practices: Better Management of Technology Development Can Improve Weapon System Outcomes (GAO/NSIAD-99-162, July 30, 1999).

Defense Acquisitions: Best Commercial Practices Can Improve Program Outcomes (GAO/T-NSIAD-99-116, Mar. 17, 1999).

Defense Acquisition: Improved Program Outcomes Are Possible (GAO/T-NSIAD-98-123, Mar. 17, 1998).

Best Practices: DOD Can Help Suppliers Contribute More to Weapon System Programs (GAO/NSIAD-98-87, Mar. 17, 1998).

Best Practices: Successful Application to Weapon Acquisition Requires Changes in DOD’s Environment (GAO/NSIAD-98-56, Feb. 24, 1998).

Major Acquisitions: Significant Changes Underway in DOD’s Earned Value Management Process (GAO/NSIAD-97-108, May 5, 1997).

Best Practices: Commercial Quality Assurance Practices Offer Improvements for DOD (GAO/NSIAD-96-162, Aug. 26, 1996).

(707509)

Orders by Internet

For information on how to access GAO reports on the Internet, send an e-mail message with "info" in the body to:

Info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

Contact one:

Web site: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

1-800-424-5454 (automated answering system)