FAS | Military Analysis | GAO |||| Index | Search |


Defense Acquisition: Improved Program Outcomes Are Possible (Testimony, 03/18/98, GAO/T-NSIAD-98-123).

GAO discussed the issues facing the Department of Defense (DOD) in its
acquisition of weapon systems, related spare parts, and other goods and
services, focusing on: (1) a different approach to improving weapon
acquisition outcomes based on best commercial practices and an
understanding of the acquisition culture; (2) DOD's management of its
acquisition workforce and organization; (3) DOD's experience with
commercial pricing of spare parts; (4) the effectiveness of DOD's
mentor-protege pilot program; and (5) federal agencies' use of multiple
award task- and delivery-order contracts.

GAO noted that: (1) improved outcomes from the weapon system acquisition
process--that is, acquiring systems better, cheaper, and faster--are
possible if the incentives that drive behaviors are changed; (2) the
best commercial companies have found processes and decisionmaking
practices that are based on knowledge and focused on production to be
successful; (3) employing such processes and practices can improve
weapon acquisitions if DOD and Congress are able to foster an
environment that provides program managers with incentives for applying
best practices; (4) over the last several years, Congress has mandated
cuts in DOD's acquisition workforce and called DOD to submit plans to
streamline and restructure its acquisition organizations; (5) DOD has
been able to reduce the acquisition workforce, but it has had more
difficulty changing the structure that underlies decisionmaking in the
acquisition process; (6) for an increasing number of sole-source spare
parts, DOD is transitioning from a cost-based pricing environment to a
market-based or commercial pricing environment where price analysis is
the principal means used to negotiate the reasonableness of prices; (7)
regarding the sole-source, commercially-priced spare parts for which DOD
is the predominant buyer, some DOD contracting personnel expressed
concerns about: (a) how to determine whether the prices offered are fair
and reasonable; (b) future contract negotiations where recent cost-based
historical prices may not be available; and (c) the sometimes
conflicting pressures between obtaining fair and reasonable prices and
negotiating contracting time to meet customer needs and avoid backlogs;
(8) DOD has spent over $200 million on a mentor-protege program to
provide incentives for major DOD contractors to furnish disadvantaged
small business owners with assistance designed to enhance their
capabilities and increase their participation as suppliers under DOD,
other federal government, and commercial contracts; (9) however, DOD
lacks information needed to determine whether the program is effective;
(10) GAO's work on multiple award task- and delivery-order contracts
show that federal agencies are not consistently achieving competition
when placing orders against these contracts; (11) GAO found that fees
charged to agencies that place orders against another agency's contract
varied greatly; (12) weaknesses in agency accounting and management
systems prevented GAO from determining if the fees were reasonable; and
(13) GAO found that the use of multiple award contracts did not impair
small business' ability to compete for such contracts.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  T-NSIAD-98-123
     TITLE:  Defense Acquisition: Improved Program Outcomes Are Possible
      DATE:  03/18/98
   SUBJECT:  Federal agency reorganization
             Small business assistance
             Department of Defense contractors
             Defense cost control
             Military downsizing
             Spare parts
             Defense procurement
             Advanced weapons systems
IDENTIFIER:  F-22 Aircraft
             Joint Direct Attack Munition
             C-17 Aircraft
             Brilliant Anti-Armor Submunition
             DOD Pilot Mentor-Protege Program
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                    <[email protected]>                        **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Before the Subcommittee on Acquisition and Technology, Committee on
Armed Services, U.S.  Senate

For Release on Delivery
Expected at
9:30 a.m.,EST
Wednesday,
March 18, 1998

DEFENSE ACQUISITION - IMPROVED
PROGRAM OUTCOMES ARE POSSIBLE

Statement of Louis J.  Rodrigues, Director, Defense Acquisitions
Issues, National Security and International Affairs Division

GAO/T-NSIAD-98-123

GAO/NSIAD-98-123T

Defense Acquisition

(707339)


Abbreviations
=============================================================== ABBREV

  BAT - Brilliant Anti-armor Submunition
  CDR - critical design review
  DOD - Department of Defense
  JDAM - Joint Direct Attack Munitions

============================================================ Chapter 0

Mr.  Chairman and Members of the Subcommittee: 

We are pleased to be here today to discuss issues facing the
Department of Defense (DOD) in its acquisition of weapon systems,
related spare parts, and other goods and services.  In response to
the many changes that have been witnessed in the defense acquisition
environment over the last few years, DOD has begun broad-based
changes to its acquisition and contracting processes.  However,
weapon programs continue to have questionable requirements;
unrealistic cost, schedule, and performance estimates; and strategies
that begin production before adequate testing has been completed. 
This discussion of acquisition issues is well-timed, as DOD
implements plans to increase its procurement budget to $60 billion in
fiscal year 2001--a 40-percent increase over last fiscal year's
budget. 

My testimony focuses on a different approach to improving weapon
acquisition outcomes based on best commercial practices and an
understanding of the acquisition culture.  My testimony also includes
some observations on (1) DOD's management of its acquisition
workforce and organization, (2) DOD's experience with commercial
pricing of spare parts, (3) the effectiveness of DOD's mentor-protege
pilot program, and (4) federal agencies' use of multiple award task-
and delivery-order contracts. 


   RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:1

Improved outcomes from the weapon systems acquisition process--that
is, acquiring systems better, cheaper, and faster--are possible if
the incentives that drive behaviors are changed.  The best commercial
companies have found processes and decision-making practices that are
based on knowledge and focused on production to be successful. 
Employing such processes and practices can improve weapon
acquisitions if DOD and the Congress are able to foster an
environment that provides program managers with incentives for
applying best practices.  In our February and March 1998 reports, we
recommended that the Secretary of Defense separate technology
development from product development, send signals through decisions
on individual programs that encourage acquisition managers to
identify unknowns and ameliorate their risks early in development,
and develop and disseminate throughout DOD and the defense industry a
policy that promotes productive supplier relationships and their
importance to improving program outcomes.  In addition, we asked the
Congress to consider supporting these efforts through its funding and
oversight mechanisms. 

Over the last several years, the Congress has mandated cuts in DOD's
acquisition workforce and called on DOD to submit plans to streamline
and restructure its acquisition organizations.  DOD has been able to
reduce the acquisition workforce, but it has had more difficulty
changing the structure that underlies decision-making in the
acquisition process.  We have previously reported that the connection
among DOD's acquisition workforce, organizations, and outcomes of the
acquisition process should be considered in making changes. 

For an increasing number of sole-source spare parts, DOD is
transitioning from a cost-based pricing environment to a market-based
or commercial pricing environment where price analysis is the
principal means used to negotiate the reasonableness of prices. 
Regarding sole-source, commercially priced spare parts for which DOD
is the predominant buyer, some DOD contracting personnel expressed
concerns about (1) how to determine whether the prices offered are
fair and reasonable, (2) future contract negotiations where recent
cost-based historical prices may not be available, and (3) the
sometimes conflicting pressures between obtaining fair and reasonable
prices and negotiating contracts in time to meet customer needs and
avoid backlogs.  So far, our work indicates that while some
contractors are now offering commercial prices that are significantly
higher than DOD paid in the past, there are some questions about how
well contracting officers understand the basis of such prices. 

DOD has spent over $200 million on a mentor-protege program to
provide incentives for major DOD contractors (mentors) to furnish
disadvantaged small business owners (proteges) with assistance
designed to enhance their capabilities and increase their
participation as suppliers under DOD, other federal government, and
commercial contracts.  However, DOD lacks information needed to
determine whether the program is effective.  Strengthened performance
reviews could ensure that sufficient and reliable information is
available to assess the program's effectiveness. 

Our work on multiple award task- and delivery-order contracts shows
that federal agencies are not consistently achieving competition when
placing orders against these contracts.  We also found that the fees
charged to agencies that place orders on another agency's contract
varied greatly.  Weaknesses in agency accounting and management
systems prevented us from determining if the fees were reasonable. 
We found that the use of multiple award contracts did not impair
small business' ability to compete for such contracts. 


   THE NEED FOR A DIFFERENT
   APPROACH TO IMPROVING WEAPON
   ACQUISITION OUTCOMES
---------------------------------------------------------- Chapter 0:2

This morning we will not focus on individual weapon system problems;
rather, we will concentrate on the underlying reasons for such
problems and what can be done about them.  In two recent reports
prepared at the request of this Subcommittee,\1 we identified several
commercial practices in the areas of product development and supplier
relationships that have the potential for significantly improving
weapon system outcomes.  Of particular note are the different
incentives we found that operate in the two sectors and their primacy
in determining program management practices.  Simply put, practices
are adopted and persist because they work--they help programs succeed
in their environment.  Thus, the way to get lasting reform is to
realign the incentives of the weapons acquisition process with
desired program outcomes; specific practices can then show the way to
better outcomes.  Changing these incentives will take the efforts of
the Congress as well as DOD and the services, for all participants in
the acquisition process influence its incentives. 


--------------------
\1 Best Practices:  Successful Application to Weapon Acquisitions
Requires Changes in DOD's Environment (GAO/NSIAD-98-56, Feb.  24,
1998) and Best Practices:  DOD Can Help Suppliers Contribute More to
Weapon System Programs (GAO/NSIAD-98-87, Mar.  18, 1998). 


      CLEAR DIFFERENCES BETWEEN
      BEST COMMERCIAL PRACTICES
      AND WEAPON ACQUISITION
      PRACTICES EXIST
-------------------------------------------------------- Chapter 0:2.1

Our work on the transition of major products to production and on the
management of suppliers shows that best commercial practices and DOD
practices differ considerably.  Before discussing these differences,
let me clarify that we do not hold the view that commercial is good
and DOD is bad.  Not all commercial firms exhibit best practices and
the leading firms make their share of mistakes.  Also, we found some
promising practices in weapon systems that could have application to
other programs. 

In our transition to production work, we characterized knowledge on
product developments in terms of three junctures:  when a match is
made between the customer's requirements and the available
technology, when the product's design is determined to be capable of
meeting performance requirements, and when the product is determined
to be producible within cost, schedule, and quality targets.  For the
purposes of comparing commercial and DOD product developments, we
have characterized the points at which virtual certainty of each of
these aspects of a product is achieved as a "knowledge point." Figure
1 illustrates the three knowledge points and the differences between
the commercial and military product developments in terms of when
they attain knowledge. 

   Figure 1:  Comparison of Three
   Key Knowledge Points for
   Commercial and Military Product
   Developments

   (See figure in printed
   edition.)

Commercial firms gain more knowledge about a product's technology,
performance, and producibility much earlier in the product
development process than DOD.  Product development in commercial
ventures is a clearly defined undertaking for which firms insist on
having the technology in hand to meet customer requirements before
starting.  Once underway, these firms demand--and get--specific
knowledge about a new product before production begins.  The process
of discovery--the accumulation of knowledge and the elimination of
unknowns--is completed for the best commercial programs well ahead of
production.  Not having this knowledge when demanded constitutes a
risk the firms find unacceptable.  Immature or undeveloped technology
cannot meet these demands and is kept out of commercial product
development programs; this technology is managed separately until it
can meet the demands for product development. 

In contrast, DOD programs allow technology development to continue
into product development.  Consequently, the programs proceed with
much less knowledge--and thus more risk--about required technologies,
design capability, and producibility.  The programs' discovery
process persists much longer, even after the start of production. 
Not having the same level of knowledge as commercial firms explains
much of the turbulence in DOD program outcomes as the transition to
production is made.  It is a predictable consequence that can be
forecast early by the use of knowledge points or other metrics.  It
is complicated by the fact that although DOD accepts more unknowns on
its programs than commercial firms, it understates the risks present. 


         KNOWLEDGE POINT 1: 
         CUSTOMER REQUIREMENTS AND
         TECHNOLOGICAL CAPABILITY
         ARE MATCHED
------------------------------------------------------ Chapter 0:2.1.1

To minimize the amount of technology development that occurs during
product development, the companies we visited employ a disciplined
process to match requirements with technological capability before
the product development process begins.  This process is grounded in
production realities that demand proof that the technology will work
and can be produced at an acceptable cost, on schedule, and with high
quality.  The companies bring proven technological knowledge to the
requirements process in the form of current, high-fidelity
information from predecessor programs, people with first-hand
experience on those programs, and new technologies deemed mature as a
result of having "graduated" from a disciplined technology
development and screening process.  In addition, they communicate
extensively with customers to match their wants and needs to the
firms' available technology and ability to manufacture an appropriate
product.  They do not stray far from their technological foundation. 

We found examples of best commercial practices for matching
requirements to technology at Boeing, Hughes, and Ford.  Boeing
communicated with the airlines to set achievable requirements for the
777-200 airplane and tested the design early.  Boeing also applied
lessons learned from past programs to (1) ensure that technology was
mature before the 777 was launched\2 and (2) eliminate additional
requirements once the development program began.  Hughes used a
technology development process that graduated new technologies from
concept into a product development program, enabling the firm to make
what it saw as quantum performance increases with mature technologies
on its new satellite, the HS-702.  Ford uses its technology
deployment process and "Wall of Invention" to separate immature
technology from a new product's development. 

DOD programs did not attain a match between technology and
requirements at the time of launch.  DOD accepted varying--but
consistently higher--degrees of technological risk on the four
programs we reviewed.  Some examples from the F-22 illustrate that a
match between requirements and technology is still not certain in
that program.  The F-22 program includes 10 newly developed
derivatives of existing materials that are important to its
low-observability feature.  The performance and maintainability of
these materials will not be completely verified until 2 years after
production begins.  Similarly, it will be after production before it
is certain that the avionics software, which features a level of
integration not previously achieved in a fighter, is a match for the
performance requirements.  The F-22 engines have many advanced
features to meet aggressive performance requirements and program
officials acknowledge that it is still unclear whether the engines
will meet all of the requirements.  Leading commercial firms do not
do this--they do not tolerate basic unknowns about the performance of
a new product to persist this long.  In fact, resolving these kinds
of unknowns is a precursor to starting a commercial product
development. 

Another example that illustrates the different knowledge standards
are the commercial and DOD decisions made on a lightweight aircraft
material--aluminum lithium.  Boeing had initially decided to use the
alloy on its 777-200 aircraft but rejected its use early in
development because it was expensive, its manufacturing processes
were not well understood, and its availability was limited.  It was
willing to pay a weight penalty rather than accept the unknowns
associated with aluminum lithium.  DOD accepted these risks and used
the alloy on the C-17 aircraft.  The first production aircraft
contained 2,200 pounds of the alloy.  Its application proved
unsuccessful as some of its unknowns became problems.  Consequently,
the decision was made to discontinue aluminum lithium and it will be
phased out of the program by the 51st production airplane. 


--------------------
\2 We define program launch or start as the point at which
organizations define a product's performance, cost, and schedule
estimates and commit to making the financial investment needed to
complete development and bring the product into production. 


         KNOWLEDGE POINT 2:  THE
         DESIGN WILL PERFORM AS
         REQUIRED
------------------------------------------------------ Chapter 0:2.1.2

The commercial firms we visited achieved near certainty that their
product designs would meet customer requirements and had gone a long
way to ensure that the product could be produced before the halfway
point of product development.  Both DOD and commercial firms hold a
critical design review (CDR) to review engineering drawings, confirm
the design is mature, and "freeze" it to minimize changes in the
future.  The completion of engineering drawings and their release to
manufacturing organizations signify that program managers are
confident in their knowledge that the design performs acceptably and
is mature.  The drawings are critical to documenting this knowledge
because they are not only precision schematics of the entire product
and all of its component parts--they also reflect the results of
testing and simulation and describe the materials and manufacturing
processes to be used to make each component.  Both DOD and commercial
companies consider the design to be essentially complete when about
90 percent of the engineering drawings are completed. 
Figure 2 compares what knowledge, in the form of released drawings,
was in hand at the time of the critical design review for the
commercial and DOD programs we reviewed. 

   Figure 2:  Comparison of When
   Commercial and DOD Programs
   Achieve Knowledge About Their
   Product's Design

   (See figure in printed
   edition.)

Commercial firms such as Boeing and Hughes told us they typically had
over 90 percent of these drawings available for the CDR.  Boeing
began releasing engineering drawings when product development began
in 1990 and completed the release process in 1992, less than 2 years
later.  Once the CDR was complete, Boeing strictly enforced the
design freeze for the 777-200.  For example, Boeing incorporated a
customer requirement to include folding wingtips, along with the
supporting bulkheads, into the
777 design at a cost of nearly $40 million.  Later, the customer
decided the folding wingtips were not necessary; however, Boeing left
the bulkheads in the wings anyway because all of the engineering
drawings were completed and the risk of introducing changes, even
though the changes would have saved weight, was considered too high
relative to cost and schedule targets. 

The C-17 and the F-22 programs had less knowledge--in the form of
test results or engineering drawings--about their designs than
commercial companies did at the time they held their CDRs.  The
programs did not get or were not projected to get to the same level
of completion on the drawings until later in the development cycle,
which placed greater reliance on the lesser information available at
the time of the review.  Specifically, the C-17 program had less than
60 percent and the
F-22 program less than one-third of the drawings available for the
CDR.  Over one-fifth of the C-17's drawings became available after
production began, and the aircraft experienced a number of problems
in production as difficulties with the design were worked out. 
Several key technologies are still unproven on the F-22, and some
will not be proven out until after
40 aircraft have entered production.  Nonetheless, the risks of
proceeding with the rest of development as planned at the time of the
CDRs for both programs were deemed acceptable. 

Even though it is still too early to predict outcomes on the AIM-9X
missile and the Joint Direct Attack Munitions (JDAM) programs, their
prospects appear promising because they have chosen mostly proven
technology from existing programs to achieve performance
requirements. 


         KNOWLEDGE POINT 3: 
         PRODUCTION UNITS WILL
         MEET COST, QUALITY, AND
         SCHEDULE OBJECTIVES
------------------------------------------------------ Chapter 0:2.1.3

The companies we visited reached the point at which they knew that
manufacturing processes would produce a new product conforming to
cost, quality, and schedule targets before they began fabricating
production articles.  Reaching this point meant more than knowing the
product could be manufactured; it meant that all key processes were
under control, such that the quality, volume, and cost of their
output were proven and acceptable.  As indicated in figure 3, the DOD
programs demanded less proof of the design's producibility before the
product transitions to production. 

   Figure 3:  Comparison of When
   Commercial and DOD Programs
   Achieve Knowledge That
   Processes Can Produce a Product

   (See figure in printed
   edition.)

The commercial firms relied on known manufacturing processes and
statistical process control data to achieve this knowledge early and,
in fact, had all their key processes under statistical control when
production began.  The ability to establish control for key processes
before production began was the culmination of all the practices
employed to identify and reduce risk.  All of the companies we
visited agreed that knowledge about technology and design up front in
the process makes the control of processes possible and the
transition to production smooth. 

The C-17 program began production in 1989 and still has less than 13
percent of its key manufacturing processes in control.  The
F-22 program is currently faring better than the C-17:  The
contractor believes it has almost 40 percent of its key manufacturing
processes in control, 2 years before production is scheduled to
begin.  However, the program does not plan to have all key processes
in control until about
4 years into production.  Both programs experienced basic
producibility problems that were not discovered until late in
development or early in production.  Problems occurred despite
completing production readiness reviews that were intended to reduce
producibility problems and having exit criteria to ensure that risks
were acceptable and enough knowledge had been gained to enter the
next development phase. 

The C-17 program discovered major design changes of the wings, flaps,
and slats were required after the critical design review.  These
changes caused costly changes to processes; forced the manufacturers
to develop workaround plans; and resulted in high rates of scrap,
rework, and repair.  The production preparations for the F-22
illustrate the limitation of a review mechanism when a substantial
amount of knowledge is unattained.  The initial production readiness
review, held in 1995 when only about one-third of the engineering
drawings were released, did not report any high risks in
manufacturing or producibility.  In 1996, an independent team
mandated by the Air Force reviewed the program and discovered
numerous manufacturing and producibility problems, such as
underestimated complexity in manufacturing processes, understated
labor requirements, immature definition of avionics flight test
requirements, and concerns about software integration. 


         MANAGEMENT OF SUPPLIERS
------------------------------------------------------ Chapter 0:2.1.4

Leading commercial companies have found that more cooperative
business relationships with suppliers have led to lower costs, higher
quality, and shorter cycle times.  In both commercial and defense
products, suppliers account for much, if not most, of the product
content and technical innovation.  In our review of supplier
relationships, we also found key differences between best commercial
practices and DOD practices.  The best practices of commercial firms
recognized as industry leaders in the area of supplier relationships
can be aggregated into four traits: 

  -- providing the central support necessary to optimize supplier
     relations, which were seen as essential to maximizing product
     success;

  -- implementing a rigorous supplier selection process, which
     created a manageable pool of strong suppliers;

  -- creating channels for open communication and continuous
     assessment of performance for both customer and supplier; and

  -- creating an environment whereby the suppliers also benefited
     from superior performance. 

The leading commercial firms go beyond simple supplier relationships
that are limited to the purchase of goods and services in return for
payment.  Their relationships evolved to the sharing of information
and interaction on a variety of business functions in a joint effort
to make a better quality product, more quickly, and less expensively. 
Both the firm responsible for the complete product and its suppliers
benefited from the process.  Thus, we see the four traits as the
components of a self-sustaining system, shown in figure 4. 

   Figure 4:  System of Four
   Traits Seen in Commercial Best
   Practices

   (See figure in printed
   edition.)

We discussed supplier relationships with several defense prime
contractors and did detailed work on the Brilliant Anti-armor
Submunition program, referred to as BAT, and the JDAM program.  We
found that in a more traditional program, like BAT, the four traits
do not comprise as powerful a system as is formed by the best
commercial practices.  While a number of the practices that make up
the middle two traits have been adopted, their impact on the BAT
program was blunted by weaknesses in central support and providing a
mutually rewarding environment.  The commitment of the prime
contractor to improve supplier relationships was not perceived by
some key suppliers as having been much more than procedural changes. 
Part of the reason is that although DOD shares responsibility for
determining what is important in managing an individual program, its
traditional approach has been to maintain an "arm's length"
relationship with prime contractors and have little involvement with
suppliers.  On the JDAM, DOD was able to create a better environment
for fostering mutual benefits between defense prime contractors and
their suppliers. 


      DIFFERENCES IN PRACTICES
      REFLECT DIFFERENT INCENTIVES
-------------------------------------------------------- Chapter 0:2.2

The differences in the practices employed by the leading commercial
firms and DOD are not necessarily explainable by differences in
tools, techniques, or talent.  Rather, the differences in the actual
practices reflect the different demands imposed on programs by the
culture or environment in which they were managed.  Indeed, the way
success and failure are defined for commercial and defense product
developments differs considerably, which creates a different set of
incentives and different behaviors from the people managing the
programs.  Specific practices take root and are sustained because
they help a program succeed in its environment.  In this sense,
practices are adopted because they work--not because they are
textbook solutions.  In our transition to production review, we
observed that, with the possible exception of having more experience
with repeated product developments than a DOD program manager,
commercial program managers were not better or more ethical than
their DOD counterparts.  On supplier relations, we noted that it is a
major undertaking for a firm to commit the resources to implement an
active supplier policy.  Such a commitment is not based on altruism
or management theory; rather, the commitment comes from the desire to
maintain a competitive edge in preserving or increasing a firm's
market share. 


         COMMERCIAL PRACTICES ARE
         DRIVEN BY THE CUSTOMER'S
         ACCEPTANCE OF THE
         FINISHED PRODUCT
------------------------------------------------------ Chapter 0:2.2.1

The commercial firms we contacted launch a product development
program only when a solid business case can be made.  The business
case basically revolves around the ability to produce a product that
will sell well enough to make an acceptable return on investment. 
The point of sale occurs after product development is complete;
program success is determined in production when the customer buys
the finished product.  If the firm has not made a sound business case
or has been unable to deliver on one or more of the business case
factors, it faces a very real prospect of failure in the form of the
customer "walking away." Production is a dominant concern throughout
the product development process and forces discipline and tradeoffs
in the design process.  This environment encourages realistic
assessments of risks and costs; doing otherwise would threaten the
business case and invite failure.  For the same reasons, the
environment places a high value on knowledge for making decisions. 
Incentives favor identifying unknowns early, designating them an
appropriate high risk, and aggressively eliminating them.  Practices,
such as achieving statistical process control before production, are
adopted because they help ensure success. 

Boeing described the business case for the 777-200 product
development as a "money wheel" that must be balanced across all of
its factors.  These factors include a market opportunity, a product
whose technical features can satisfy the market, available investment
capital, a cycle time short enough to get the product to market on
time, and a unit production cost that will yield an acceptable return
on investment.  Boeing informed us that if any factor gets out of
line, either through estimating errors or changing conditions, the
"wheel" will not turn, and profitability--and perhaps corporate
reputation--could be lost.  The program manager is judged by these
standards, unlike in DOD. 

Once a company decides to launch a product development, strong
incentives--both positive and negative--serve to keep the programs on
track.  To meet market demands, leading commercial companies build
relatively short cycle times into decisions to begin a product's
development.  Boeing's 777-200 went to production less than 5 years
after development began, Hughes' HS-702 took about 26 months, and
Chrysler developed its Dodge Durango sport utility vehicle in 24
months.  These short timeframes make the day of reckoning--sale of
the produced item to the customer--close at hand.  Consequently,
production--on time, at rate, at cost, and with quality--looms as a
near-term reality that continues to greatly influence subsequent
design and configuration decisions within the framework of the
business case.  The incentives that operate in the commercial
environment encourage program managers to want risks identified
early, be intolerant of unknowns, and not rely on testing as the main
vehicle for discovering the performance characteristics of the
product.  By protecting the business case as the key to success,
program managers are conservative in their estimates and aggressive
in risk reduction.  Ultimately, preserving the business case enables
them to say "no" to pressures to accept risks or unknowns. 


         DOD PRACTICES REFLECT THE
         NEED TO SUCCEED IN
         FUNDING AND MANAGING THE
         DEVELOPMENT EFFORT
------------------------------------------------------ Chapter 0:2.2.2

The business case for a major weapon system is quite different.  It
is characterized by a stated need for a military capability; a
proposed weapon system for which the demands of a successful launch
dictate optimistic technological, cost, and schedule estimates; and a
development effort for which candor about progress is curbed by the
competition for continued funding. 

Traditionally, needs for new weapons have been generated by
individual branches within each service.  Once a need has been
established, a product development vying for launch faces intense
competition for initial funding.  DOD typically defines and launches
a program years earlier in the process than a commercial product
development, and thus, the case for the product is made when much
less is known about technology, cost, and schedule.  The knowledge
required to make the business case to launch a commercial product
development is generally not available for a DOD program until well
into the engineering and manufacturing development phase.  In a very
real sense, the point of sale begins much earlier on a DOD program
and recurs more often because the customer (DOD and the Congress)
pays for the product on an annual installment basis from program
launch.  Success, then, for most of the product development cycle, is
measured in terms of ability to secure the next installment.  Because
this approval must be won every year, it creates incentives to make
the program's case look attractive. 

The competition for funding at the time of launch encourages aspiring
DOD programs to include performance features and design
characteristics that rely on immature technologies.  Untempered by
knowledge to the contrary, the risks associated with these
technologies are deemed acceptable.  Because production can be 15
years from the launch decision, it is difficult for production
realities and concerns to exert as much influence on a DOD product
development as they do on commercial products.  Instead, design
features and performance are more dominant.  More unknowns are
accepted on a DOD program, and their attendant risks are often
understated.  This combination, which can be devastating to a
commercial business case, can help a weapon system program get
launched and survive. 

Other pressures on DOD programs at launch make tough demands for
knowledge that does not yet exist.  A product development deemed
worthy cannot be launched unless development and production funding
is available over the right time period.  The product's development
and production cost, as well as timing, must fall within available
funding.  Because DOD relies largely on forecasts of cost, schedule,
and performance that are comparatively soft at this stage, funding
competition encourages the cost and schedule estimate to be squeezed
into profiles of available funding.  Additional requirements, such as
high reliability and ease of maintenance, serve to make the fit even
tighter. 

As a product development proceeds in DOD, its success is still
measured in terms of the funding it receives.  Success translates
into getting the funding request approved each year; failure can mean
anything from a significant funding cut to cancellation.  This view
of success is reinforced by the fact that, unlike commercial
programs, DOD programs do not receive full corporate support
throughout development.  Individual programs face scrutiny by service
executives, the Office of the Secretary of Defense, independent cost
estimating and test agencies, audit agencies, and several committees
and subcommittees of the Congress.  Given this amount of competition
and oversight, the detection of a problem on an individual program
makes that program vulnerable to criticism and possible loss of
funding support.  Ironically, it is these same pressures that
encourage overreaching at the time of program launch.  By the time a
DOD program makes it through this development process and begins
production, the customer is deeply invested and unlikely to walk
away.  Thus, success, in terms of program continuance, is
substantially ensured before end items are produced. 

The pressures and incentives in the DOD environment explain why the
behaviors of managers and other sponsors of product developments
differ from those in commercial programs.  According to a 1994 study
done for the Under Secretary of Defense for Acquisition, government
program managers found their formal role of objective program
management at odds with their informal role of program advocates.\3
According to the study: 

     "A feeling of responsibility for program advocacy appears to be
     the primary factor causing government managers to search
     aggressively and optimistically for good news relating to their
     programs, and to avoid bad news, even when it means discrediting
     conventional management tools that forecast significant negative
     deviations from plan."

In this environment, risks in the form of ambitious technology
advancements and tight cost and schedule estimates are accepted as
necessary for a successful launch.  Problems or indications that the
estimates are decaying do not help sustain the program in subsequent
years, and thus, their admission is implicitly discouraged.  An
optimistic production cost estimate makes it easier to launch a
product development and sustain annual approval; admission that costs
are likely to be higher could invite failure.  There are few rewards
for discovering and recognizing potential problems early in the DOD
product development.  For commercial product developments, an
optimistic production cost estimate will mean failure of sales or
profit; admission of cost increases early invites aggressive
problem-solving behaviors to restore the business case.  The behavior
of tolerating unknowns and not assigning them the same risk value as
in the commercial environment is rational in the DOD environment
because there is little incentive to admit to high risks before it is
absolutely necessary, as long as the resulting estimates are accepted
by DOD and the Congress.  In fact, admitting risk may doom the
program. 

Behaviors toward testing follow a similar logic.  On commercial
product developments, much more is known about the product's
performance at the beginning of development.  Testing is used to
confirm knowledge and identify weaknesses or limits in the product. 
It is consistent with a firm's anxiety to eliminate unknowns to
preclude failure in production.  DOD product developments are much
more dependent on testing to discover technical performance
characteristics and answer the question of whether the product will
work.  DOD tests serve more than the purpose of discovering or
confirming performance characteristics--they are examinations on
which the program must get good grades or face failure in the form of
withdrawal of support.  Good test results can help a program, whereas
negative test results are equated with failure.  Unknowns, then,
present a safer course of action; if testing does not occur until
late in the product development, forecasts of product performance
will serve as the best information available. 


--------------------
\3 Critical Issues in the Defense Acquisition Culture, Defense
Systems Management College, Dec.  1994. 


         A REWARDING ENVIRONMENT
         IS KEY TO FOSTERING THE
         BEST SUPPLIER
         RELATIONSHIPS
------------------------------------------------------ Chapter 0:2.2.3

In our work on supplier relationships, we also found that the
environment DOD created on an individual program greatly affected the
contribution suppliers made to the outcomes of the program.  The best
supplier practices were sustained when a commercial firm created an
environment in which it became an attractive customer.  Firms did
this by not only rewarding superior suppliers with future business
but by building partnerships, allowing top suppliers to participate
in product planning and design, sharing business plans, and relaxing
the procedures for doing business together.  In turn, the key
suppliers were willing to go the extra mile, commit their own
resources to enhance prospects for future business, and comply with
the rigor that the source selection and evaluation mechanisms
demanded.  The suppliers' responses improved product output and
reinforced the initial commitment that the product developer made to
strengthening supplier relationships. 

Mutual trust--earned through action--was essential to creating this
environment.  For example, Chrysler's relationships with some
suppliers had evolved to the point that it no longer needed to make
large investments in some key technology areas because of the
relationships it had developed with some suppliers.  Instead, the
suppliers made the technology investment themselves and had enough
confidence in their relationship with Chrysler that they did not fear
the long-term commitment that this entailed.  For its part, Chrysler
trusted the suppliers to make the investments that would keep their
vehicles competitive. 

On weapon system programs, DOD shares responsibility with the prime
contractors for the acquisition policies that shape the suppliers'
environment.  Thus, the role it plays on individual programs has a
direct bearing on the sophistication of supplier relationships and
the success of best supplier practices.  The supplier relationships
on the BAT program reflect DOD's traditional role of distancing
itself from suppliers.  This role can be traced, in part, to the fact
that DOD has not articulated a particular supplier policy to guide
program managers.  By default, DOD's concerns over interfering with
the contractual relationship between the prime and a supplier have
encouraged an arms-length approach to suppliers by managers. 

Nonetheless, some key BAT suppliers did not see their environment as
conducive to such relationships.  They viewed their role as only
complying with the design requirements handed down to them by the
upper-tier firms.  They believed that attempts to do more--such as
offer design suggestions or make long-term investments--would not
reap benefits.  Some suppliers believed no consideration was given to
their years of working together when it came to the low-rate
production contract proposal. 

On the JDAM program, DOD was much more proactive and involved with
the suppliers.  Its status as a pilot program that was afforded early
statutory and regulatory relief helped support the program office's
involvement in seeing that best supplier practices were used.  As a
result, high performing suppliers were selected, all tiers of
suppliers participated in meeting the program's priorities, and
long-term benefits were offered to the prime contractor and its
suppliers for good performance.  The ultimate success of this
approach in producing a weapon that will perform as required remains
to be seen.  Nonetheless, suppliers praised the approach for the
relationships it fostered. 


      A BROADER PERSPECTIVE OF THE
      ACQUISITION CULTURE IS A
      PRECURSOR TO REFORM
-------------------------------------------------------- Chapter 0:2.3

Our findings relative to the different environments that commercial
product developments and weapon system programs operate in are
consistent with the results of a study we completed in 1992.\4 At
that time, we took a retrospective look at the previous 15 years of
our reviews of weapon systems and the acquisition process to identify
the underlying factors that contributed to recurring acquisition
difficulties.  We concluded that if changes in the acquisition of
weapons were to be of a lasting nature, acquisition problems needed
to be looked at from another perspective--as the consequences of a
way of acquiring weapons that had become deeply rooted over the
years.  Those findings have since been reinforced by the 1994 Defense
Systems Management College study, which included the participation of
80 experienced government and defense industry acquisition managers. 

The acquisition culture can be defined as the collective behavior of
the various participants\5 in the acquisition process and the forces
that motivate their behavior.  In fact, the process may be more
realistically portrayed as the interaction of its participants than
the methodological procedure depicted on paper.  This culture has
evolved as the acquisition process has become a vehicle for meeting
the diverse needs of its participants.  This depiction of the
acquisition culture does not stem from a pejorative view of
individual participants or organizations.  Rather, they do what they
believe is right given the pressures they face.  The difficulty lies
in the fact that there is no consensus on what is right.  In the
absence of such a consensus, the acquisition process serves to
satisfy the diverse needs of its participants within the umbrella of
providing U.S.  forces with the best weaponry.  In so doing, the
incentives of the process--both positive and negative--favor
maximizing programs. 

The acquisition culture offers an explanation for why problems with
weapon acquisitions persist despite numerous attempts at reform. 
Reforms, in essence, have championed sound management practices, such
as realistic estimating, thorough testing, and accurate reporting
without necessarily affecting why they did not happen in the first
place.  For example, there have been recommendations aimed at
improving the realism of cost estimates, but these are hard to
implement when the acquisition process itself does not reward
realism.  The same can be said about streamlining organizational
structure to increase the flow of realistic information, advocating a
"fly before buy" approach to testing weapons before major production
commitments are made, and baselining programs in exchange for stable
funding and minimal interference. 

DOD has embarked on several initiatives that draw lessons from
commercial practices, such as cost as an independent variable,
integrated product teams, use of past performance data, and
performance specifications.  These initiatives are based on best
commercial practices and are not as compliance-oriented as some past
reforms.  They could have a positive effect on the outcomes of weapon
system programs if the environment for launching programs and
appraising risks can be changed to provide the right incentives.  In
our work on supplier relationships, we found that attempts to apply
techniques, such as integrated product teams, were not seen by some
suppliers as being effective because their environment had not
changed.  On the JDAM program, where conditions allowed for a
somewhat unique approach, reform initiatives were more effective. 
Ultimately, if incentives still exist to launch a program with an
overly optimistic cost, schedule, and performance baseline, then cost
as an independent variable and integrated product teams will not
succeed. 


--------------------
\4 Weapons Acquisition:  A Rare Opportunity for Lasting Change
(GAO/NSIAD-93-15, Dec.  1992). 

\5 Acquisition process participants include the military services,
the Offices of the Secretary of Defense and the Joint Chiefs of
Staff, independent oversight organizations, contractors, professional
associations, and the Congress. 


      CHARTING A COURSE OF ACTION
      FOR BETTER OUTCOMES
-------------------------------------------------------- Chapter 0:2.4

We believe the acquisition reforms underway have a sound basis and
have the potential for improving the outcomes of weapon systems.  We
also believe the current leadership is genuinely committed to making
a difference in the status quo.  However, we are convinced that
lasting improvements in the outcomes of acquisition programs will not
be realized unless the incentives that drive behaviors in the
acquisition process are changed.  The first steps to making such
changes involve decisionmakers: 

  -- Accepting collective responsibility for the incentives that
     drive the acquisition process, rather than placing blame on
     individuals or procedure. 

  -- Agreeing that a cultural focus on acquisition reform--which
     addresses the "why" or the incentives that affect behavior--is
     needed to complement the traditional focus on the "how" (process
     and control) and the "who" (organization). 

  -- Accepting that it is the actions taken and the decisions made on
     individual programs that communicate the broader message of
     "what will work" to others in the process.  Within the current
     process, circumstances can usually make a compelling case for
     taking actions on individual programs that would otherwise
     contradict sound principles. 

Against this backdrop, it is possible for acquisition reforms and
commercial practices for gaining knowledge and assessing risks to
help produce better outcomes on DOD acquisitions.  For such practices
to work, however, the knowledge they produce must help a DOD program
succeed in its environment.  The Congress and DOD can help redefine
success in weapons acquisition to make the acquisition environment
conducive to such practices.  At least two factors are critical to
fostering such an environment.  First, program launch decisions must
be depressurized.  That is, they must be relieved of the need to
overpromise on performance and resource estimates.  This may require
altering how decisions are made on determining what weapons are
needed.  Second, once a program is underway, it must be made
acceptable for program managers to identify unknowns as high risks so
that they can be aggressively worked on earlier in development. 
Following are specific actions we have reported on that are in line
with addressing the deeper causes of acquisition problems. 

Recommendations we have made to the Secretary of Defense include: 

  -- Redefine the point for launching programs as the point at which
     technology development ends and product development begins. 

  -- On individual program decisions, send the signals that create
     incentives for acquisition managers to identify unknowns and
     ameliorate their risks early in development. 

  -- Develop a policy that promotes productive supplier relationships
     and emphasizes the importance of suppliers in improving
     acquisition outcomes and communicate this policy throughout the
     acquisition workforce and the defense industry through training
     and other means. 

Matters we have asked the Congress to consider include: 

  -- Support the Secretary of Defense's efforts to create the right
     environment through changes to the acquisition process that
     provide program managers clear incentives for gaining sufficient
     knowledge at key points in weapon acquisition programs. 

  -- Provide the funds needed to manage technology development
     efforts outside the bounds of individual weapon system programs,
     if the Secretary of Defense takes steps to separate technology
     development from product development. 

  -- Help create the right incentives on individual programs by
     favorably considering DOD funding requests to mitigate high
     risks early in a program and cautiously considering late
     requests for funds to resolve problems that should have been
     addressed earlier. 

DOD has concurred with these recommendations and says that it will
take action.  We are encouraged by this and hope to see these actions
reflected and supported in funding and other decisions made on
individual programs. 


   LINKING WORKFORCE REDUCTIONS
   WITH BETTER PROGRAM OUTCOMES
---------------------------------------------------------- Chapter 0:3

Over the last several years, the Congress has mandated cuts in DOD's
acquisition workforce and called on DOD to submit plans to streamline
and restructure its acquisition organizations.  With DOD's desire to
increase procurement funding partially through savings from
infrastructure reductions and the need to have better weapon system
program outcomes, such actions are called for. 

In defense authorization acts for fiscal years 1996, 1997, and 1998,
DOD was mandated to cut people from its acquisition workforce.  We
have reported that DOD most likely will achieve the mandated cuts of
25 percent over 5 years in the acquisition workforce, although the
cuts will be offset somewhat by individuals moving elsewhere in DOD
and additional functions being performed by contractors.  These
mandates allow the Secretary of Defense wide latitude to determine
how and how much more to cut. 

Legislative mandates for significant streamlining and restructuring
actions have also been placed on DOD.  These actions have had much
less success.  Section 277 of the National Defense Authorization Act
for Fiscal Year 1996 required DOD to develop a 5-year plan to
restructure and consolidate laboratory and test and evaluation
centers.  DOD developed the Vision 21 plan, but its implementation
has been put on hold pending decisions on another base realignment
and closure round.  Section 906 of the same act required the
Secretary to submit a plan on how to restructure current DOD
acquisition organizations.  DOD's response did not assess specific
streamlining and restructuring options but rather concluded that its
efforts had been sufficient because the workforce had been reduced. 
In section 912 of the National Defense Authorization Act for Fiscal
Year 1998, DOD was again required to address streamlining and
consolidating acquisition organizations, this time by submitting an
implementation plan to do so.  The next report is due on April 1,
1998. 

DOD has several related actions underway to redefine and restructure
the acquisition workforce.  In redefining the acquisition workforce,
we encourage DOD to do so in a way that enhances its ability to: 

  -- effectively deliver the training needed to operate successfully
     in today's unfamiliar environment;

  -- understand the cost of acquisition functions, whether they be
     performed by government employees or contractors; and

  -- accurately identify overlap and duplicative functions between
     and among the services in setting requirements for, developing
     and procuring, and supporting weapon systems. 

As for restructuring the workforce, DOD has taken actions to reshape
the Office of the Secretary of Defense in the Secretary's Defense
Reform Initiative to focus on corporate level tasks.  The Office's
responsibilities must be balanced with the responsibilities of the
individual services for equipping the forces to ensure that
organizations at all levels support incentives needed to rationalize
weapon system requirements and acquire them cheaper, better, and
faster.  Such issues have yet to be addressed.  As the congressional
directions indicate, the size, organization, and capability of the
workforce DOD relies on to buy equipment is important in determining
the outcomes of acquisition programs.  And as DOD's actions
illustrate, reducing workforce levels is easier than changing the
structure that underlies decision-making in the acquisition process. 
Whether DOD's acquisition workforce and organizations drive the
outcomes of the acquisition process or are a reflection of them, they
are connected in a way that should be considered in contemplating
solutions. 


   COMMERCIAL PRICING OF
   SOLE-SOURCE SPARE PARTS
---------------------------------------------------------- Chapter 0:4

Mr.  Chairman, I would now like to discuss commercial pricing of
sole-source spare parts.  At this Subcommittee's request, we have
begun to examine how and with what information and guidance, DOD
contracting officers determine whether prices for sole-source
commercial items are fair and reasonable in an environment where
there may not be sufficient competitive market forces to control
prices.  This universe is a small portion of the $100-plus billion
that DOD spends on contracting each year.  Based on fiscal year 1997
DOD contracting data, about 9 percent of DOD's contracting dollars,
or $10.8 billion, went for commercial item purchases, and about 2
percent, or $2.7 billion, represented commercial items purchased
without the benefits of competition.  We examined selected
sole-source contracts to compare commercial prices with prices
previously paid by DOD.  While our work is not complete, we have been
asked to share our preliminary observations.  These observations are
based principally on our discussions with contracting officers and
other contracting personnel, and on our review of contract files at
three Defense Supply Center buying activities, one Air Force buying
activity, and one Navy buying activity. 

For an increasing number of sole-source spare parts, DOD is
transitioning from a cost-based pricing environment that relied on
certified cost or pricing data to establish prices to a market-based
or commercial pricing environment where price analysis is the
principal means used to determine the reasonableness of prices.  It
is premature, at this time, to draw conclusions about DOD's ability
to respond to this new environment. 


      VIEWS OF CONTRACTING
      PERSONNEL
-------------------------------------------------------- Chapter 0:4.1

In our discussions, some DOD contracting officers expressed concern
about the breadth of the commercial item definition and are
struggling with how to determine whether commercial prices offered by
contractors for sole-source items are fair and reasonable,
particularly those that are predominately or only sold to DOD.  While
this concern about the commercial pricing of sole-source spare parts
exists today, some contracting personnel expressed more concern about
future commercial price negotiations.  This is because contracting
officers now have recent cost-based prices to evaluate current
prices, and they rely heavily on this data to establish a negotiating
benchmark for determining price reasonableness.  For the future, they
expressed concern about their ability to negotiate fair and
reasonable prices without the benefit of recent cost-based prices. 

Another point that came out in our discussions was the sometimes
conflicting pressures on contracting officers to obtain fair and
reasonable prices for sole-source commercial items and to get
contracts negotiated so that purchases can be made to meet the
customer's needs and avoid backlogs.  In fact, one buying unit
provided us information showing that unfilled backorders for aircraft
and ship spare parts are increasing.  According to the buying unit's
data, unfilled backorders for spare parts increased from around
12,000 in October 1996 to around 29,000 in December 1997.  The
backlog, according to contracting officials, is caused in large part
by the increasing difficulty of negotiating prices in the new
commercial pricing environment.  At this point in our work, we do not
know the impact this backlog is having on aircraft or ship readiness,
or how widespread the problem may be. 


      REVIEW OF SELECTED
      SOLE-SOURCE COMMERCIAL ITEM
      CONTRACTS
-------------------------------------------------------- Chapter 0:4.2

Our review of selected sole-source contracts indicates that some
contractors are now offering commercial prices significantly higher
than DOD paid in the past.  While these price differences, discussed
below, are significant, DOD officials and others emphasized that
comparing current commercial prices with historical prices alone may
not tell the full story, and that one must analyze the makeup of the
price, the terms and conditions of the contract, and compare services
offered under previous contracts to those to be provided under the
new contract.  Clearly, DOD contracting officers must understand the
basis of commercial prices.  Our work raises concerns about how well
contracting officers are responding to higher commercial prices. 

For example, the Navy purchased material used to repair naval
aircraft from a contractor for under $100 a yard prior to its being
offered as a commercial item.  The manufacturer for this material was
acquired by another company, and the acquiring company raised the
price to $800 a yard, which represented a discount from its $950
catalog price.  The Navy purchased about $1.1 million of this
material at the $800 price.  According to the Navy contracting
officer, the government is the only buyer of this material and it
requested the contractor to provide cost information supporting the
higher price.  The contractor declined to provide cost information
and informed the Navy that the offered price was based on the
company's financial goals, not on its costs.  The contractor referred
the contracting officer to its price catalog.  The reasonableness of
the
$800 price could not be tested against commercial sales since there
were none.  According to the contracting officer, even though he
considered the price high, the Navy needed the material to solve a
readiness problem. 

In another example, on three different occasions from 1994 to 1996,
the Air Force purchased from a sole-source contractor an electronic
engine control unit as a spare part.  Because this component was
bought as a commercial item, no cost data was requested or provided. 
In all, the Air Force bought 54 electronic engine control units for
about $19 million.  Each time, the Air Force paid the full commercial
catalog price, ranging from about $330,000 to $370,000 per unit. 

The contracting officer performed no additional price analysis
comparing either historical prices or quantities ordered against
commercial sales.  Such an analysis would have shown that the company
recently had sold 949 of these items, of which 82 percent were sold
to commercial customers at other than catalog prices.  In 1989, the
Air Force paid $80,000 each for 18 of these same units.  These units
were not purchased directly from the electronic engine control
manufacturer, but were procured under a contract with the aircraft
manufacturer.  Finally, the Air Force, in 1995, purchased from the
engine control manufacturer 90 electronic engine controls used on
another aircraft engine at a $63,200 unit price. 

With regard to commercial prices, our ongoing work has identified
other cases where commercially offered or catalog prices are
significantly higher than prices previously paid by DOD.  For
example, the Air Force recently received two such proposals, not yet
negotiated.  In the first case, in response to an Air Force
solicitation for 96 aircraft repair kits, the contractor submitted a
commercial unit price of about $3,144, for a total price of almost
$302,000.  The Air Force's analysis of historical prices showed that
it paid only about $900 a unit for 80 repair kits in 1996. 

In the second case, the contractor offered a $1,748 unit price for
792 aircraft engine fuel control kits, for a total of about $1.4
million.  According to the Air Force's procurement history records,
the unit price paid for this item in 1993 was only $464 for 150 of
these kits.  We were told that after the Air Force questioned the
contractor's proposed price as being too high, the contractor
verbally offered a 40-percent discount from its catalog, reducing the
proposed price to about $1,049 per unit. 

In a third case, the Navy received a unit price offer of about
$100,000 for communications test set equipment.  According to the
Navy's procurement history file, the equipment was purchased in early
1995 for around $25,000.  The Navy is currently attempting to
negotiate this contact.  However, to date, the contractor has not
provided an explanation for the price increase or any commercial
sales data. 

The fact that commercially proposed or catalog prices are high
relative to past paid prices presents contracting officers with a
negotiating challenge.  It is clear that initial contractor proposals
or catalog prices are the beginning point of the negotiation process
and should not be accepted as being fair and reasonable.  Sound price
analysis and aggressive negotiations are fundamental to DOD's ability
to achieve fair and reasonable commercial prices.  For example, in a
recently negotiated Air Force contract, the contractor had initially
offered a 30-percent discount off its commercial catalog prices for
about 8,000 engine spare parts.  However, the Air Force determined
that a significantly greater discount was warranted based on its
analysis of historical prices.  The Air Force's price analysis was
based on escalated historical prices paid for 120 items purchased on
a sole-source basis from this contractor and on projected future
demands for these items.  The historical prices used in its analysis
for many of the items were based on certified cost or pricing data
previously obtained from the contractor.  After the Air Force raised
concerns about the offered discount, the contractor proposed a
61-percent discount off the catalog prices, which the Air Force
accepted.  Based on the Air Force's projected future buy of the 120
sole-source items, the 61-percent discount represents about a
$25-million reduction from the contractor's initial offer. 

We support DOD's effort to reform its acquisition process and adopt
more efficient commercial practices.  Currently, DOD is increasing
its effort to purchase commercial products and adopt commercial
pricing methods.  The current contracting environment for commercial
items, particularly for sole-source items where there may be
insufficient market forces to contain prices, presents negotiating
challenges for DOD contracting officers. 

While today's DOD contracting environment is difficult, tomorrow's
may be even more challenging.  At the present time, contracting
officers rely heavily on cost-based historical prices as a baseline
for negotiating fair and reasonable commercial prices.  In the
future, this data will be less available.  Whether the risks will be
moderated as DOD contracting officers receive additional training in
commercial pricing and become more comfortable with price analysis as
the primary tool to determine price reasonableness and negotiate
commercial prices remains to be seen. 


   EFFECTIVENESS OF DOD'S
   MENTOR-PROTEGE PROGRAM
---------------------------------------------------------- Chapter 0:5

Section 831 of the National Defense Authorization Act for Fiscal Year
1991 established the Pilot Mentor-Protege Program.  The purpose of
the program is to provide incentives for major DOD contractors
(mentors) to furnish disadvantaged small business concerns\6

(proteges) with assistance designed to enhance their capabilities and
increase their participation as suppliers under DOD contracts, other
federal government contracts, and commercial contracts.  Over the
last few years, we have been asked to review various aspects of the
mentor-protege program.  Most recently, the National Defense
Authorization Act for Fiscal Year 1998 requires us to report, on or
before March 31, 1998, on DOD's implementation of the program and the
extent to which the program is achieving the purposes established by
the Congress. 

Our reports have shown a number of implementation problems.  For
example, in our 1992 report, we recommended, among other things, that
DOD develop and implement adequate internal controls in the
application and approval process and in the oversight of protege
development.\7 In our 1994 report, we were not able to recommend that
the pilot program be extended because sufficient information was not
available to determine whether the program's purposes could be
achieved or whether reauthorization and extension was warranted.\8

In our latest review, we found that continuing data limitations
preclude assessing whether the program is achieving the purposes
established by the Congress.  DOD has undertaken actions to review
the program that are intended to provide the basis for such an
assessment.  Such actions include conducting a survey of mentors and
proteges and requesting the Defense Contract Management Command to
conduct performance evaluations of each agreement.  However, we
believe shortcomings in the survey methodology and incomplete
performance evaluations will limit DOD's ability to assess the
program's overall effectiveness. 

The Congress has appropriated about $233 million for the program
since fiscal year 1992.  The funding was generally obligated through
either cooperative agreements where both the government and
contractor work together to obtain a common purpose, separate
contracts or line items in DOD prime contracts.  DOD has decided that
the services and the defense agencies should be responsible for
managing reimbursable mentor-protege agreements.  In addition, the
services have been inconsistent in paying fees to mentors for
providing assistance to proteges and reimbursing proteges for various
expenses. 

After spending over $200 million on its pilot mentor-protege program,
DOD lacks information needed to determine the program's
effectiveness.  Accordingly, in our imminent report, we are
recommending that DOD strengthen its performance reviews to ensure
that sufficient and reliable information is gathered on planned and
actual mentor assistance as well as on the protege firms' business
development.  In addition, we are suggesting that the Congress may
wish to clarify mentor-protege program legislation as to whether
mentors can be paid fees in addition to expenses and proteges can be
reimbursed for various expenses.  Neither the mentor-protege program
legislation nor DOD's implementing regulation specifically addresses
these expenses. 


--------------------
\6 To qualify as a disadvantaged small business, a company must not
exceed the Small Business Administration's standards for number of
employees or annual sales and must be independently owned (at least
51 percent) and operated by socially and economically disadvantaged
individuals. 

\7 Defense Contracting:  Interim Report on Mentor-Protege Program for
Small Disadvantaged Firms (GAO/NSIAD-92-135, Mar.  30, 1992). 

\8 Defense Contracting:  Implementation of the Pilot Mentor-Protege
Program (GAO/NSIAD-94-101, Feb.  1, 1994). 


   MULTIPLE-AWARD CONTRACTS
---------------------------------------------------------- Chapter 0:6

Concerned that federal agencies were avoiding competitive
requirements when ordering under task- and delivery-order contracts,
the Congress directed agencies to consider awarding multiple
contracts--rather than a single contract--when a task- or
delivery-order contract format was planned.\9 To provide for
competition in ordering, agencies are to provide each of the multiple
contractors a fair opportunity to be considered for orders placed
under the contract. 

Based on our ongoing work, agencies have not been consistent in
achieving competition for orders.\10 One agency issued a high
proportion--64 percent--of orders on a sole-source basis.  This
multiple-award contract has a potential value of over a billion
dollars.  In another multiple-award contract having the potential to
exceed several billion dollars, agency announcements of planned
orders identify "recommended" firms specifically invited to submit
proposals.  This practice has resulted in just one proposal being
received on most orders.  We also noted that several agencies will
increase the value of orders after contractors had been provided an
opportunity to be considered.  While it may be appropriate to award
follow-on or additional work to an incumbent contractor in many
circumstances--such as when the work involves continuing development
of a system the incumbent initiated--we are concerned that it may not
be appropriate in every instance, and that competition could be
undermined. 

Most contracts included in our review permit other federal agencies
to place orders under the contracts.  The primary agency responsible
for overall contract administration may charge a fee, when other
federal agencies place orders under the contract.  Fees should be
limited to recovering the actual costs under the contract.  While
agencies generally intend their fees to recover actual costs,
weaknesses in accounting and management systems at some agencies
obscure comparisons of fees to the costs incurred.  For example, one
agency charged fees that ranged from $125 to $99,000 for
administering a single order under the contract.  This agency did not
maintain accounting records or develop analyses that would justify
this disparity. 

Finally, use of the multiple-award contracting mechanism need not
impair the ability of small business to compete for federal
contracts.  Concerns had been expressed that-- because multiple-award
contracts sometimes consolidated the requirements of numerous
programs and offices, and provided for a large and diverse scope of
work--small business could not realistically compete for such
contracts.  The Office of Federal Procurement Policy, the Small
Business Administration, and the agencies where we did our work had
all taken steps to ensure that use of multiple-award contracts did
not exclude small business from the federal marketplace.  An adverse
effect on small business was not apparent in the statistical data
available to date.  In particular, small business received half or
more of the multiple-award contracts awarded at two agencies, and won
orders proportionate with the number of contracts received.  The
experience of these two agencies suggests that multiple-award
contracts can be structured to help participation by small business. 


--------------------
\9 A task- and delivery-order contract provides for an indefinite
quantity, within stated limits, of supplies or services to be
furnished during a fixed period, with deliveries or performance to be
scheduled by placing orders with the contractor. 

\10 Our work to date has focused on multiple-award contracts
administered by the Defense Information Systems Agency, the
Department of Transportation, the General Services Administration,
the National Institutes of Health, and the U.S.  Air Force's
Electronic Systems Center and Standard Systems Group. 


-------------------------------------------------------- Chapter 0:6.1

Mr.  Chairman, this concludes my prepared statement.  I would be
happy to respond to any questions you or other Members of the
Subcommittee may have. 


*** End of document. ***




FAS | Military Analysis | GAO |||| Index | Search |


Maintained by Webmaster