FAS | Military Analysis | GAO |||| Index | Search |


Major Acquisitions: Significant Changes Underway in DOD's Earned Value Management Process (Letter Report, 05/05/97, GAO/NSIAD-97-108).

GAO provided information on the Department of Defense's (DOD) Earned
Value Management process, focusing on the: (1) problems facing the
cost/schedule control system (CS2) process; (2) progress DOD has made
with reforms; and (3) challenges DOD faces in fostering and managing
potentially significant changes.

GAO noted that: (1) the core concept of the CS2 process, earned value,
is recognized as a sound way to measure progress on a major acquisition
programs; (2) over the years, however, the process has evolved to where
the needs of some of its key users are being satisfied, while others are
not; (3) because the data contained in the CS2 reports are typically up
to 2 months old, the reports do not function as an early warning system
needed by program managers; (4) moreover, the process has not fully
integrated cost, schedule, and technical data as intended; (5) DOD has
acknowledged the problems with CS2 for a decade, but reforms have
proceeded slowly mainly because responsibility for the process has
resided with the oversight organizations that have been its architects;
(6) DOD attempted to effect change in 1989 by transferring top-level
responsibility for the system from the comptroller staff to the
acquisition staff; (7) despite this transfer, little progress was made
because execution of CS2 at the field level remained within the
comptroller community; (8) nonetheless, DOD has embarked on several
reforms that could dramatically change the CS2 process; (9) recently,
DOD accepted industry's earned value management criteria as a
replacement for the government's long-standing CS2 criteria; (10) DOD
has also transferred responsibility and control over the process from
the services to the Defense Contract Management Command (DCMC), which
currently provides the on-site interface between the government program
office and the contractor; (11) another reform underway involves giving
DOD program managers latitude to tailor their contract data to the
specific needs of their program, such as the categories and the level of
detail; (12) these recent steps to reform the CS2 process have potent
implications; (13) for example, adopting the industry criteria could
result in less burdensome and more useful contractor management
information systems, but could also lessen the government's ability to
oversee defense programs; (14) in light of both the day-to-day demands
of managing the process on individual contracts and implementing recent
reforms, DCMC faces a significant challenge as it takes over stewardship
of the process; (15) service officials are concerned about how quickly
it can meet these demands, given its decline in staffing over the last
several years; and (16) ultimately, DCMC will have to ensure that the p*

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  NSIAD-97-108
     TITLE:  Major Acquisitions: Significant Changes Underway in DOD's 
             Earned Value Management Process
      DATE:  05/05/97
   SUBJECT:  Defense procurement
             Financial management systems
             Contract administration
             Defense contracts
             Contract monitoring
             Cost overruns
             Management information systems
             Reporting requirements
             Cost analysis
IDENTIFIER:  DOD Integrated Program Management Initiative
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                    <[email protected]>                        **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Subcommittee on Acquisition and Technology, Committee
on Armed Services, U.S.  Senate

May 1997

MAJOR ACQUISITIONS - SIGNIFICANT
CHANGES UNDERWAY IN DOD'S EARNED
VALUE MANAGEMENT PROCESS

GAO/NSIAD-97-108

Earned Value Management

(707153)


Abbreviations
=============================================================== ABBREV

  CS\2 - cost/schedule control system
  C/SCSC - cost/schedule control system
  DCMC - Defense Contract Management Command
  DOD - Department of Defense
  EDI - electronic data interchange
  EVMS - Earned Value Management Systems
  IBR - integrated baseline review

Letter
=============================================================== LETTER


B-276406

May 5, 1997

The Honorable Rick Santorum
Chairman
The Honorable Joseph I.  Lieberman
Ranking Minority Member
Subcommittee on Acquisition and Technology
Committee on Armed Services
United States Senate

Despite the regularity with which defense acquisition programs have
experienced cost overruns and schedule delays, the Department of
Defense (DOD) does have an extensive system intended to provide
program managers and others with early warnings of cost and schedule
problems.  In 1967, DOD issued a set of cost/schedule control system
(CS\2 ) criteria that it required defense contractors to meet. 
However, it has become widely accepted by DOD and the defense
industry alike that this process is in need of reform.  This report
addresses the problems facing the CS\2 process, the progress DOD has
made with reforms, and the challenges DOD faces in fostering and
managing potentially significant changes. 


   BACKGROUND
------------------------------------------------------------ Letter :1

DOD's CS\2 was established in 1967 as a tool to measure the value of
work performed as compared to the actual costs, a concept referred to
as earned value.  Earned value goes beyond the two-dimensional
approach of comparing budgeted costs to actuals.  It attempts to
compare the value of work accomplished during a given period with the
work scheduled for that period.  By using the value of work done as a
basis for estimating the cost and time to complete, the earned value
concept should alert program managers to potential problems sooner
than expenditures alone can. 

To illustrate, assume a contract calls for 4 miles of railroad track
to be laid in 4 weeks at a cost of $4 million.  After 3 weeks of
work, only $2 million has been spent.  By analyzing planned versus
actual expenditures, it appears the project is underrunning the
estimated costs.  However, an earned value analysis reveals that the
project is in trouble because even though only $2 million has been
spent, only 1 mile of track has been laid; thus, the contract is only
25 percent complete.  Based on the value of work done, the project
will cost $8 million ($2 million to complete each mile of track) and
the 4 miles of track will take a total of 12 weeks (3 weeks for each
mile of track) to complete instead of the originally estimated
4 weeks. 

The communities that have a vested interest in earned value generally
and CS\2 specifically are the (1) program managers, who are charged
with overall management responsibility for acquisition programs; (2)
contractors, who are responsible for successful execution of the
contract; and (3) overseers, such as acquisition executives,
financial managers, contract surveillance officials, and cost
estimators who are tasked with tracking and estimating program costs. 
For earned value to be effective, it must serve the basic needs of
all these users.  An engineer might consider its most useful output
to be technical status information on particular components.  An
accountant may view its most important product to be the cost versus
budget information it provides.  A program manager may share both
views or may value the scheduling data the most.  Thus, regardless of
the system or process used to record information, earned value should
provide insightful information to all three communities. 

In its 1967 financial management regulations, DOD issued 35 CS\2
criteria that were to be applied to most major weapon acquisitions
(see app.  I for a listing of the criteria).  The criteria are not an
accounting system per se, but rather general management or internal
control guidelines to be used on flexibly priced\1 contracts.  The
criteria require that the contractor's management control system
provide data that (1) relate time-phased budgets to specific contract
tasks; (2) indicate work progress; (3) properly relate cost,
schedule, and technical accomplishment; (4) are valid, timely, and
auditable; (5) supply managers with summary level information; and
(6) are derived from the same internal management control systems
used by the contractor. 

Over the years, the basic criteria were supplemented by additional
guidance and procedures for contractor system reviews and reporting,
which have become "de facto" requirements.  For example, the
government conducted a series of implementation and surveillance
reviews of contractors' management control systems to ensure they
complied with the criteria.  In 1972, DOD developed a Joint
Implementation Guide to standardize CS\2 implementation procedures. 
The guide contained a checklist of about 160 specific questions,
which were referred to as subcriteria, to be covered with contractor
employees during the implementation reviews.  These requirements
converge with other requirements in determining how a contractor
designs its management control system.  For example, DOD independent
cost estimators have required a different breakout of contractor cost
information than the CS\2 did and have specified that cost data be
collected in uniform or consistent categories.  The contractor's
system must be elaborate enough to satisfy all of these data
requirements, regardless of their source or purpose. 


--------------------
\1 Flexibly priced contracts include all types of cost reimbursable
contracts and those fixed-price contracts with incentive fee
arrangements.  Contracts not having such arrangements are called firm
fixed price.  The CS\2 requirement must be imposed on flexibly priced
contracts with a value of $70 million in research, development, test,
and evaluation and $300 million in procurement. 


   RESULTS IN BRIEF
------------------------------------------------------------ Letter :2

The core concept of the CS\2 process--earned value--is recognized as
a sound way to measure progress on major acquisition programs.  Over
the years, however, the process has evolved to where the needs of
some of its key users are being satisfied, while others are not. 
Specifically, DOD program managers are not satisfied with the
timeliness of the CS\2 reports.  Because the data contained in the
reports are typically up to 2 months old, the reports do not function
as an early warning system needed by program managers.  Moreover, the
process has not fully integrated cost, schedule, and technical data
as intended.  The want of such information can invite subjective and
potentially optimistic judgments to fill the void.  Contractors
maintain that accommodating extensive government certification
reviews, collecting and arraying data in prescribed categories, and
preparing detailed reports requires significant effort and cost to
the government and draws some of their engineering resources away
from program execution.  Commercial firms that use earned value
systems produce reports more frequently, more quickly, and in less
detail than the CS\2 process.  Users outside the program
offices--such as financial managers and cost estimators--find that
the data generally meets their needs.  These users generally place
more value on consistency among cost categories and less value on
timeliness than program managers. 

DOD has acknowledged the problems with CS\2 for a decade, but reforms
have proceeded slowly mainly because responsibility for the process
has resided with the oversight organizations that have been its
architects.  DOD attempted to effect change in 1989 by transferring
top-level responsibility for the system from the comptroller staff to
the acquisition staff.  Despite this transfer, little progress was
made because execution of CS\2 at the field level remained within the
comptroller community.  Nonetheless, DOD has embarked on several
reforms that could dramatically change the CS\2 process.  Recently,
DOD accepted industry's earned value management criteria as a
replacement for the government's long-standing CS\2 criteria.  DOD
has also transferred responsibility and control over the process from
the services to the Defense Contract Management Command (DCMC), which
currently provides the on-site interface between the government
program office and the contractor.  Another reform underway involves
giving DOD program managers latitude to tailor their contract data to
the specific needs of their program--such as the categories and the
level of detail.  In many cases, program managers are using direct
(on-line) access to obtain data from the contractors' internal
management information systems. 

These recent steps to reform the CS\2 process have potent
implications.  For example, adopting the industry criteria could
result in less burdensome and more useful contractor management
information systems, but could also lessen the government's ability
to oversee defense programs.  In light of both the day-to-day demands
of managing the process on individual contracts and implementing
recent reforms, DCMC faces a significant challenge as it takes over
stewardship of the process.  Service officials are concerned about
how quickly it can meet these demands, given its decline in staffing
over the last several years.  Ultimately, DCMC will have to ensure
that the process meets the basic needs of all its key users--program
managers, contractors, and oversight personnel. 


   THE CS\2 SYSTEM HAS SERVED
   OVERSIGHT NEEDS BETTER THAN
   PROGRAM MANAGEMENT NEEDS
------------------------------------------------------------ Letter :3

An earned value system faces stiff and somewhat competing demands
from its users:  (1) providing the right analyses in time for program
managers to use; (2) enabling adequate oversight and analysis of
multiple programs beyond the program office level; and (3) minimizing
the effort required of the contractor to provide the necessary
systems, data, and analysis.  DOD policy states that earned value is
an integrated program management tool, but because it has
historically been a comptroller function, CS\2 has been viewed by
other users as a compulsory and burdensome financial reporting
system.  Moreover, it has not fully satisfied the needs of many
program managers for up-to-date and integrated information on cost,
schedule, and technical progress. 


      PROGRAM MANAGERS: 
      TIMELINESS OF CS\2 DATA
      LIMITS ITS UTILITY
---------------------------------------------------------- Letter :3.1

Government program managers are responsible for managing the cost,
schedule, and technical performance of their weapon acquisition
programs.  The CS\2 process was designed to integrate these three
parameters into an effective early warning tool.  However, program
managers have historically expressed concerns that CS\2 data, as
reported, are too late and too voluminous to be useful for day-to-day
program management.  The primary CS\2 report, the cost performance
report, can be up to 60 days old and can contain over 100 pages of
detailed information.  We confirmed that many program managers still
see data timeliness as an issue.  According to some managers, using
CS\2 data is like "managing by looking through a rear view mirror."
DOD acknowledged some of these issues in the 1987 preface to the
Joint Implementation Guide. 

We contacted managers of Acquisition Category 1D\2 programs in
engineering and manufacturing development to evaluate whether the
CS\2 data met their needs.  Their comments disclosed that while they
strongly support the concept of earned value, the timeliness of the
data is still a significant concern.  Ten of the 15 program managers
who responded believed that the data were delivered too late to be an
effective real-time management tool.  Twelve of these managers
responded that the cost performance reports did not contain problems
they were not previously aware of.  One manager stated that because
the reports are between
30 and 90 days old, he usually winds up addressing long resolved
issues, which creates unnecessary work for him.  Another expressed
the concern that by the time the report reaches him, the problem is
usually worse.  Part of the problem is that the CS\2 system is driven
by users in an oversight, not in a program execution role, another
stated. 

When the CS\2 reporting requirements were initially generated in
1967, the data were as timely as commonly available technological
tools would allow.  The work was broken out into short, discrete work
packages and grouped into cost accounts.  Engineers in the
contractor's plant were often made responsible for managing and
reporting on these cost accounts each month.  This information was
accumulated, consolidated at the contractor's plant, and mailed to a
variety of offices within DOD.  Financial analysts within DOD then
reviewed and entered this data into software applications for further
analysis.  By this time, the data were at least
2 months old. 

Technological advances have eliminated the need for government
analysts to manually enter the data; nevertheless, many program
offices still rely on the mail to obtain the report in hard copy or
on a diskette.  Program managers are not satisfied with the
timeliness of these reports.  With the advent of readily accessible
real-time communication tools like personal computers, electronic
mail, and the Internet, however, program managers are unwilling to
wait 2 months for program status information.  Because of this time
delay, program managers generally use other means to satisfy their
needs for information, like integrated product teams.  With
technological advances in communication, program managers can
directly access contractor data on a real-time basis.  This type of
access allows the government program manager to analyze and react to
the same vintage information the contractor is using to manage. 

In addition to timeliness problems, the CS\2 reports emphasize the
cost and, to some extent, the schedule data without fully integrating
technical information.  Assessment of technical risks is left to
subjective, often optimistic judgments of program personnel.  One of
the managers we surveyed pointed out that CS\2 does not show if the
technical performance is being achieved or if the work on the
critical path is being done on time or within cost.  In other words,
even though a discrete work segment may be completed, CS\2 data do
not directly inform a manager of the success or quality of the work. 
Instead, the standard cost performance report attempts to assign a
cost value to schedule data.  Thus, if a particular design task has
fallen behind schedule, the standard report would assign a cost to
the delay and present that cost.  While this format may be ideal for
a cost analyst, it cannot by itself facilitate the timely management
of a technical problem, nor can it highlight the potential impact on
critical path schedules. 

Another limitation is that the detailed data collection categories
may not correspond to individual program structures.  Although the
need for overall reporting consistency is critical to the oversight
community, program managers are willing to accept progress
information presented in the same categories and format that the
contractor uses to manage the program.  Most program managers we
surveyed were interested in obtaining real-time data directly from
the contractor in whatever format the contractor used, as long as the
format remained consistent and the data could be verified.  According
to DOD officials, program managers have had the flexibility to modify
reporting requirements for many years, but the needs of the oversight
community for consistent reporting formats and categories have taken
precedence.  Many of the program managers we contacted are now taking
advantage of this flexibility to tailor their reporting requirements. 
DOD officials stated that these limitations reflect more on how the
CS\2 criteria and cost estimating requirements have been implemented
than on the criteria themselves.  They noted that the CS\2 criteria
allow such flexibility but that subsequent reporting and
certification procedures have led to rigid implementation practices. 


--------------------
\2 An acquisition category 1 program is defined as a major defense
acquisition program with estimated expenditures of over $355 million
in research, development, test, and evaluation, or over $2.135
billion in procurement (in fiscal year 1996 dollars).  A category 1D
program is monitored by the defense acquisition executive, not a
service executive.  At the time of our review, there were 16 category
1D programs in engineering and manufacturing development. 


      CONTRACTORS:  CS\2
      VALIDATION AND REPORTING ARE
      BURDENSOME
---------------------------------------------------------- Letter :3.2

Although contractors generally recognize the basic CS\2 criteria as
sound management principles, they believe DOD's implementation
process contains many burdensome requirements.  Independent studies
have found that CS\2 reporting was too detailed, repetitive, and
voluminous to be used effectively as a management tool either by the
government or by industry.  Further, they found that the requirement
may actually undermine program performance by diverting the time and
attention of the company program manager. 

The formal reports represent only the end product of what is required
in CS\2 .  Perhaps more significant are the cost and management
control systems the contractors must have in place to record the
required information and the government reviews of those systems to
certify their acceptability.  To facilitate cost estimating, DOD
guidance specifies a structure for breaking the work down on
developing a major weapon into uniform categories or segments.  At
the most basic level, CS\2 guidance requires that the work is broken
down into discrete short-span work packages, which are consolidated
into a cost account.  A cost account is a management control point at
which actual costs can be accumulated for an element of work.  Table
1 illustrates an excerpt of a work breakdown structure for an ongoing
aircraft program. 



                                Table 1
                
                Work Breakdown Structure for an Aircraft
                                Program

----------------------------------------------------------------------
Level 1: Aircraft

Level 2: Air vehicle

Level 3: Avionics

Level 4: Communication/navigation/identification

Level 5: Navigation software

Level 6: Individual cost accounts
----------------------------------------------------------------------
Source:  DOD. 

Cost accounts on complex weapon programs can number over 1,000. 
While these costs have to be recorded and tracked by the contractor
at some level, cost-estimating requirements in conjunction with CS\2
implementation practices have dictated how the accounts are defined
and at what level of detail.  For example, if a contractor's internal
management system differs from the cost-estimating structure, the
contractor may have to track costs one way to satisfy cost-estimating
requirements and another way to match how it actually manages.  This
requires more engineering effort by the contractor because it is the
technical staff who typically manage the cost accounts. 

For example, in aircraft programs, DOD requires contractors to record
and report earned value information for the aircraft sections. 
However, they could be managing by aircraft ribs (structural
components that strengthen larger airframe sections).  To satisfy
DOD's requirements, these contractors would have to artificially
segregate costs for ribs going to the fuselage from those going into
the wings and tail sections.  They would also have to prepare lengthy
CS\2 variance reports, which takes them away from their engineering
duties and provides information that does not directly relate to how
the program is being managed. 

The standardization that work breakdown categories provide may
facilitate government analysis and oversight above the program level,
but may not aid--and in fact could inhibit--program management.  DOD
policy has given program managers the flexibility to modify the level
of detail in CS\2 reports and more recently, the prescribed work
breakdown categories.  Program managers have used this latitude in
modifying their CS\2 reporting requirements and are tailoring their
reporting requirements.  However, the contractors still track costs
according to the standardized work breakdown structure to satisfy the
cost estimators' needs for data consistency.  Recently, the Cost
Analysis Improvement Group, an organization that provides independent
program cost estimates to the Secretary of Defense, has taken steps
to clarify the needs of the cost-estimating community with respect to
those of CS\2 .  The group, which relies on actual cost data from
contractor accounting systems, has reemphasized the need for
consistent cost reporting while reducing the burden on contractors. 
Accordingly, the group is in the process of improving its reporting
systems to allow contractors to report in their own formats and
appropriate level of detail, using a common data format that in turn
enables DOD to convert the data into the format needed by the
cost-estimating group. 

The requirement to have management control systems that comply with
CS\2 criteria is contained in the basic weapons system contract. 
However, the detailed requirements for these systems have been
established through an extensive certification process.  During the
certification reviews, the government used a checklist of 158
specific questions to assess compliance at all levels of the
contractor organization.  The various reviews conducted on the
contractor system during the life of a contract are summarized in
table 2 below, along with estimates of the resources involved. 



                                Table 2
                
                  Traditional CS\2 Compliance Reviews

                                                                  Staf
Review title        Purpose of review                       Days  f
------------------  --------------------------------------  ----  ----
Implementation      Contractor plans to implement CS\2      2-3   4-5
visit

Readiness           CS\2 implementation progress            5     5-
assessment                                                        15

Demonstration       Compliance of contractor management     15-   10-
review              control systems                         20    25

Extended            Revisions to management control system  10-   10-
subsequent                                                  15    25
application review

Subsequent          New contract application of CS\2        5     6-
application review  requirements                                  10

Baseline review     Proper implementation of contract       3     4-6
                    baseline
----------------------------------------------------------------------
Source:  Cost/Schedule Control System Criteria, The Management Guide
to C/SCSC;
Quentin W.  Fleming; 1992. 

The time and staff required for these reviews can vary, depending
upon the complexity of each contract.  In addition to the government
personnel required to perform these reviews, an even larger number of
contractor personnel may be required to support the reviews, which
adds to the cost of a contract.  For example, cost account managers
are interviewed during the demonstration review, using the checklist
of 158 questions.  Because the CS\2 implementation guidance dictates
that work be segregated into small, short work packages, this could
entail numerous contractor staff.  In addition, once a contractor's
system has been validated by the government, it is then subject to
periodic surveillance reviews that are performed throughout the life
of the contract.  Contractors ultimately viewed the
government-approved management system as one that the government had
imposed upon them.  They could not redesign or modify such a system
without first getting government approval.  If a change were
approved, the government would initiate another system review to
revalidate the revised system.  Contractors would thus shun system
improvements to avoid the additional reviews. 


      FINANCIAL MANAGEMENT:  CS\2
      IS SATISFACTORY
---------------------------------------------------------- Letter :3.3

Because the financial management community has presided over the
evolution of CS\2 for most of the past 30 years, the system has been
optimized to meet its needs.  The initial regulation governing CS\2
policy, DOD Instruction 7000.2, was issued by the DOD Comptroller as
part of the financial management regulations.\3 It was implemented
and administered by an infrastructure of DOD financial managers. 
This infrastructure permeates all levels of DOD.  At the program
office level, analysts review the reports, participate in contractor
system reviews, and monitor contract progress.  At the buying command
and at service headquarters, financial managers were in charge of the
various contractor system reviews, analyzing the reports, and
projecting trends and estimates to complete.  At the Office of the
Secretary of Defense, cost analysts and program oversight personnel
still review this data in support of major milestone decisions and
through periodic oversight reports like the Defense Acquisition
Executive Summary.  At the contractor's plant, DCMC representatives
participate in the system reviews, provide on-site surveillance and
review the reports for accuracy.  Until December 1995, the
Performance Measurement Joint Executive Group, comprised of financial
management experts, was the chief decision-making body for CS\2
issues. 

The role played by the financial management community is somewhat
unique to the federal government.  Unlike the commercial world, where
the company developing a new product funds the development with its
and/or investors' money, weapon system development is funded by the
government--the customer.  The government acts as an agent for the
public trust and therefore has a responsibility to oversee the
expenditure of those funds.  The cost performance report, as one of
the main reports to assist the government in that responsibility,
provides auditable data from which an analyst can generate
independent estimates at completion and may also project trends based
on contract performance.  Historical cost data from the contractor's
systems also enable estimators to develop parametric models from
which future weapon systems' cost can be estimated.  Consistent with
these needs, the financial management community, in its stewardship
over the CS\2 process, has placed a premium on reliable and
consistent cost data. 

The cost analyses performed on weapon acquisition programs are an
important internal control that can highlight performance problems
that program managers, as advocates, may overlook.  The controversy
and subsequent cancellation of the Navy's A-12 aircraft illustrates
the value of CS\2 data being available to organizations outside the
program manager's office.  According to the 1990 administrative
inquiry conducted for the Secretary of the Navy, the cost performance
data from the A-12 contractors clearly indicated significant cost and
schedule problems.  The results of an oversight review of the cost
performance reports disclosed that the A-12 contract would probably
exceed its ceiling by $1 billion.  However, neither the contractors
nor the Navy program manager relied upon this data; instead, they
used overly optimistic recovery plans and schedule assumptions.  The
inquiry concluded that the government and contractor program managers
lacked the objectivity to assess the situation and they disregarded
financial analysts who surfaced the problems. 


--------------------
\3 In 1991, DOD Instruction 7000.2 was canceled and its requirements
included in DOD Instruction 5000.2, Defense Acquisition Management
Policies and Procedures.  This regulation was superseded in 1996 by
DOD Regulation 5000.2-R, Mandatory Procedures for Major Defense
Acquisitions. 


   COMMERCIAL FIRMS USE
   STREAMLINED EARNED VALUE
   PROCESS
------------------------------------------------------------ Letter :4

Commercial firms are increasingly adopting the concept of earned
value for development projects but are applying it in a more
streamlined manner than DOD has historically.  Company managers are
getting real-time progress information in formats consistent with how
they manage.  Major defense contractors are also overwhelmingly
emphasizing up-front technical planning and scheduling as opposed to
traditional cost and schedule monitoring.  United Defense Limited
Partnership, a major defense contractor, has inserted earned value
concepts throughout its entire management structure.  A senior
official from that company stated that a successful program hinges on
good technical planning and scheduling, and for earned value to
succeed, it must be useful to everyone, not just a requirement
imposed onto one functional group by another.  It must also be used
to manage internally or it will not be taken seriously by those
operating it. 

Motorola is also convinced that technical planning and scheduling is
paramount to a successful project.  It is using earned value to
manage its multi-billion dollar satellite communication system,
IRIDIUM“, with streamlined data accumulation, reporting, and
oversight mechanisms.  For example, technical and schedule data are
monitored at very detailed levels whereas costs are not included
until much higher levels of reporting are reached.  Similarly,
Lockheed-Martin Missiles and Space has announced it is adopting
earned value for all its contracts, regardless of whether the
customer is military or commercial.  Lockheed-Martin has benchmarked
its processes and identified the best practices in program
management.  It estimates that by adopting these practices for each
contract, it could reduce its non-value added activities by almost
two-thirds. 

When a firm like Motorola makes large investments in a major
development like IRIDIUM“, detailed information requirements are
essential, and in many ways, similar to DOD's.  According to the
firms we spoke with who are funding major product developments and
are using an earned value system, they produce internal status
reports more frequently, with less detail, that are more current than
what the traditional CS\2 process has provided to DOD.  They also
place emphasis on technical and schedule planning.  Moreover, the
manner in which the data is organized and reported is typically
aligned with the way the companies manufacture.  Government program
managers we contacted indicated that this is precisely what they
want--real-time information presented in a manner that mirrors how
the contractor manufactures. 

The major differences between commercial earned value practices and
traditional defense practices are shown in table 3. 



                                Table 3
                
                Comparison of Commercial Earned Value to
                Traditional Defense CS\2 Implementation
                               Practices

                                                    Traditional DOD
Characteristic                  Commercial program  program
------------------------------  ------------------  ------------------
Frequency of status reports     Weekly/bi-weekly    Monthly

Age of information              Real-time/weekly    Up to 60 days
                                                    after reporting
                                                    period ends

Method of data dissemination    Direct access to    Mailing of reports
                                database            on paper or disk

Work breakdown structure        Level 3             Level 3-7

Variance analysis reporting     Critical path       All elements
                                items

Management focus                Technical and       Costs
                                schedule
----------------------------------------------------------------------
As indicated, commercial firms obtain data much more frequently than
DOD programs typically do.  Program status information was available
to commercial managers on either a real-time basis or as close as the
companies' computer capabilities would allow.  Another significant
difference is that variance analysis reporting is much less detailed
in commercial firms.  Companies set tolerance limits within
appropriate manufacturing processes and report on breaches of those
limits as opposed to reporting on all variances regardless of the
element.  If the element is not critical to meeting technical
progress, schedule, or cost, it may be monitored but not necessarily
reported. 

Management emphasis differs between the two groups, as well.  We
found that on commercial programs, the manufacturers tended to focus
on adherence to schedule as opposed to costs.  They do not forsake
the other benefits of earned value but focus on technical and
schedule goals before costs.  Their philosophy is that focusing on
key technical accomplishments per the planned schedule will cause
planned costs to fall in line.  For example, in the IRIDIUM“ program,
Motorola did not add costs to the program status reporting until
almost halfway up the reporting channel, a much higher level of
aggregation than a typical DOD program.  Company management reasoned
that because engineers and program managers on the manufacturing
floors are concerned with meeting technical and schedule
accomplishment, recording earned value information at that level is
more informative in terms of labor hours than in terms of dollars. 
DOD officials believe that the CS\2 criteria and basic guidance allow
defense programs the flexibility to manage earned value information
in a similar manner, but in practice, this flexibility has been
limited by over implementation of the guidance. 


   SOME EARLY REFORMS HAVE MADE
   LIMITED IMPACT
------------------------------------------------------------ Letter :5

Despite DOD's acknowledgement of CS\2 implementation problems, until
recently, little progress has been made in resolving them. 
Fundamentally, this is because the CS\2 data was responsive to the
financial management community that managed how the CS\2 process was
implemented.  Most of the reforms that have been undertaken have made
improvements, but have not alleviated the more significant problems
with CS\2 .  The integrated baseline review, however, may prove to be
an exception because it has resulted in a marked reduction in
traditional CS\2 oversight reviews thus far. 

The 1987 Joint Implementation Guide clarified the objectives of CS\2
as "For contractors to use effective internal cost and schedule
management control systems, and for the government to be able to rely
on timely and auditable data produced by those systems for
determining product-oriented contract status." It went on to stress
that improved communication between the government and contractors
could reduce improper implementation of the CS\2 criteria.  To
stimulate reform, DOD transferred the responsibility for CS\2 policy
from the DOD Comptroller to the Under Secretary of Defense for
Acquisition in August 1989.  This top-level organizational change was
not mirrored in the three services, however.  As a result, CS\2
implementation was fragmented, with top-level policy being managed by
the Office of the Secretary of Defense acquisition community and
day-to-day implementation being managed by the services' financial
management community.  The Performance Measurement Joint Executive
Group, comprised of financial management personnel, maintained
central oversight of the CS\2 process. 

In October 1993, the Under Secretary of Defense for Acquisition &
Technology undertook an initiative to return earned value to its
original purpose:  a tool to integrate cost, schedule, and technical
performance management.  This initiative attempted to reduce the
review burden and limit reporting requirements.  In addition, to
change the emphasis from government oversight to contractor
responsibility, it encouraged industry to develop its own management
standards.  About a year later, after limited progress, the Office of
the Secretary of Defense requested that the Service Acquisition
Executives personally revitalize CS\2 reform under the Integrated
Program Management Initiative.  A top-level Executive Steering Group
was formed to reengineer earned value implementation among the
services.  This group noted that while there were no exceptions to
the CS\2 requirement, there was broad latitude to change
implementation practices.  DOD and recent service policy memoranda
have consistently stressed the need to streamline traditional CS\2
reporting requirements by reducing the detailed information required,
such as minute categories for reporting and tailoring other report
formats.  In December 1993, DOD initiated a major effort to revise
the Joint Implementation Guide.  After multiple iterations and
coordination difficulties, it was finally reissued in December 1996. 

To provide more timely information to program managers, DOD is
requiring all new contracts to use electronic data interchange (EDI)
as a data transfer method.  Under the EDI concept, the contractors
will make the cost performance reports, along with other reports,
electronically accessible to program managers, financial analysts,
and other users like DCMC staff.  To ensure that the cost performance
report data can be transmitted and received in common data formats,
DOD has developed a standardized data set for contractors to use when
transmitting their reports. 

One of the benefits of using EDI for cost performance reporting is
the time saved by electronically transmitting data to customers
instead of physically mailing written reports.  The time savings has
been estimated at about 1 to 2 weeks, which includes time associated
with re-keying information into the customers databases so they can
perform their analyses.  Notwithstanding some of the problems that
are being encountered with electronic transfers of data, we believe
the time savings associated with this method of communication is not
likely to make the reports more useful.  At best, the reports will
still be received by the program offices and other interested users
about 30 to 45 days after the end of a reporting period. 

One of the more successful initiatives to date is the integrated
baseline review (IBR), which DOD implemented in 1994.  The IBR
focuses on the development of a detailed and achievable technical,
cost, and schedule baseline for a program.  The objectives of the IBR
are twofold:  to improve the use of cost performance data by
contractor and government program managers and to reduce the number
of CS\2 reviews.  As part of the overall risk assessment process, the
IBR is intended to integrate the technical content of the work with
cost and schedule parameters.  It is planned and executed by a
multi-functional government and contractor team composed of
engineering, logistics, manufacturing, contracting, and financial
personnel.  Unlike traditional CS\2 baseline reviews, the IBR is led
by the program manager, not a financial manager.  By involving the
program manager directly in this review, the process highlights the
merits of using earned value to track progress.  It has also reduced
the number of CS\2 compliance reviews.  For example, CS\2 reviews
have decreased from 56 in 1993 to 5 in 1995 while the number of IBRs
have increased from 3 in 1993 to 29 in 1995.  The program managers
that we surveyed strongly supported the IBR as a valuable program
management tool. 


   RECENT REFORMS HAVE MAJOR
   IMPLICATIONS FOR CS\2 PROCESS
------------------------------------------------------------ Letter :6

DOD has recently taken a three-pronged approach to reform the CS\2
process.  This approach includes an internal organizational
realignment to override cultural resistance to change, the
recognition of commercial industry criteria to return the management
of the system to the contractor, and direct electronic access by
program managers to contractor information to improve the timeliness
of CS\2 data.  These changes may have significant impact on the
extent, type, and number of CS\2 system reviews that occur; the
interpretation of how the criteria/guidelines will be implemented;
and the extent and timeliness of data received by government
managers. 


      ORGANIZATIONAL RESTRUCTURING
      SHIFTS CS\2 STEWARDSHIP AWAY
      FROM FINANCIAL MANAGEMENT
      COMMUNITY
---------------------------------------------------------- Letter :6.1

In December 1995, DOD disbanded the Performance Measurement Joint
Executive Group and made DCMC the executive agent for CS\2 issues. 
Policy matters are now handled by the Executive Steering Group, which
includes representatives from the Service Acquisition Executives and
the DCMC Commander, to provide a program management orientation.  In
October 1996, the Under Secretary of Defense for Acquisition and
Technology formally transferred CS\2 compliance responsibility from
the services to DCMC.  This latest decision completes the transfer of
virtually all CS\2 policy, reform, and compliance decisions away from
the financial management community.  Nonetheless, members of the
various communities will continue to be involved in the day-to-day
operations of CS\2 .  How their roles, practices, and interaction
with one another will change under the aegis of DCMC remains to be
seen. 

This transfer has significant repercussions for DCMC.  Service
representatives have expressed concern that because of its recent
downsizing, DCMC may not have the resources to adequately perform its
increased responsibilities.  In addition, about 85 percent of the
CS\2 field staff dedicate only part of their time to cost performance
monitoring.  DCMC officials are aware of these concerns and are
working on a strategy to accomplish its newly expanded mission. 
Since it is likely that DCMC will not receive more than 6 additional
billets to supplement its current level of approximately 150 cost
performance monitors, DCMC is reengineering its approach to CS\2
implementation along the lines of statistical process control
measurement techniques.  This would represent a
management-by-exception approach to CS\2 reviews.  This may be a less
costly and burdensome approach that could have potential for reducing
the resources and infrastructure associated with traditional CS\2
implementation practices, especially when coupled with the
streamlined industry earned value criteria discussed below.  How well
such an approach meets the basic needs of all the communities is yet
unknown. 

DCMC has begun to work with contractors to change the surveillance
focus from the traditional oversight role to one based on what DOD
refers to as insight--the ability to identify problems through
process-based indicators such as the number of retroactive changes to
cost accounts or the frequency of replanning actions.  Each major
defense contractor will work with DCMC to identify appropriate
process and control objectives.  Ultimately, once each contractor
designates its unique control limits, DCMC staff would only be
required to review those processes that are out of tolerance.  In
addition, DCMC plans to work with industry to encourage firms to
assume responsibility for their management systems and processes. 


      INDUSTRY STANDARD DEVELOPED
      BY CONTRACTORS
---------------------------------------------------------- Letter :6.2

In August 1996, five industry associations published an industry
standard, Industry Standard Guidelines for Earned Value Management
Systems (EVMS), as a replacement to the current government CS\2
criteria.  DOD formally recognized the guidelines in the industry
standard in December 1996.  The Director, Defense Procurement, has
issued an interim rule to enable contractors to begin using the EVMS
without having to wait for the formal regulatory change process.  The
long-range plan is for the standard to be approved by the American
National Standards Institute\4 and the International Standards
Organization\5 .  This would formally move earned value into common
usage worldwide.  In addition, Australia, Canada, and the United
States have signed a memorandum of understanding concerning common
cost and schedule management for their acquisitions. 

The EVMS contains 32 criteria, which are similar to the DOD criteria
in principle.  For example, both sets of criteria are divided into
five broad categories:  organization; planning, scheduling, and
budgeting; accounting; analysis and management reports; and revisions
and data maintenance.  Each category describes the internal controls
that should exist to facilitate proper program management.  They also
contain provisions for breaking out work into discrete work packages,
documenting changes to the performance baseline, and measuring cost,
schedule, and technical accomplishment.  The industry standard gives
the contractor the flexibility to revise the system as needed to
reflect work consistent with internal management structures and to
track costs at a higher organizational level than the level typically
tracked in DOD programs.  (See app.  I for a more detailed
comparison.)

Despite their general similarities, the two criteria have significant
differences that could affect government oversight.  One such
difference is that the government may no longer have the same review
and approval authority over contractor management systems that it had
in the past.  DOD does not accept the self-certification provisions
of the EVMS standard.  Instead, it would like to find some middle
ground between self-certification and traditional government
certification.  DOD's goal is to encourage contractors to conduct
self-evaluations with the government acting as an observer.  However,
DOD's recently revised implementation guidance does not preclude a
government review, when warranted, or a third-party certification
arrangement.  The third-party certification concept represents the
standard industry practice for quality assurance systems.  Although
not required by the International Standards Organization--the
industry organization responsible for the quality assurance
standards--it is a generally accepted practice to get a third-party
certification in order to meet its quality standards.  This approach
may mitigate some potential risks of the self-evaluation process and
prove to be less burdensome than DOD's traditional review and
certification process. 

Other differences associated with the EVMS standard include the
absence of specific requirements for access to data by the customer
(in this case, DOD) for oversight purposes and the fact that program
baseline and contractor management systems may be changed as needed,
with notification of, but not approval by, the customer.  Use of this
standard on future procurements should give the contractor greater
ability to manage and improve its processes.  However, it could
impede government oversight unless DOD is able to obtain accurate and
timely information on contract performance.  Therefore, acquisition
program managers will have to ensure that access to data and
reporting provisions are included on individual contracts.  Likewise,
the increased flexibility to revise the program baseline may make it
more difficult to track divergences from original program goals. 


--------------------
\4 The American National Standards Institute serves as the
administrator and coordinator of the U.S.  private sector voluntary
standardization system.  It promotes and facilitates voluntary
consensus standards. 

\5 The International Standards Organization is a nongovernmental
entity whose mission is to promote the development of standardization
and related activities in the world.  Its focus is to facilitate the
international exchange of goods and services and to develop
intellectual, scientific, technological, and economic cooperation. 


      TECHNOLOGY ADVANCES CAN
      IMPROVE DATA TIMELINESS AND
      QUALITY
---------------------------------------------------------- Letter :6.3

Current technology offers improvements that could enhance the
timeliness of the CS\2 data as well as integrate cost and schedule
performance with technical performance.  For several years, DOD has
recognized the need to improve data timeliness and has focused on
developing EDI capability.  We found that several program managers
have experimented with getting direct access via personal computers
to contractor databases.  This permits them to see at the same time
the same data the contractor uses to manage the program.  The
government program managers who have used these systems are very
pleased with the quality and timeliness of the information.  We spoke
to managers in several program offices who believe that having direct
access to the same data the contractor uses to manage is critical. 

For example, the Joint Stand-Off Weapon system program has a direct
access system that was custom built for the program.  It contains
cost, schedule, manufacturing, test, and engineering/technical
information using a variety of displays.  The system uses
commercially available software and the data is updated twice a
month.  Cost data is available only to the program manager and two
others to protect the contractor's proprietary rate information. 
Similarly, the Sea-Launched Attack Missile Extended Range program is
using a direct access system that enables its technical staff to pull
up weekly status data by component or subsystem.  Each integrated
product team also reports progress each week.  Information is
displayed in terms of labor hours, not cost.  Cost data is
consolidated and presented at a high level.  Improvements in the
direct-access processes are being made to allow subcontractor data to
be included and also to include such items as indirect charges to the
contract and overhead. 

These practices represent a departure from the traditional
implementation of CS\2 , but technically have always been permissible
under the basic 35 criteria.  Nonetheless, using the streamlined
direct access process has not been universally accepted.  Some DOD
officials have expressed concerns about the potential for inaccurate
data in these systems that a monthly cost performance report review
and audit would pick up.  In addition, the direct access may only
provide detailed data on the prime contractor status, and summary
level information on subcontractor status.  Another concern is that
the cost information may not include full overhead charges.  To
implement this approach, the government may need to install special
high-speed transmission or "trunk lines" that could be expensive and
prohibit small programs from using it.  Because of these concerns,
DOD has not endorsed this approach, but is focusing on EDI
implementation. 

Managers of these two programs believe that direct access systems are
very beneficial and have found work-arounds for some of these
concerns.  According to the author of the EDI cost performance report
data set, the ultimate goal is to standardize the data format in
reports.  The author believes that the optimal solution would be to
standardize to a common data format so that DOD could reach into a
contractor's database to access, download, and analyze information. 
This would allow a program office to monitor the progress of a
contractor's performance, using the currently accepted reporting
formats, on a near real-time basis.  This standardization would also
allow for the almost instantaneous analysis by the users of this
information no matter what software or hardware the contractor or the
government use.  This capability would eliminate some of the concerns
to unique direct access systems while still giving the program
manager the flexibility to design a system that meets the needs of
his particular program. 

Another promising effort is a technical risk assessment process being
developed and tested by the Navy's Program Executive Office for Air
Anti-Submarine Warfare, Assault, and Special Mission Programs.  This
effort is intended to work in tandem with the CS\2 information to
enable the manager to integrate technical progress with cost and
schedule data.  In an early test, it was able to highlight and
quantify technical risks on one program much earlier than the CS\2
process did.  The process requires that the critical technical
performance and schedule drivers be identified at the start of the
program.  By developing specific risk curves and progress plans for
each parameter, the program manager can obtain insights not just into
work progress, but also technical success.  This provides an early
warning of technical problems, permitting the manager to mitigate
cost and schedule impacts.  For example, in a retrospective analysis
of an aircraft cockpit program, the software predicted problems more
than a year before the CS\2 process did because the CS\2 process, as
applied, emphasized cost and schedule integration without adequate
regard for technical performance.  This software is currently being
tested by the Federal Aviation Administration and is scheduled to be
used on the H-1 Helicopter Upgrade program. 


   CONCLUSIONS AND RECOMMENDATIONS
------------------------------------------------------------ Letter :7

DOD has taken major steps to create an organizational environment
that can facilitate making needed changes to the CS\2 process.  DCMC
faces a number of challenges as it begins its stewardship of the
process.  These include implementing initiatives to improve its
utility to program managers and to streamline certification and
reporting requirements, determining how best to manage the
certification review process, and deciding on a number of proposed
improvements.  To be successful, DCMC will need to understand the
resource implications of these challenges. 

Another challenge DCMC faces, perhaps more subtle than the above
reforms, is to better balance the needs of the different communities
that depend on earned value information.  A common understanding of
these basic needs will be instrumental to DCMC's ability to protect
each user's basic needs from others' secondary desires. 
Specifically, as DCMC endeavors to make CS\2 more responsive to
program managers, it will have to guard against basic oversight needs
going unmet.  We recommend that the Secretary of Defense (1)
promulgate the basic needs of the organizations that depend on earned
value information in some manner, such as in the implementing
guidance for EVMS and (2) take steps to ensure that the "wants" of
one organization do not encroach upon the basic needs of other
organizations that depend on earned value information as the
management of the CS\2 process transitions to DCMC and as DCMC makes
decisions on reforms in the future. 

In addition, several initiatives relevant to the CS\2 process are
ongoing, including EDI, IBRs, direct access, technical risk
assessment, self-certification, and the application of statistical
process control techniques to the surveillance process.  Some of
these are being demonstrated in varying degrees on different
programs, while others are still in the policy stage.  It is
important that DOD ensures that as these initiatives are tested on
individual programs, data is captured in a disciplined enough manner
to support decisions on what to implement and how.  Such data will
not only help get the most out of each initiative, but will also make
it easier to recognize possible interactions among initiatives. 


   AGENCY COMMENTS
------------------------------------------------------------ Letter :8

DOD generally concurred with our conclusions and recommendations.  It
agreed with the need to make sure that the "wants" of one
organization do not outweigh the needs of another.  It cited the
establishment of two groups that will help ensure that the needs of
all organizations that use earned value information are met.  These
are (1) the Integrated Program Management Initiative Executive
Steering Group, which includes representatives from the Office of the
Secretary of Defense, the services, DCMC, and other organizations;
and (2) the Performance Management Advisory Council under DCMC, which
will have service representatives from the earned value community and
from the project management or acquisition communities.  According to
DOD, the Council will also participate in a contractor cost data
reporting initiative, along with the cost- estimating community.  DOD
agreed that the relationships among the organizations as well as
their information needs should be made clear in its EVMS guidance. 

In the draft of this report, we recommended that DOD rebaseline the
basic needs of the organizations that depend on earned value
information.  DOD believed that it did not need to formally
rebaseline these needs because it had developed a good understanding
of them in arriving at the significant reforms it is currently making
to CS\2 and in identifying the earned value skills needed by
business, cost-estimating, and financial management specialists.  We
agreed to delete the recommendation from the final report with the
proviso that DOD would reassess its progress on earned value reform
and on meeting the needs of the key organizations.  Accordingly, DOD
stated that the Integrated Program Management Initiative Executive
Steering Group would assess such progress.  DOD also stated that the
group will coordinate all earned value management improvement efforts
with other initiatives, such as electronic access and risk
management. 


   SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :9

To develop information for this report, we contacted, interviewed,
and obtained documents from officials of the Office of the Secretary
of Defense and DCMC because of the policy-making responsibilities and
reform initiatives occurring at these levels.  In addition, we
obtained information from officials at the Office of Management and
Budget; the Federal Aviation Administration; the Office of the
Secretary of Defense Cost Analysis Improvement Group; service
headquarters; Army Materiel Command; Navy Air Systems Command;
Program Executive Office for Air Anti-Submarine Warfare, Assault, and
Special Mission Programs; Air Force Materiel Command; Aeronautical
Systems Center; and Defense Plant Representative Office personnel
from General Electric, Lynn, Mass; Boeing, Seattle, Wash; Lockheed
Martin, Sunnyvale, Ca; and McDonnell Douglas, St.  Louis, Mo.  We
contacted all program managers responsible for Acquisition Category
1D programs in engineering and manufacturing development.  We
received responses from all but 1 of these 16 programs.  We also
contacted officials from the governments of Australia and Sweden to
obtain information on their requirements for cost schedule control
systems. 

To obtain industry's perspective on policy issues and implementation
practices, we contacted representatives from the following industry
or professional associations:  National Security Industrial
Association, Aerospace Industries Association, Performance Management
Association, and Project Management Institute.  We also discussed
these issues with representatives from the following commercial and
military contractors:  Boeing, General Electric, Lockheed Martin,
Loral, Magnavox, McDonnell Douglas, Motorola, Northrop/Grumman,
Raytheon, Supply Tech, Inc., Texas Instruments, Textron, and United
Defense Limited Partnership. 

To address historical information and obtain independent views on how
current policy initiatives could affect future implementation
practices, we contacted representatives from the Massachusetts
Institute of Technology and Wright State University and the following
consulting groups:  Coopers & Lybrand LLP, KPMG Peat Marwick LLP,
Humphreys & Associates, Fleming Management Consultancy, and Write
Concepts.  We also obtained historical information from the
Management Systems Deputy, Air Force Assistant Secretary for
Financial Management and Comptroller. 

We performed our review from June 1996 to March 1997 in accordance
with generally accepted government auditing standards. 


We are sending copies of this report to other interested
congressional committees; the Secretary of Defense; the Commander,
DCMC; and the Director of the Office of Management and Budget.  We
will also make copies available to others upon request. 

Please contact me at (202) 512-4383 if you or your staff have any
questions concerning this report.  The major contributors to this
report were Paul Francis, Rae Ann Sapp, and Jeff Hunter. 

Katherine V.  Schinasi
Associate Director
Defense Acquisitions Issues


COMPARISON OF DOD AND INDUSTRY
CRITERIA
=========================================================== Appendix I

DOD's Criteria                            Industry's Criteria
----------------------------------------  ----------------------------
Organization                              Organization

1. Define all authorized work and         Define the authorized work
related resources to meet the             elements for the program. A
requirements of the contract, using the   work breakdown structure,
framework of the contract work breakdown  tailored for effective
structure.                                internal management control,
                                          is commonly used in this
                                          process.

2. Identify the internal organizational   Identify the program
elements and the major subcontractors     organizational structure,
responsible for accomplishing the         including the major
authorized work.                          subcontractors responsible
                                          for accomplishing the
                                          authorized work, and define
                                          the organizational elements
                                          in which work will be
                                          planned and controlled.

3. Provide for the integration of the     Provide for the integration
contractor's planning, scheduling,        of the company's planning,
budgeting, work authorization, and cost   scheduling, budgeting, work
accumulation systems with each other,     authorization, and cost
the contract work breakdown structure,    accumulation processes with
and the organizational structure.         each other, and as
                                          appropriate, the program
                                          work breakdown structure and
                                          the program organizational
                                          structure.

4. Identify the managerial positions      Identify the company
responsible for controlling overhead      organization or function
(indirect costs).                         responsible for controlling
                                          overhead (indirect costs).

5. Provide for integration of the         Provide for integration of
contract work breakdown structure with    the program work breakdown
the contractor's functional               structure and the program
organizational structure in a manner      organizational structure in
that permits cost and schedule            a manner that permits cost
performance measurement for contract      and schedule performance
work breakdown structure and              measurement by elements of
organizational elements.                  either or both structures as
                                          needed.

Planning & budgeting                      Planning, scheduling, &
                                          budgeting

6. Schedule the authorized work in a      . . . . interdependencies
manner that describes the sequence of     required to meet the
work and identifies the significant task  requirements of the program.
interdependencies required to meet the
development, production, and delivery
requirements of the contract.

7. Identify physical products,            Identify physical products,
milestones, technical performance goals,  milestones, technical
or other indicators that will be used to  performance goals, or other
measure output.                           indicators that will be used
                                          to measure progress.

8. Establish and maintain a time-phased   Establish and maintain a
budget baseline at the cost account       time-phased budget baseline,
level against which contract performance  at the control account
can be measured. Initial budgets          level, against which program
established for this purpose will be      performance can be measured.
based on the negotiated target cost. Any  Budget for far-term efforts
other amount used for performance         may be held in higher level
measurement purposes must be formally     accounts until an
recognized by both the contractor and     appropriate time for
the government.                           allocation at the control
                                          account level. Initial
                                          budgets established for
                                          performance measurement will
                                          be based on either internal
                                          management goals or the
                                          external customer negotiated
                                          target cost including
                                          estimates for authorized but
                                          undefinitized work. On
                                          government contracts, if an
                                          over target baseline is used
                                          for performance measurement
                                          reporting purposes, prior
                                          notification must be
                                          provided to the customer.

9. Establish budgets for all authorized   Establish budgets for
work with separate identification of      authorized work, with
cost elements (labor, material, etc.).    identification of
                                          significant cost elements
                                          (labor, material, etc.) as
                                          needed for internal
                                          management and for control
                                          of subcontractors.

10. To the extent the authorized work     To the extent it is
can be identified in discrete, short-     practical to identify the
span work packages, establish budgets     authorized work in discrete
for this work in terms of dollars,        work packages, establish
hours, or other measurable units. Where   budgets for this work in
the entire cost account cannot be         terms of dollars, hours, or
subdivided into detailed work packages,   other measurable units.
identify the far-term effort in larger    Where the entire control
planning packages for budget and          account is not subdivided
scheduling purposes.                      into work packages, identify
                                          the far-term effort in
                                          larger planning packages for
                                          budget and scheduling
                                          purposes.

11. Provide that the sum of all work      Provide that the sum of all
package budgets plus planning packages    work package budgets plus
within a cost account equals the cost     planning package budgets
account budget.                           within a control account
                                          equals the control account
                                          budget.

12. Identify relationships of budgets or  No comparable provision.
standards in underlying work
authorization systems to budgets for
work packages.


13. Identify and control level of effort  . . . . Only that effort
activity by time-phased budgets           which is unmeasurable or for
established for this purpose. Only that   which measurement is
effort which cannot be identified as      impractical may be
discrete or as apportioned effort will    classified as level of
be classed as level of effort.            effort.

14. Establish overhead budgets for the    Establish overhead budgets
total costs of each significant           for each significant
organizational component whose expenses   organizational component of
will become indirect costs. Reflect in    the company for expenses
the contract budgets, at the appropriate  that will become indirect
level, the amounts in overhead pools      costs. Reflect in the
that will be allocated to the contract    program budgets, at the
as indirect costs.                        appropriate level, the
                                          amounts in overhead pools
                                          that are planned to be
                                          allocated to the program as
                                          indirect costs.

15. Identify management reserves and      Same.
undistributed budget.

16. Provide that the contract target      Provide that the program
cost plus the estimated cost of           target cost goal is
authorized but unpriced work is           reconciled with the sum of
reconciled with the sum of all internal   all internal program budgets
contract budgets and management           and management reserves.
reserves.

Accounting                                Accounting considerations

17. Record direct costs on an applied or  Record direct costs in a
other acceptable basis in a formal        manner consistent with the
system that is controlled by the general  budgets in a formal system
books of account.                         controlled by the general
                                          books of account.

18. Summarize direct costs from cost      When a work breakdown
accounts into the work breakdown          structure is used, summarize
structure without allocation of a single  direct costs from control
cost account to two or more work          accounts into the work
breakdown structure elements.             breakdown structure without
                                          allocation of a single
                                          control account to two or
                                          more work breakdown
                                          structure elements.

19. Summarize direct costs from the cost  Summarize direct costs from
accounts into the contractor's            the control accounts into
functional organizational elements        the contractor's
without allocation of a single cost       organizational elements
account to two or more organizational     without allocation of a
elements.                                 single control account to
                                          two or more organizational
                                          elements.

20. Record all indirect costs that will   Same.
be allocated to the contract.

21. Identify the basis for allocating     No comparable provision.
the cost of apportioned effort.

22. Identify unit costs, equivalent unit  . . . or lot costs when
costs, or lot costs, as applicable.       needed.

23. The contractor's material accounting  For earned value management
system will provide for:                  systems, the material
a. Accurate cost accumulation and         accounting system will
assignment of costs to cost accounts in   provide for:
a manner consistent with the budgets      Accurate cost accumulation
using recognized, acceptable costing      and assignment of costs to
techniques.                               control accounts in a manner
                                          consistent with the budgets
                                          using recognized,
                                          acceptable, costing
                                          techniques.

b. Determination N of price variances by  o comparable provision.
comparing planned versus actual
commitments.

c. Cost performance . measurement at the  . . but no earlier than the
point in time most suitable for the       time of progress payments or
category of material involved, but no     actual receipt of material.
earlier than the time of actual receipt
of material.

d. Determination of N cost variances      o comparable provision.
attributable to the excess usage of
material.

e. Determination of N unit or lot costs   o comparable provision.
when applicable.

f. Full F accountability for all          ull accountability for all
material purchased for the contract,      material purchased for the
including the residual inventory.         program, including the
                                          residual inventory.

Analysis                                  Analysis and management
                                          reports

24. Identify at the cost account level    At least on a monthly basis,
on a monthly basis using data from, or    generate the following
reconcilable with, the accounting         information at the control
system:                                   account and other levels as
a. Budgeted cost of work scheduled and    necessary for management
budgeted cost of work performed.          control using actual cost
b. Budgeted cost of work performed and    data from, or reconcilable
applied (actual where appropriate)        with, the accounting
direct costs for the same work.           system:
c. Variances resulting from the above     1. Comparison of the amount
comparisons classified in terms of        of planned budget and the
labor, material, or other appropriate     amount of budget earned for
elements together with the reasons for    work accomplished. This
significant variances.                    comparison provides the
                                          schedule variance.
                                          2. Comparison of the amount
                                          of the budget earned with
                                          the actual (applied where
                                          appropriate) direct costs
                                          for the same work. This
                                          comparison provides the cost
                                          variance.

25. Identify on a monthly basis, in       Identify budgeted and
detail needed by management for           applied (or actual) indirect
effective control, budgeted indirect      costs at the level and
costs, actual indirect costs, and         frequency needed by
variances along with the reasons.         management for effective
                                          control, along with the
                                          reasons for any significant
                                          variances.

26. Summarize the data elements and       Summarize the data elements
associated variances listed in [24] and   and associated variances
[25] above, through the contractor        through the program
organization and work breakdown           organization and/or work
structure to the reporting level          breakdown structure to
specified in the contract.                support management needs and
                                          any customer reporting
                                          specified in the contract.

27. Identify significant differences on   Identify, at least monthly,
a monthly basis between planned and       the significant differences
actual schedule accomplishment and the    between both planned and
reasons.                                  actual schedule performance
                                          and planned and actual cost
                                          performance, and provide the
                                          reasons for the variances in
                                          the detail needed by program
                                          management.

28. Identify managerial actions taken as  Implement managerial actions
a result of criteria items [24] through   taken as the result of
[27] above.                               earned value information.

29. Based on performance to date, on      Develop revised estimates of
commitment values for material, and on    cost at completion based on
estimates of future conditions, develop   performance to date,
revised estimates of cost at completion   commitment values for
for work breakdown structure elements     material, and estimates of
identified in the contract and compare    future conditions. Compare
these with the contract budget base and   this information with the
the latest statement of funds             performance measurement
requirements reported to the government.  baseline to identify
                                          variances at completion
                                          important to company
                                          management and any
                                          applicable customer
                                          reporting requirements,
                                          including statements of
                                          funding requirements.

Revisions and access to data              Revisions and data
                                          maintenance

30. Incorporate I contractual changes in  ncorporate authorized
a timely manner, recording the effects    changes in a timely manner,
of such changes in budgets and            recording the effects of
schedules. In the directed effort before  such changes in budgets and
negotiation of a change, base such        schedules. In the directed
revisions on the amount estimated and     effort prior to negotiation
budgeted to the functional organization.  of a change, base such
                                          revisions on the amount
                                          estimated and budgeted to
                                          the program organizations.

31. Reconcile original budgets for those  Reconcile current budgets to
elements of the work breakdown structure  prior budgets in terms of
identified as priced line items in the    changes to the authorized
contract, and for those elements at the   work and internal replanning
lowest level of the DOD program work      in the detail needed by
breakdown structure, with current         management for effective
performance measurement budgets in terms  control.
of (a) changes to the authorized work
and (b) internal replanning in the
detail needed by management for
effective control.

32. Prohibit retroactive changes to       Control retroactive changes
records pertaining to work performed      to records pertaining to
that will change previously reported      work performed that would
amounts for direct costs, indirect        change previously reported
costs, or budgets, except for correction  amounts for actual costs,
of errors and routine accounting          earned value, or budgets.
adjustments.                              Adjustments should be made
                                          only for correction of
                                          errors, routine accounting
                                          adjustments, effects of
                                          customer or management
                                          directed changes, or to
                                          improve the baseline
                                          integrity and accuracy of
                                          performance measurement
                                          data.

33. Prevent revisions to the contract     Prevent revisions to the
budget base except for government-        program budget except for
directed changes to contractual effort.   authorized changes.

34. Document internally, changes to the   Document changes to the
performance measurement baseline and      performance measurement
notify the procuring activity             baseline.
expeditiously through prescribed
procedures.

35. Provide the contracting officer and   No comparable provision.
the contracting officer's authorized
representatives with access to the
information and supporting documents
necessary to demonstrate compliance with
the cost/schedule control system
criteria.
----------------------------------------------------------------------



(See figure in printed edition.)APPENDIX II
COMMENTS FROM THE DEPARTMENT OF
DEFENSE
=========================================================== Appendix I



(See figure in printed edition.)



(See figure in printed edition.)


*** End of document. ***





FAS | Military Analysis | GAO |||| Index | Search |


Maintained by Webmaster