Archive

CoP News front page

Agile Community of Practice Newsletter Archive


Organization Documents

Latest PDAS Update (2021)

The latest update to the PDAS series shifts focus from MDAPs to MTA programs, prototype OTs, and DoD initiatives. Also, Nunn-McCurdy updates.

DoD Review of AoA Process

Report assessing implementation of the AoA by the Department: time required to complete, suggestions to improve, etc.

AAF Memo - Data Transparency

Ms. Ellen M. Lord memo discusses data transparency to enable the acquisition pathways. Signed June 15, 2020.

IPMDAR DID Memo

Authorizes use of the new IPMDAR DID. Signed April 22, 2020.

PDAS Series (2018-2020)

The annual PDAS reports use SAR data to examine Nunn-McCurdy breaches and cost performance / cost growth of MDAPs. See PDAS 2020, 2019, 2018.

Organization Overview

Presentation by Ms. Stacy Cummings on the vision of the ADA (formerly AE) organization, including the new Adaptive Acquisition Framework.

DoD Response to NDAA

Report to Congress, May 31, 2019 on the use of data analysis, measurement, and other evaluation-related methods in DOD acquisition programs.

2015 Report to Congress

Report to Congress on ADA (formerly PARCA), March 2015. See also previous reports from 2014, 2010, and 2009.

Public Law 111-23

The Weapon Systems Acquisition Reform Act (WSARA) of 2009, also known as Public Law 111-23.

Framing Assumptions

An informational paper providing guidance on Framing Assumptions (FAs), statements which shape cost, schedule, or performance expectations of an acquisition program.

Performance Assessments

Pursuant to Public Law 111-23, section 103, ADA carries out performance assessments of major defense acquisition programs (MDAPs) periodically or when requested by the Secretary of Defense, the Under Secretary of Defense for Acquisition and Sustainment, the Secretary of a military department or the head of a Defense Agency. Performance assessments evaluate the cost, schedule and performance of the program relative to current metrics including performance requirements and baseline descriptions.

Selected Assessments: Nunn McCurdy

ADA identified two root causes for the Nunn-McCurdy breach: the first accounting for one-quarter of the cost growth was due to factors exogenous to the program; and the second was that the Government did not follow adequate acquisition rigor to deal with uncertainty and risk inherent in large construction projects like ACWA, which develop and use new processes, handle dangerous materials, and are subject to comprehensive regulation. The Cost Assessment and Program Evaluation (CAPE) cost estimate created during Nunn McCurdy certification included significant costs for these risks, and the FY 2013 PB includes additional MILCON funding to be consistent with the CAPE estimate. The program has continued to retire some risk and is now rigorously monitoring burndown of remaining risk. Original cost estimates were established when designs at the Pueblo Chemical Agent-Destruction Pilot Plant (PCAPP) and Blue Grass Chemical Agent-Destruction Pilot Plant (BGCAPP) facilities were 60% and 13% complete, respectively. Both designs are now complete and construction is 98% complete at PCAPP and 60% complete at BGCAPP. ADA’s root cause analysis also described a lack of contractor incentive to reduce program uncertainty and cost. To further incentivize the contractors to complete agent destruction operations in a safe and accelerated manner, the program office initiated discussions with the contractor to implement the special milestone incentives authorized by the FY 2007 NDAA. These incentives ($164M) were incorporated in the contracts prior to 4Q FY 2012; however, the FY 2013 continuing resolution funding restrictions have impacted these contract actions.

ADA’s June 2012 root cause analysis of the Evolved Expendable Launch Vehicle (EELV) program identified three causes for the Nunn-McCurdy breach: first, the inherently unstable nature of the demand for launch services; second, the international space market and industrial base issues; and, third, poor program execution due to an environment with little incentive for cost control for the content not associated with the fixed infrastructure. The first two root causes were exogenous and beyond the program’s control. ADA believes an acquisition strategy that ensures a sufficient pool of competitive launches and a contract strategy that addresses enterprise fixed costs are important factors in addressing the third root cause. While an alternate launch provider’s ability to meet new entrant certification criteria is the first barrier to competition, there is danger of additional barriers if funding, schedule, and national security issues erode the pool of 14 cores in potentially competitive launches. Furthermore, the program’s ability to control fixed costs will significantly impact the government’s ability to realize cost savings from ULA and future potential competition for launch services. High fixed costs are also contrary to the need for agility as launch demands change. The Evolved Expendable Launch Vehicle (EELV) program was recertified as an ACAT 1D program on July 12, 2012.

ADA’s second assessment of the EELV program identified three root causes behind the Nunn-McCurdy breach: first, the inherently unstable nature of the demand for launch services; second, the international space market and industrial base issues; and, third, poor program execution due to an environment with little incentive for cost control for the content not associated with the fixed infrastructure. The first two root causes were exogenous and beyond the program’s control. ADA believes an acquisition strategy that ensures a sufficient pool of competitive launches and a contract strategy that addresses enterprise fixed costs are important factors in addressing the third root cause. There are two upcoming phases of competition for the EELV program that are contingent upon certification of a New Entrant. Furthermore, the Air Force Program Executive Office for Space Launch has no long-term concerns related to the launch forecast and believes the EELV program is well suited to react to changing launch manifest requirements. Finally, the Air Force continues to examine options to restructure EELV Launch Capability efforts to allocate discrete and unambiguous costs to each launch vehicle and payload. EELV received Milestone C re-approval on February 10, 2013.

The FY 2013 budget drastically changed the Global Hawk program by effectively terminating Block 30 and delaying the GSRA/CSRA subprogram initiation. The uncertainty created by the FY 2013 budget and by subsequent congressional language has made it difficult to establish meaningful baselines, requirements or long term planning. This makes sound investment decisions in the areas of reliability, maintainability, support, and modernization a challenge. With the exception of Material Reliability, performance metrics on the Global Hawk Block 30 have improved or stabilized since the June 2011 Nunn-McCurdy certification to continue.

This memorandum assessed the health of the Global Positioning System (GPS) constellation and identified key challenges representing significant risk to maintaining worldwide GPS coverage. These challenges include: delays in creating the next generation of ground control segment (OCX) and how these delays impact the replenishment of the constellation with new GPS-III satellites; aging of the GPS constellation and the importance of GPS-IIR satellites for a healthy constellation; and planning for contingency operations, which would mitigate risk to the constellation. The OCX schedule continues to slip from the original baseline, with current estimates approaching the Air Force estimate for when the constellation must be replenished with an operational GPS-III satellite. Currently, the OCX Block 1 ground segment is required before a GPS-III satellite can transmit a legacy signal. As aging GPS-IIA and -IIR satellites are retired from the constellation, GPS-III satellites and the OCX ground segment will be needed to meet constellation requirements. As the GPS-IIA satellites will be replaced before the IIR satellites, these IIR satellites must be maintained since their health is likely to drive overall constellation health until the GPS-III and OCX ground segment become available. Contingency operations would be a modification to the current ground control system, allowing use of GPS-III satellites before the delayed OCX Block 1 is completed.

ADA’s 2012 Root Cause Analysis identified four reasons for the Nunn McCurdy breach. Three causes exogenous to the program accounted for 190% of PAUC growth: first, the decision by the Army to eliminate all planned production; second, the Secretary of Defense’s direction to participate in a Combatant Command exercise; and, third, an Army decision to extend JLENS EMD by 12 months to support the Army Integrated Air and Missile Defense program. Engineering challenges accounted for the remaining 15% of cost growth. Since ADA’s December 2012 performance Assessment, the program has completed two Early User Testing events at the Utah Test and Training Range. The first test result was that JLENS is operationally effective with limitations; not suitable in the areas of Reliability, Availability and Maintainability and MANPRINT; and is survivable with limitations. There were 29 system aborts in the first test. Root causes of 24 of these aborts have been resolved. The radar system can detect targets, provide accurate tracks, and potentially support the Army’s Integrated Fire Control network; however, the soldier operators were poorly trained, the software was underdeveloped with undocumented work-arounds, and the system lacks Cooperative Engagement Capability integration and certification. The system has not met Electromagnetic Environmental Effects measures and has low availability. JLENS is a stand-alone system with no funding to support further development. After EUT testing, Orbit 1 will be moved to Aberdeen Proving Grounds to participate in exercise Noble Eagle. Orbit 2 will be stored at White Sands Missile Range in FY 2014.

The F-35 program continues to aggressively confront the large number of issues inherent in a complex development program. System development issues such as the Arresting Hook System (AHS), the Helmet Mounted Display System (HMDS), Envelop Expansion, and Fatigue Life are ongoing as new challenges are introduced. The program has made substantial changes that put it on a more realistic path to address significant development and production cost challenges, but subsequent performance has included schedule slips and delays to critical software releases. Software development, production costs, operations and support costs, and certification testing remain a risk. ADA will continue following flight test progress, production rates, costs, deliveries, and challenges associated with program concurrency.

This memorandum summarized the fourth assessment of the Remote Minehunting System (RMS) program, which was certified for continuation on June 1, 2010. ADA’s May 2010 root cause analysis identified three reasons for the Nunn-McCurdy breach: first, a decrease in quantity; second, an unrealistic cost estimate; and, third, poor program management and governance, particularly a failure to effectively address the Remote Multi-Mission Vehicle’s (RMMV’s) insufficient reliability. Since the 2010 Nunn-McCurdy breach, significant improvements have been made with the program. The last phase of V4.2 In-Water Testing is underway with preliminary data implying that V4.2 will meet the 75 hour Mean Time Between Operational Mission Failures RMMV Material Reliability requirement. The Program Office anticipates completion of V4.2 In-Water Testing in July, 2013. The program is likely to meet its RMMV reliability requirement without a V4.3 design iteration, leaving AN/AQS-20A reliability as the major hurdle to the RMS Operational Availability requirement. Shipboard testing on a Freedom Class seaframe is an important outstanding requirement. The program is on track to meet the May 2014 objective for Milestone C. An RMS Operational Assessment, a prerequisite for Littoral Combat Ship Mission Module IOT&E, is planned for early FY 2014.

Selected Assessments: Full Rate Production

The Gray Eagle is an Army ACAT IC program that provides tactical intelligence, video, imagery, communications relay, and precision missile support to Army maneuver units. The Gray Eagle completed Initial Operational Test and Evaluation (IOT&E) in August 2012 and was found to be operationally effective and suitable. As a result, the program was authorization to procure up to 49 Gray Eagle UAVs and delegated from an ACAT ID. During IOT&E, the program achieved Combat Availability requirements despite failure to meet subsystem reliability attributes, which have subsequently been revised to be consistent with operations and support funding levels.

This memorandum provided an assessment of P-8A Poseidon (P-8A) program performance issues ahead of the imminent Full Rate Production (FRP) authorization decision. P-8A is a Navy ACAT ID program that achieved Milestone C in August 2010. Of a total planned procurement quantity of 117, 85 (73%) remain to be procured through FRP. The P-8A airframe represents an improvement over the legacy P-3 airframe, and maintaining the production schedule reduces the risk for the fleet transition from the legacy P-3, allowing the Navy to maintain operational capabilities. However, hardware/software integration issues have resulted in mission area deficiencies that must be mitigated. The Navy’s incremental strategy addresses these issues, but contains known risks.

The SM-6 is a solid propellant, tail-controlled surface to air missile, which incorporates a separate booster that enables air defense to theater ranges. The original December 2011 Full Rate Production (FRP) review was deferred to perform supplemental testing to validate corrections that caused two previous reliability failures. Three of five Key Performance Parameters (KPPs) will not be fully demonstrated until Follow-On Test and Evaluation (FOT&E); however, combined modeling and simulation (M&S) and land-based testing provide some confidence in meeting these KPPs. As of February 2013, the one large active Low Rate Initial Production (LRIP) contract was 57% complete, ahead of schedule, and under budget. The proposed missile buy profile ramp-up in FY 2017 and beyond may not be affordable; therefore, ADA recommended it be adjusted in the FRP acquisition program baseline. ADA will follow the FOT&E results and production progress.

WIN-T Increment 2 takes the Increment 1 network capability mobile. The program has 932 of 2,100 (44%) procurement units under contract. The FRP decision in September 2012 was deferred because the Director, Operational Test and Evaluation (DOT&E) found that the program had limited effectiveness, was not operationally suitable, and was not survivable. The program proceeded with a series of Corrective Action Plans and completed a Follow-On Operational Test and Evaluation (FOT&E) to address these deficiencies. Prior to FOT&E the Army lowered reliability requirement Mean Time Between Essential Function Failure (MTBFF) a second time. The FOT&E was completed in May 2013 with improvement to the Soldier Network Extension (SNE), the line-of-sight Highband Network Waveform, and the SATCOM Net-Centric Waveform. Other improvements included higher data throughput speeds and resolution of multiple information assurance issues. The FOT&E also demonstrated a number of remaining deficiencies. The SNE and Point of Presence nodes start and restart procedures were complicated and time consuming, and Combat Net Radio gateways and Vehicle Wireless Package did not support the Fire Support Officer. The Army needs to address the remaining limitations and develop a long-term plan to resolve these limitations.

Root Cause Analyses

Pursuant to Public Law 111-23, section 103, ADA conducts root cause analyses "for major defense acquisition programs...when required by section 2433a(a)(1) of title 10, United States Code (as added by section 206(a) of this Act), or when requested by the Secretary of Defense, the Under Secretary of Defense for Acquisition and Sustainment, the Secretary of a military department, or the head of a Defense Agency."

Supporting FFRDC Reports

ADA utilizes Federally Funded Research and Development Centers (FFRDCs) to support its work in the Nunn-McCurdy Certification process. These FFRDC analyses represent the work of RAND Corporation and the Institute for Defense Analyses (IDA). These are independent reports that do not represent ADA's final assessment; nevertheless, they contain detailed information about the conditions resulting in cost growth.