HealthCare.gov: what a troubled launch teaches architects

At a glance


Organization	US Centers for Medicare & Medicaid Services (CMS)
Period	2011 to 2014
Problem	A national health-insurance marketplace that many people couldn’t use when enrollment opened
Findings	Unsettled requirements, unclear ownership, delayed reviews and no verification of performance before launch
Themes	Architecture governance, non-functional requirements, integration, contractor management

What happened

HealthCare.gov is the federal marketplace where people in many US states compare and buy health insurance. Its two main parts were the federally facilitated marketplace (FFM), the shopping and enrollment system, and the federal data services hub, which checks applicants’ information against other government systems.

Enrollment opened on October 1, 2013. Many people could not get into or use the website. In July 2014 the Government Accountability Office (GAO), Congress’s audit agency, published its review of how CMS had planned and managed the work.

What the GAO found

Requirements were unsettled. CMS issued the main contracts before key requirements were known, such as how many states the federal marketplace would serve and how many people might enroll. Requirements kept changing, which caused delays, wasted work and cost growth.

Ownership was unclear. Guidance was unclear about who could approve contractors spending more money, and oversight was inconsistent.

Reviews were delayed and skipped. The marketplace’s readiness review moved from March to September 2013, weeks before launch, and required approvals were not obtained. CMS launched without verifying that the system met its performance requirements.

Contractor management was weak. CMS raised concerns about the marketplace contractor’s quality and responsiveness in writing in April and November 2013, and moved its staff into the contractor’s offices in September 2013 to direct work on site.

Costs grew sharply. Money committed to the marketplace rose from $56 million in September 2011 to more than $209 million by February 2014, and for the data hub from $30 million to nearly $85 million. A new marketplace contract signed in January 2014 for $91 million had risen to more than $175 million by June 2014, with some key features still unfinished.

flowchart LR
    user[Applicant] --> ffm[Marketplace: shop and enroll]
    ffm --> hub[Federal data services hub]
    hub --> a[Other federal agencies]
    hub --> b[States]
    ffm --> ins[Insurers]

Architecture lessons

None of GAO’s findings is about a particular technology. They are about the conditions architecture needs:

Fix the scale requirements early. Expected users, peak load and the number of integrating partners shape the whole design. Starting without them guarantees rework.
Treat non-functional requirements as acceptance criteria. Performance, capacity and availability must be tested and signed off before launch, not assumed.
Integration is the riskiest part. A hub connecting many agencies, states and insurers needs end-to-end testing with real partners well before go-live.
Give one accountable owner the decisions. Who can change scope, approve spend and declare readiness must be clear.
Plan for incremental release. A soft launch, a waiting room, or opening to a subset of states or users limits the blast radius of first-day problems.

Sources

US Government Accountability Office, “Healthcare.gov: Ineffective Planning and Oversight Practices Underscore the Need for Improved Contract Management” (GAO-14-694) (opens in a new tab), July 2014

Written and edited by Abhinav Taduka, enterprise software architect.