Government admits to design flaws in healthcare.gov

Here we are on day seven of healthcare.gov and the dominant story still seems to be that completed applications are presumed, but not proven to exist.

Finally, the Department of Health and Human Services (HSS) admitted, in this report by the Wall Street Journal [paywalled], that flaws in the design both of the software and hardware for healthcare.gov needed to be fixed.

They'll have to move quickly. Remember, this is open enrollment for 2014, but if December 15, 2013 (the deadline for 2014 enrollment) comes and it's still not a good situation then the site has failed its mission.

I know I've tried since day one and failed. I've read many stories of friends and others who have tried and I've yet to meet anyone who has gotten all the way even to getting quotes, let alone completing an application.

CNBC cites insurance industry sources as saying that "as few as 1 in 100 applications on the federal exchange contains enough information to enroll the applicant in a plan," and that half the applications insurance companies are receiving are incomplete and cannot be processed further.

O'Reilly columnist James Turner, beat me to this article, asking "What Developers Can Learn From healthcare.gov". His first thought was mine: "It has become abundantly clear that the site was never stress-tested under anything like the type of load it is encountering."

In one news story I heard about the problems people are having, the "expert" said, "of course, they couldn't test it as millions of people trying it at the same time." Putting aside the assumption that it's really millions of people, of course they can test that.

There are lots of great tools and services (HP LoadRunner, RadView, Neustar, etc.) for simulating large traffic loads.

And healthcare.gov is hosted on an Akamai netblock. I have asked Akamai if they provide such services; I suspect that with a content delivery network (CDN) like Akamai, load testing is trickier than on a normal network. I haven't heard back from Akamai yet.

Turner also noticed, as I did, that the application process seems segmented into various stages, and users are queued up for admission to each stage. It can be some time before you are brought to the correct page for the next stage. I've waited 15 or 20 minutes more than once. In a more intelligent system, the tasks in each stage could be broken up more so that progress could be made more steadily. If nothing else, this would make the user experience better.

He also notes that there are simple errors in the instructions. Usernames for the system must have numerals in them, which in itself is a debatable requirement, but the instructions don't say that they are required and the error message you get for not having them doesn't note the nature of the error. I've gotten this one, and was told simply that "larry.seltzer" was not an available username. At the time, I assumed that someone had already taken it.

I feel safe concluding, both from the nature of the errors and from the fact that they're on Akamai that bandwidth is not the problem. This shouldn't be a bandwidth-intensive application anyway.

Could it be server resources? In the era of cloud computing, one of the things you can do is buy extra capacity for when you expect heavy traffic. One would think that they did this; if not, then they failed to account for perfectly predictable errors.

HHS gave a few large volume numbers for visitors to the site, but no numbers for successful applications. The evidence we've gotten for that is anecdotal and, in at least one case that got a lot of news attention, false.

On Saturday, the Department of Health and Human Services announced that the site would be taken down for hours at a time in the middle of the night on weekends for scheduled maintenance.

President Obama is asking us not to give up, but it's hard not to get frustrated when you can go through a long part of the process only to be terminated with an inexplicable error. And now that HHS has admitted to the design flaws, it makes perfect sense to give up until we hear that they've done a better job.

Can you imagine if Amazon.com or any other private web service launched with this level of service?

There simply wouldn't be any excuse for it.