California payroll: Another failure waiting to happen?

California payroll: Another failure waiting to happen?

Summary: The State Controller’s Office is launching an assessment to recover its failed payroll project. This new project seems doomed even before it starts.


The State of California is soliciting bids for contractors to conduct an “independent assessment” for the $371 million failed payroll project known as MyCalPays and today called 21st Century Project.

Project history. The MyCalPays project was started in 2004 and suffered a series of reversals, including the firing of Bearingpoint, the original vendor. In 2010, California hired software company, SAP, to continue the project; by 2013, SAP was also fired, amid a storm of accusations and counter-accusations

A failure analysis (PDF download) conducted by the state concluded:

It will be difficult to determine, if anyone ever does, exactly what doomed the project, the largest payroll modernization in the nation. The system was designed to take over human resources and payroll responsibilities for 240,000 state employees. At the end, the project had nearly tripled in cost, to $373 million, and was years behind schedule.

In March, 2014, the state’s Legislative Analyst’s Office released an informative update (PDF download) describing the project’s history.

Current situation. Today, the state seeks an independent vendor to examine the current situation and make recommendations on how to proceed. The budget for this assessment is $1.275 million.

The Request for Offer (PDF download) document describes the deliverable goals such as the following:

Determine which system requirements are met and which ones remain unmet by the MyCalPAYS system.

identify what deliverables are useable given the Contractor’s understanding of the requirements that have been met, what requirements remain unmet, and an analysis of the scalability of the existing design to the statewide population. The Contractor shall identify all areas that must be redesigned in order for the system to serve the 240,000 statewide workforce population.

Determine the cost for a vendor to complete (as needed) modification and implementation of the MyCalPAYS system

The Contractor will identify a single price that a prospective integrator is likely to charge to complete the project

In short, the findings and recommendation deliverable(s) must answer the following questions:

  1. Which requirements are met by the MyCalPAYS solution, as currently implemented?
  2. What requirements have not been met with the MyCalPAYS solution?
  3. What portion of the current MyCalPAYS design and system may be usable?
  4. What is the estimated cost for a vendor to complete (as appropriate) modifications and implementation of the system ensuring that it is compliant with the stated system requirements?

The state specifies the following requirements for team members assigned to this assessment project:

  1. Two full implementation cycles of SAP payroll
  2. Five years SAP payroll experience
  3. System assessment

In addition, the contracted firm must have performed at least two successful IT project assessments similar to this one and have five years' relevant experience.

One staffing firm, b2b workforce technologies, offers a job listing corresponding to these requirements. Since this business appears to be a recruiting company, rather a system integrator, presumably a vendor hoping to bid on this assessment asked b2b to find qualified people. I left a message for their Vice President of Recruiting, but he did not return my call.

The RFO document describes the state’s responsibilities on this assessment project being rather minimal:

  • Designate a contact point
  • Provide contractor personnel with a “standard cubicle”
  • Ensure access to state staff
  • Review and approve (or reject) deliverables


In my opinion, as someone who regularly advises CIOs on strategies for IT success, I believe this assessment is doomed for the following reasons:

Complex requirements. In effect, the RFO document asks the contractor to figure out what went wrong, what works or does not work, how to fix these problems, and what is necessary to make the system scale.

In addition, the contractor must determine the cost for a vendor to complete the project. Also, the contractor must identify the “single price a prospective integrator is likely to charge to complete” the integrations required.

The requirements also seem predicated on the assumption that existing technology is salvageable. The state should be open to the possibility that it may be necessary to discard previous work and start over.

In summary, the project underlying this assessment is sufficiently complicated that even the auditors could not untangle all the threads.

Minimal state involvement. The state’s role seems defined to push full responsibility for completing the project onto the vendor. Given the intertwined nature of vendor and client in these situations and the negative history of this particular project, establishing such boundaries seems unrealistic and naive.

The state should recognize that collaboration is key to success on this kind of initiative; silos don’t work.

Politics is baked in. Rather than start from a fresh slate, looking for the best solution, the RFO includes a one-sided project history casting blame on SAP without accepting any responsibility itself. For example:

SAP’s failure to meet its contractual obligation to cure, left the State with no choice but to terminate the contract with SAP

Given the track record of this project, it is time for the state to acknowledge its shortcomings rather than blame everyone else. As the old saying goes: “Accepting responsibility is the first step to learning and improving.”

By casting blame from the start, the state begins this project without a critical examination of its crucial role in this situation. If history is a guide, the state will likely continue to blame external vendors despite its own shortcomings.

Inexperience. Great people can overcome almost any hurdle, making the relatively minimal personnel requirements on this project so surprising. I believe that personnel with only five years’ experience are not qualified to undertake this project successfully, given the size, scope, history, and politics. I asked two HR technology experts to comment on this issue.

Naomi Bloom is one of the most respected HR technology advisors in the world. She raises concerns about the level of experience needed to execute this assessment successfully:

This kind of assessment requires someone who has worked on and reviewed similar projects for many years. The right person should possess very strong SAP payroll expertise, so they can sort out the issues in a reasonable period. In addition, they must have sufficient confidence to stand up to the politics of the situation. If I were recruiting for this, I’d want the most experienced person I could find.

Jarret Pazahanick is a top SAP and SuccessFactors consultant, with deep experience in SAP payroll:

The sad truth is these public sector projects are often low bid or have several consulting “layers” so they rarely get the highest caliber consultants; seeing the job posting, it appears that history could repeat itself. I would advise the State to find an extremely senior consultant to help lead them in this important discovery exercise.

I raised concerns about this qualifications issue with a spokesperson for the California Department of Technology. He told me that five years is the "minimum" level of experience and that bids will come from vendors who are “pre-qualified.” He added this contract does not have to be awarded to the lowest bidder.

To address my concerns, the spokesperson said, “The people who drafted the RFO document are very experienced in project management.” When I asked whether these folks are the same ones who managed the original failed project, he stumbled a bit and then said, “no,” because there were personnel changes. However, the RFO says there are “seven (7) project team members (both staff and consultants) in the Project Management Office remaining.” Hmmm....

I’m skeptical about this whole endeavor, but you can draw your own conclusions.

Update Aug 30: A further thought and point of advice for the Department of Technology and the State Controller's Office... Is there a single person responsible for this project? I don't mean the project manager, but a trusted executive sponsor whose credibility and integrity are beyond question; someone able and willing to roll up their sleeves to bring the various interest groups together around project definition, goals, and execution. If the answer is no, then you must start there.

Frankly, I don't think you are ready to move forward right now and I wonder about any firm willing to take on the project at this time, given the circumstances.

Also read

Did California’s CIO mislead public on IT success?

California abandons $2 billion court management system

Understanding Marin County’s $30 million ERP failure

Turnaround strategy for Los Angeles school district payroll failure

Bizarre ERP implementation experiment at Arizona State

The IT failures blame game (part 2)

Who’s accountable for IT failure? (part one)

Exploring the Devil’s Triangle

Topics: CXO, Enterprise Software

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Hmmmmmmm......

    I'm glad i moved out of California back in 2010...... : )
  • Same old pattern

    Give me something

    What do you want?

    I don't know you decide.

    Okay, here you go.

    That's not what I wanted. You're fired! And here's a lawsuit because you didn't fulfill requirements!
    Buster Friendly
    • Replication of the LAUSD payroll project

      Just inflated to Godzilla size .... or Pacific Rim ;-)
    • We do expect...

      ...governments to at least handle payroll efficiently and correctly. This should cause people to be fired.
      John L. Ries
  • SAP a big expense and an even bigger headache on the best of days.
    • You really think?

      You really think the problem is SAP and not the CA government?
      Buster Friendly
      • Earth to Buster

        IT system failure is not just a government thing.

        SAP vs. Waste Management:

        IBM vs. Avantor Performance Materials:

        IBM vs. Bridgestone Americas, Inc.:

        All of these examples involved the use of SAP and there was no government agency involved.

        P.S. Am not picking on SAP.
        Rabid Howler Monkey
        • Blame the Contractor ....

          Yeah ..... If the buying party cannot manually define, properly document, and define clear cut requirements, projects of these type will always fail. Unfortunately that is the plurality plus of most of these.
          • rhonin: "Blame the Contractor .... "

            You clearly read something that was not in my comment. That every time an IT project fails it's the fault of the contractor.

            News flash! Large IT projects are complex beasts and *both* the client and the contractor organizations have to get things right in order to succeed.

            My point was that IT project failure is not strictly a government-related event. In addition, there are multiple reasons for large IT project failure and requirements are not the only reason. I have personally seen IT projects fail due to lack of buy in by affected organizations and individuals, including those in low- to mid-level management. And, sometimes, again for various reasons, the contractor really is not up to the task.
            Rabid Howler Monkey
  • Question

    Was the contracting agency the Controller's office, or a department under the jurisdiction of the Governor? I would expect the former, given that the Controller is responsible for writing the checks.
    John L. Ries