March didn't get off to a great start for over 23,000 certificate owners, after the general manager of a major HTTPS certificate reseller emailed his customers' private keys to a partner -- compromising the certificates and forcing their revocation.
The saga blew up in the last two days -- and it's a complicated mess. The end result are confused customers scrambling for answers.
It began when customers reported that DigiCert, a major internet issuer of HTTPS certificates, emailed customers of Trustico, an HTTPS certificate reseller, to warn them that their HTTPS certificates had been reported "compromised" by Trustico.
DigiCert said the certificates will be revoked, forcing sites to drop HTTPS altogether until they get a new certificate.
The reasons for why and how the certificates were compromised are unclear. Just weeks earlier, Trustico said it would stop issuing certificates issued by Symantec, which last year was bought by DigiCert, citing Google's decision to stop trusting Symantec certificates after the company violated industry-wide rules that govern certificates issuance.
Trustico emailed DigiCert to revoke some 50,000 certificates it issued -- presumably to preempt disarray in the near future when those soon-to-be-untrusted certificates start throwing up browser security warnings -- which the company said in a statement it was allowed to do. (Some in the security community believe that Trustico's revocation request was in part to force customers to move their certificates to the reseller's other partner, Comodo, which it will use instead of Symantec.)
DigiCert refused, citing those same industry rules, which say only customers can revoke their certificates, unless there is actionable proof that the private keys were compromised.
"At my request for proof of compromise, we received a file with 23,000 private keys matched to specific Trustico customers," said Jeremy Rowley, DigiCert product chief, in an email to a public security list.
That email containing the keys, sent by Trustico, forced DigiCert to trigger a mass revocation of the 23,000 certificates, much to the chagrin of Trustico's director Zane Lucas, who said in a reply that he was "not informed" that DigiCert would later email Trustico's customers informing them of the revocation.
What followed became a match of splitting hairs over technicalities and alleged rule breaking and subtle threats of legal action.
But a point of contention was that the private keys were able to be sent to DigiCert in the first place.
"Keeping 23,000(!) private keys at your online platform seems more than alarming and is careless and the public should be made aware of this fact," said one reply.
Baffling to some is that the private keys were compromised in the first place. HTTPS certificates are secured with a private key that the customer owns, and nobody else. If that private key is stolen, it can be used to impersonate a site or siphon off data to and from the company's servers.
Eric Mill, an expert in web encryption, said in the email thread that Trustico's "storage of private keys appears to be done without the user's knowledge or consent."
Those private keys were later posted in order to ensure browsers are told to reject the certificates.
One of those certificates secured a government email server, according to Kevin Beaumont, a security researcher and expert, in a tweet.
What's worse, he said, is that the private keys aren't secured with a password.
As of Thursday, DigiCert is standing its ground and has so far refused to revoke the remaining 27,000 certificates.
The case isn't expected to let up anytime soon.
The certificate issuing industry had its fair share of controversies, but this latest incident could be one of the worst -- a security focused industry can't be built on trust when the private keys are anything but private.