While checking my morning email, I saw notes from several friends who run tiny businesses. Each told me that they rely on Square, the popular online payment system, and that it wasn't working for them and did I know how to fix it. I didn't. I still don't. I could tell them that they weren't alone. Many people using both Square and Cash App reported they couldn't make or take payments.
The disruptions began on Thursday afternoon at approximately 4:30 PM Eastern time, as indicated by a surge in reports on the outage tracking website, DownDetector.
Square reported it was investigating a disruption impacting "Square Stands and Readers connected via USB on Point of Sale version 6.24. Impacted Sellers may see a message indicating that the reader has disconnected. If you are seeing this on your account, it is recommended that you connect to Bluetooth for the time being. If you have not updated to 6.24 yet, we recommend waiting to do so until we hear more information."
On Thursday, Square saw a peak of over 18,000 outage reports, while Cash App had more than 9,000 reports by 7 PM Eastern. Although the number of reported issues decreased, they persisted into Friday morning, with both services receiving over 1,000 outage reports around 10 AM Eastern.
The result? Small businesses, from taco trucks to coffee shops, had to tell their customers they couldn't accept credit cards or e-payments.
Square recommended that users switch to offline mode. That didn't work well either. Users reported, "We had thought [we] had taken offline payments, [but] when we went to check recently, there were hours of transactions unlogged."
The company claims, however, that "Offline Mode Payments … are being uploaded, but there will be a slight delay before they appear as completed. Any new Offline Mode Payments will be completed as normal in the coming hours."
Meanwhile, with Cash App, the company urged users not to reattempt any payments or deposits that showed error messages and said users should not log out of the app. Why? We don't know.
On X, Cash now says, "Our services are mostly back online. We're sorry if you haven't been able to use parts of the app." Which parts? The ability to send payments and get cash out. The main parts.
Customers are understandably upset. To put it more bluntly, as one did on X, "I NEED MY MONEY." Imagine.
Both companies, subsidiaries of Block, claim that all's well now. Customers tell another story.
DownDetector data showed that as of Friday morning, there were nearly 1,500 reports related to Square's app, website, and payment processing issues. Still, this was a significant drop from the 18,000 reports on Thursday night. Cash App had close to 1,700 reports concerning fund transfers and other payment issues.
The exact number of affected customers and the extent of the problem remain unclear. Both Cash App and Square have yet to respond to requests for comments on the matter.
In its Friday update, Square mentioned that it is closely monitoring the situation after its engineering teams implemented a fix. The company anticipates "slight delays" in some transfers due to the disruption.
The outage's root cause remains undisclosed. Neither company has yet to provide an official statement on the matter.
Kolton Andrus, CTO and founder of Gremlin, a company specializing in understanding software failure and how to avoid it, said this episode "underscores just how serious the problem is. Software reliability remains a struggle. Why? Well, one reason is that most of the current best practices, like SLOs [service level objective] and incident response, are reactive: they only kick in after something bad has happened." Companies must develop strategies and best practices to proactively manage software reliability at an enterprise scale.
After all, Andrus continued, "The downstream impact of outages can be severe: small businesses and retail stores not able to pay their rent this month can be a catastrophic outcome of software reliability failure. We've not thought enough about what happens downstream when software fails."
Square and Cash App users can tell you exactly what it's like: Not good. Top-of-the-stack service companies need to address these issues, or they'll find their customers looking for someone who takes their service commitments more seriously.