DTA knew of weak iPhone COVIDSafe performance despite go-live
The Digital Transformation Agency (DTA) knew that Australia's COVIDSafe coronavirus contact tracing app had severe flaws, despite sending it out for public use on 26 April 2020.
Documents [PDF] published by the agency on a Friday before the June long weekend revealed that Bluetooth encounter logging tests conducted on the day of the app going live showed locked iPhones, an iPhone X to iPhone 6 specifically, were transmitting data at a "poor" rating -- 25% or below.
This was slightly improved on May 14 to a "moderate" rating, which is a 25% to 50% success rate. As at May 26, this rating sat at moderate.
Software engineer Richard Nelson had earlier this week published research that showed locked iPhones were practically useless when it came to logging encounters through COVIDSafe.
He said a locked iPhone with an expired ID could not generate a new ID and that, without an ID, the device would record other devices around it, but it could not be recorded by others.
"A device in this state will record other people around it, but will not be recorded by others. If all relevant devices are in this state, no encounters are logged," he wrote.
"One could imagine Alice packing her bag, putting her iPhone in, and going out for the day to a football game. With her device in this state, nobody else will record her presence, and if anyone around her tested positive she would not be contacted."
The DTA said in May that functional and performance testing was conducted for the Apple iOS and Google Android versions of the COVIDSafe App prior to release, saying at the time that a total of 179 functional tests were conducted, including Bluetooth encounters between various device types, in various states.
"All tests satisfied the baseline design requirements," the DTA said. "Performance tests were also conducted against the technical requirements."
The DTA said in these tests, the system had "met and sustained the requirements and remained stable through the testing process".
But the DTA document reveals that it isn't just locked iPhones that have struggled with performance. It detailed that an active iPhone X -- which, upon release, set Aussie's back a mere AU$1,579 for the 64GB base model -- to a locked 2015 model iPhone 6s encounter was performing at "moderate" on April 26. Its status only changed to "good", which is a 50% to 80% success rate, on tests conducted a month later.
It wasn't just iPhone to iPhone encounters recording errors in tests conducted on the day the app went live; Galaxy S10 to iPhone X encounters performed at 25% or below for both active to locked and locked to locked.
A month later, both statuses were recorded as moderate.
Android to Android pings were working "excellent", at least for the Samsung Galaxy S10, which for the 128GB model currently retails from AU$1,149, to Note 9, which buying through Telstra will see a minimum of AU$1,152 spent, where active to active, active to background unlocked, active to locked, and locked to locked tests were concerned.
The DTA said the devices used as part of Android testing of the app functionality itself were the Huawei Pro 30, Samsung 9, Motorola Razor, Oppo R17, and Pixel 2. These devices all passed the 179 tests, which included a user registering and launching the app from their device.
For iOS, the devices used as part of DTA testing were an iPad, iPhone SE, iPhone 6, iPhone 8, and iPhone X. All 201 iOS tests returned a pass mark.
Items listed as out of scope for its tests on both Android and iOS were security and penetration testing and load and stress testing.
In announcing the app, the government said that in order for it to be effective, users should have the app running in the background when coming into contact with others.
"Your phone does not need to be unlocked for the app to work," were Minister for Government Services Stuart Robert's exact words at many a press conference in the weeks following COVIDSafe launch.
As of Friday, over 6.3 million Australians have downloaded the app.
MORE ON COVIDSAFE
- Locked iPhones rendered almost useless in Australia's COVIDSafe tracking efforts
- COVIDSafe update closes denial of service bug and makes notifications optional
- Australia's COVIDSafe contact tracing story is full of holes and we should worry
- 6 million COVIDSafe downloads and a AU$60b JobKeeper data error
- Health Minister denies states and territories unable to access COVIDSafe data
- COVIDSafe legislation enters Parliament with a few added privacy safeguards
- Australia's wobbly start to COVIDSafe app transparency