Salesforce deploys Einstein Vision to spot great white sharks
As the number of great white shark sightings off the coast of California has ticked up in recent years, beachgoers and researchers alike have been looking for more information.
Unfortunately, data on sharks can be hard to come by. The state of the art for tracking great whites, which involves tagging the animals, can only tell us so much. Some try to augment the data collection with more manual methods -- they may climb a nearby mountaintop with a pair of binoculars, or get into the water on a stand-up paddleboard, and simply scan the water.
With under-utilized data right at the ocean's edge, researchers from Salesforce and UC Santa Barbara saw an opportunity to leverage AI. This year, they've been scanning a portion of the California coastline with drones and analyzing the footage using Salesforce's computer vision technology in an effort called "Project SharkEye."
So far, the project has yielded some valuable and surprising results, according to UC Santa Barbara Prof. Douglas McCauley. The data-collection technique "gives me greater data to understand where and when and why sharks are increasing in number," he said.
Given that the presence of sharks is a strong indicator of a healthy marine ecosystem, that data can provide McCauley with valuable insight into the state of our oceans.
And as the father of a six-year-old who's just learning to surf, McCauley also values what the data can tell us about the safety of California beaches. Good information, he said, can help surfers and beachgoers "not sink into a sinkhole of irrational fear, but practice good decision-making."
McCauley, who leads UCSB's Benioff Ocean Initiative, teamed up with Salesforce AI experts and other researchers to launch Project SharkEye. The team leveraged Salesforce's Einstein Vision algorithm, which is typically used for visual search, brand detection and product identification. They trained the algorithm on a number of variables, such as how to distinguish between great whites and other shark species (such as the harmless leopard shark). The algorithm had to rule out non-shark objects, such as kelp floating in the water, and it had to determine whether a sighting was a unique shark or a previously-identified shark. The algorithm is now able to identify great whites with more than 90% accuracy.
After training the model, the team began observations on a single beach near Santa Barbara. Using a single, hobbyist drone and four FAA-certified drone pilots, they surveyed the coast nearly every day beginning in June. The footage was uploaded to the Salesforce platform, where Einstein analyzed it and flagged any sharks.
Over the course of the summer, the team recorded around 23 shark sightings (in some cases these sightings were of the same shark). Importantly, about half were juvenile sharks, less than eight feet in length. White sharks that young don't have the teeth structure to even attempt eating mammals, McCauley said.
The team also recorded more shark activity later in the summer -- counter to expectations.
"It gets us really curious about what we can take from this data, as we amass more and more, to build predictive models," McCauley said.
The research team made a point of engaging with local stakeholders, such as lifeguards and surf shop owners, keeping them informed of any shark sightings.
Next year, the team intends to roll out Project SharkEye as a toolkit that others can leverage with their own hobbyist drones.
"Our hope is by going public now, we'll get more interest and give people these free tools they can use with low-cost drones to capture that information, share and disseminate it," Michael Jones, senior director of research at Salesforce, said to ZDNet.
Salesforce plans to offer an adapted version of its Field Service Lightning mobile app (typically used for onsite job management) to stakeholders, so they can access the collected information.
Meanwhile, the project also enabled Salesforce to move forward the state of AI, Jones said. The team will be publishing an academic research paper covering the methodology used to avoid the problem of double-counting sharks. "Anybody using drones to count moving objects, they're going to encounter this problem," Jones said.
In addition to the challenge of counting moving sharks, the algorithm had to account for a number of variables that could impact object detection -- such as rain or other inclement weather, or high waves.
"Every day I read about AI breakthroughs, and most of those are in a lab in perfect conditions," Jones said. "This was very imperfect."