AI startup SiMa.ai debuts 'purpose-built' AI chip for edge computing
SiMa says its MLSoC, shown here in its packagee, is the first purpose-built chip to handle not just the matrix multiplication operations of AI in embedded use cases, but also traditional functions of computer vision that need to run in the same application.
Within the very broad landscape of artificial intelligence computer chips, products to serve the "edge" market, which encompasses drones, the internet of things devices, phones, and low-power server environments, provide fertile ground for vendors as one of the less-developed areas in the market compared to data center technology.
As ZDNet reported earlier this year, dozens of startups have been getting tens of millions in venture funding to make chips for AI in mobile and other embedded computing uses. Because the edge market is less settled, there are plenty of different ways that vendors will approach the problem.
On Tuesday, AI chip startup SiMa dot ai formally unveiled what it calls its MLSoC, a system-on-chip for speeding up neural networks with lower power consumption. The company argues that the new chip, which has begun shipping to customers, is the only part that is "purpose-built" to handle tasks with a heavy emphasis on computer vision tasks, such as detecting the presence of an object in a scene.
"Everybody is building a machine learning accelerator, and just that alone," said Krishna Rangasayee, co-founder and CEO of SiMa.ai, in an interview with ZDNet.
"What is very different about the embedded edge market," versus cloud computing, said Rangasayee, is that "people are looking for end-to-end application problem solvers," rather than just a chip for machine learning functions.
"They're looking for a system-on-a-chip experience where you can run the entire application on a chip."
Competitors, Rangasayee said, "Handle a narrow slice of the problem" by only performing the neural net function of machine learning.
"Everybody needs ML, but it's one portion of the overall problem, not the entire problem," Rangasayee said.
Also: The AI edge chip market is on fire, kindled by 'staggering' VC funding
Built with Taiwan Semiconductor's 16-nanometer fabrication process, the SiMa.ai chip has multiple parts fashioned as a single chip. The parts include a machine learning accelerator and code-named "Mosaic," which is dedicated to matrix multiplications that are the foundation of neural net processing.
Also onboard is an ARM A65 processor core, often found in automobiles, and a variety of functional units to aid in the specific task of vision applications, including a standalone computer vision processor, a video encoder, and a decoder, 4 megabytes of on-chip memory, and a multitude of communications and memory access chips, including an interface to 32-bit LPDDR4 memory circuits.
The chip hardware comes with SiMa.ai software to make it much easier to tune for performance and to handle many more workloads.
More details on the MLSoC are available on SiMa.ai's website.
SiMa.ai's product is aimed at a variety of markets, including robots, drones, autonomous vehicles, industrial automation, and applications in the healthcare and government markets.
The government market has been a particularly swift adopter of the technology, Rangasayee said.
"I learned at my previous company how important software is, and this is really going to be dependent on the strength of our software," SiMa CEO Krishna Rangasayee said. "Yes, our silicon is great, and we are very proud of it. And without silicon you are not a company," he said. "But to me, that's the necessary function, not the sufficient. The sufficient function is to provide an effortless ML experience"
"I'm surprised how fast the government sector is moving," he said. The typical impression, noted Rangasayee, is that it takes governments five to seven years to procure new technology, but things are happening much faster than that. Applications the government is particularly interested in are things such as use of ML onboard tanks and for detectors that search for improvised explosive devices. Satellites are a promising application as well, he said.
"It's a multi-trillion-dollar-market still using decades-old technology," Rangasayee observed of the various civilian and government applications.
Many of today's computer vision systems for autonomous craft and other applications are using "traditional load-store architectures, Von Neumann architectures," Rangasayee said, referring to the basic design of most computer chips on the market.
The means, he said, that chips being used for machine learning and for computer have not advanced in terms of how they hand compute bandwidth and data combined with one another.
Also: To proliferate AI tasks, a starter kit from Xilinx, little programming required
"We have a unique ML SoC, the first system-on-a chip that comprehends ML, and so people can do classic computer vision, and solve legacy problems, in addition to ML, in one single architecture," Rangasayee said.
SiMa.ai has received $150 million in venture capital over multiple rounds from Fidelity Management & Research, the investment advisor to mutual fund giant Fidelity, and Dell Technologies Capital, among others. Longtime chip industry insider Lip-Bu Tan, formerly head of chip design software firm Cadence Design, is on SiMa.ai's board of directors.
The word "sima" is a transliteration of the Sanskrit word for "edge."
In addition to Rangasayee, Moshe Gavrielov, formerly the CEO of Xilinx, is the company's chairman.
The term Edge AI has become a blanket term to refer to everything that is not in a data center, though it may include servers on the fringes of data centers. It ranges from smartphones to embedded devices that suck micro-watts of power using the TinyML framework for mobile AI from Google.
SiMa.ai goes up against a raft of mobile and embedded competitors. In the edge market, competitors include AMD, now the parent of Xilinx, intellectual property giant ARM, Qualcomm, Intel, and Nvidia. However, those companies have been traditionally focused on larger chips running at far greater power, on the order of tens of watts.
The SiMa.ai chip boasts what its creators say is one of the lowest power budgets of any chip on the market to complete typical tasks such as ResNet-50, the most common neural net for processing the ImageNet tasks of labeling pictures.
SiMa offers the MLSoC on an evaluation board for applications testing.
The company said the part can perform 50 trillion operations per second, or "teraoperations," at a power draw of 10 teraoperations per second per watt. That means the part will consume 5 watts when doing neural network tasks, though it may go higher with other functions engaged.
That class of chip running at a few watts puts SiMa.ai in the company of a host of startups, including Hailo Technologies, Mythic, AlphaICs, Recogni, EdgeCortix, Flex Logix, Roviero, BrainChip, Syntiant, Untether AI, Expedera, Deep AI, Andes, and Plumerai, to name just the most obvious ones.
The only companies in that are "in our line of sight," Rangasayee said, are Hailo and Mythic. But, "our large differentiation is they are building ML accelerators only. We are building full ML SoCs."
Thanks to SiMai.ai's building in ARM cores and dedicated image circuitry along with the Mosaic neural network code, customers will have a greater ability to run existing programs while adding code from popular ML frameworks such as PyTorch and TensorFlow.
Also: To measure ultra-low power AI, MLPerf gets a TinyML benchmark
"The interesting thing to me is the pent-up demand for a purpose-built platform to support legacy is pretty high," Rangasayee told ZDNet. "They can run their application almost from day one — that's a huge advantage we have."
"We are the first company to crack the code on solving any computer vision problem, because we don't care for the code base, it can be in C++, it can be in Python, or any ML framework," Rangasayee explained. The broad support for programs, he said, inclines the company to view itself as the "Ellis Island" of chips. "Give us your poor, give us your tired ... We'll take 'em all!" he said.
That broad support means the company has a larger audience of tens of thousands of customers rather than just a niche, Rangasayee asserted.
Another point in the chip's favor, according to Rangasayee, is that it has ten times the performance of any comparable part.
"The thing our customers care about is frames per second per watt" in terms of the image frames in for every watt of power, Rangasayee said. "We are minimum 10 [times] of anyone," he said. "We are demonstrating that day in and day out to every one of our customers."
The company doesn't yet offer benchmark specs according to the widely cited MLPerf benchmark scores, but Rangasayee said the company intends to do so further down the road.
"Right now, the priority is to make money," Rangasayee said. "We are a very small company" at 120 employees. "We can't dedicate a team to do MLPerf alone."
"You could do a lot of tweaking around benchmarks, but people care about end-to-end performance, not just an MLPerf benchmark.
"Yes, we have numbers, and yes we do better than anybody else, but at the same time, we don't want to spend our time building in benchmarks. We only want to solve customer problems."
Although Tuesday's announcement is about a chip, SiMa.ai places special emphasis on its software capability, including what it calls "novel compiler optimization techniques." The software makes it possible to support "a wide range of frameworks," including TensorFlow, PyTorch, and ONNX, the dominant programming libraries that machine learning uses to develop and train neural networks.
The company said its software allows users to "run any computer vision application, any network, any model, any framework, any sensor, any resolution."
Said Rangasayee: "You could spend a lot of time on one application, but how do you get thousands of customers across the finish line? That's really the harder problem."
Toward that goal, the company's software effort, Rangasayee said, consists of two things: compiler innovations in the "front end" and automation in the "backend."
The compiler will support "120-plus interactions," which affords the "flexibility and scalability" of bringing many more kinds of applications into the chip than would ordinarily be the case.
The backend portion of the software uses automation to allow more applications to be "mapped into your performance" rather than "waiting months for results."
"Most companies are putting a human in the loop to get the right performance," said Rangasayee. "We knew we had to automate in a clever way to get a better experience in minutes."
That software innovation is designed to make the use of the MLSoC "pushbutton," he said, because "everyone wants ML; nobody wants the learning curve." That is an approach that Rangasayee's former employer, Xilinx, has also taken in trying to make its embedded AI chips more user-friendly.
"I learned at my previous company how important software is. And this is really going to be dependent on the strength of our software," said Rangasayee. "Yes, our silicon is great, and we are very proud of it, and without silicon you are not a company," he said.
"But to me, that's the necessary function, not the sufficient; the sufficient function is to provide an effortless ML experience."