Robots plus generative AI: Everything you need to know when they work as one
We keep dreaming -- albeit they're most often nightmares -- of intelligent robots such as Terminators. However, while robots are now essential for factories and making inroads in the home, no one could mistake them for being intelligent. Roomba cat rides and smart vacuums are fun and useful. Intelligent? Not so much.
But since generative AI exploded on the scene, people have started combining AI and robots to create a new age of smart robots. As we enter the last quarter of 2023, it's clear that the melding of these two will create an era of unparalleled technological synergy.
This is going to happen, by the way. As Vaclav Vincalek, virtual CTO and founder of 555vCTO.com, observed, "The real question is: 'How will generative AI integrate with robotics?' I just can't foresee how going forward, anyone building a smart robot wouldn't integrate generative AI in some way or another."
Companies such as Boston Dynamics and Sanctuary AI, Vincalek said, "have already made it their mission to build robots that are more than simply cameras on wheels." These companies, he added:
"...are striving for robots that can master any terrain and remain balanced. Generative AI for 'robot vision' will help such companies achieve this. Tasks like object detection, image segmentation, and image generation will improve at a staggering rate thanks to generative AI. This technology will also bolster fine-tuning one of the most difficult tasks in robotics: designing a hand that can both lift heavy objects AND handle fragile items like eggs."
Not enough human workers
These advances may come just in time. While we often worry about people losing their jobs to AI and robotics, in the long run, we need smart robots to fill jobs.
As Gearoid Reidy, a Bloomberg columnist, recently wrote, "Over half of [Japan's] businesses say they can't find enough full-time staff." With one in 10 people now over 80, there simply aren't enough workers to go around. For example, self-driving cars are still very much a work in progress, but when your alternative is 80-year-old taxi drivers, the need for self-driving, smart cars becomes much more urgent.
This is not an issue limited to Japan. Germany's industries are facing similar problems -- with baby boomers retiring and a smaller cohort of workers entering the job market due to low birth rates. The result? With 45.8 million workers today, Germany expects to lose 7 million of them by 2035. The solution? Ralf Winkelmann, managing director of FANUC Germany, told Reuters: "Robots [will] enable the survival of companies that see their future at risk due to staff shortages."
It's no different in the United States. For all the political hysteria over illegal immigrants stealing US jobs, according to the US Chamber of Commerce, the reality is that Americans are aging out of the workforce. At the same time, the growing number of elderly will need more care and support than ever.
Also: Gen AI a job threat? On the contrary, human workers have much to gain
Other jobs, especially in the hospitality and food-service industries, are going begging. No one wants to flip burgers for minimum wage. Why would they? The US federal minimum wage remains $7.25, while the national average minimum wage is $9.08. Neither have kept up with the cost of living in over 50 years. ZipRecruiter reports the US average hourly pay for a livable wage is $26.93. Looking ahead, it appears smart robots are one of the few viable answers for an ever-shrinking workforce.
Smart robots will also be able to handle mucky jobs that, frankly, almost no one wants to do. For example, people are incompetent at sorting out recycling stuff from ordinary trash. AMP Robotics, however, is using AI and computer vision to distinguish cans, bottles, paper, and many other packaging types and recyclables. The AI then guides robots to pick and place the material, automating the sorting process in a highly manual industry where finding workers is very challenging.
Still, lower-wage workers tend to have the most to lose. The rise of AI-powered automation, noted MIT economist Daron Acemoglu, "is going to have fairly large distributional effects, especially for low-skill service workers. It's a labor-shifting device, rather than a productivity-increasing device." The result, as the historical record shows, will see the income gap between more- and less-educated workers grow ever larger.
Another part of the problem is that sometimes, robotics actually ends up hurting workers instead of helping them do their jobs. Take, for example, the New York Times recently reported on how Smart Bar systems, automated cocktail dispensers, made work harder for waiters who found themselves. spending "more time tending to the machines and less time chatting with customers, a change that she found reduced her tips by about 30%."
This isn't just limited to blue-collar jobs. For example, many people are replacing writers with generative AI. Just like serving up cocktails, though, what the produce may look good at first glance, but a closer examination will find they're filled with errors. That means someone needs to edit the document and fix the mistakes. Now, editors can fix editorial mistakes, but they're not equipped to repair factual errors -- or hallucinations, as they're known in AI circles. That job needs to be done by subject matter experts. The result? More time can be wasted fixing bad documents rather than creating good documents.
Jobs destroyed and created
In the long run, according to the World Economic Forum, the number of jobs destroyed will be surpassed by the number of "jobs of tomorrow" created. But in the meantime, there will be plenty of labor disruption. It will impact service jobs the most, but white-collar jobs will also be hit. As Acemoglu wrote, "Automation could increase productivity while reducing wages and employment." Exactly how all this will play out remains to be seen.
Still, smart robots will be a boon for industries where industrial robots are already playing a role. As Massimiliano Moruzzi, CEO of manufacturing automation firm Xaba explained:
"Adopting and deploying industrial robots for manufacturing is tedious, time-consuming, and expensive. It requires highly skilled individuals who know specialized robotics programming languages. In addition, the accuracy of robots used today is limited by the lack of machine learning models that accurately represent the physics of a robot's operation."
That's a big reason why industrial robots, with prices starting at $25,000 per robot, are only found in specialized or bigger businesses.
Also: AI has the potential to automate 40% of the average work day
But, Moruzzi continued, "This can be changed with an AI-driven control system." Any industrial robot can be empowered with both deep intelligence, which controls the body, and cortex intelligence, which interprets sensor data such as vision. Outfitted with such AI, a robot can more fully control its body and better navigate its environment. This, Moruzzi said, "would solve the challenges of deploying industrial robots and significantly reduce the time and costs of robotics deployments."
Robotics will also transform supply chain management. Usually, when I'm writing about that subject, I'm referring to the management of software code supply chains. This time, though, I'm talking boxes, crates, and warehouses.
Smarter automation
COVID pushed automation harder than ever in warehouses, explained Nilay Parikh, CEO at Arvist, which supplies AI solutions for warehouses. "The integration of robots into warehouse operations has had a profound impact on the industry, enabling faster order fulfillment, accurate inventory tracking, and reduced labor costs."
Moreover, with AI increasingly being built-in, tomorrow's robots will be capable not only of carrying out repetitive tasks with precision but -- and this is the part Parikh emphasized -- "collaborating seamlessly with human workers, which will be key to making progress towards true automation in the future."
Some of these combinations are proving to be both useful and amusing. For example, the well-known general-purpose robotics manufacturer Boston Dynamics has paired its robotic "dogs" with OpenAI's ChatGPT to create a "smart" tour guide, Spot, with a British accent and attitude.
Also: AI will change the role of developers forever, but leaders say that's good news
This is more than a cute trick. Matt Klingensmith, Spot's chief software engineer, noted how this robot-AI combination has what is called "Emergent Behavior -- the ability to perform tasks outside of what they were directly trained on. Because of this, they can be adapted for a variety of applications, acting as a foundation for other algorithms."
Now, we're not talking about emergent behavior like an AI proving itself sentient -- but they can surprise us. For instance, Klingensmith noted that when Spot was asked a question for which it did have an answer, Spot responded with, "I don't know. Let's go to the IT help desk and ask!" It then did just that. Klingensmoth had neither prompted nor programmed Spot to ask for help. Rather, the smart guide worked out the association between the location "IT help desk" and the action of asking for help on its own.
Figuring out how to encourage this kind of useful behavior will be a major step forward in making AI smart robots much more useful.
New skills and partners
On a different level, Amol Ajgaonkar, Insight Enterprises' Product Innovation CTO, believes we should think of the partnership between AI and robotics in terms of a framework for plugins. "Companies can break down key actions they want a robot to take into generative AI plugins or skills," explained Ajgaonkar. "When deployed and utilized in this manner, it will allow us to take two fundamental steps forward. First, robots will be able to communicate with different systems easily. Second, it will allow humans to communicate with robots and systems not just to get status information but provide complex instructions that can orchestrate actions across multiple robots."
Where will all this take us? Zhenyu Gan, a Syracuse University mechanical and aerospace engineering professor, believes integrating "robotics and generative AI will revolutionize various industries by providing more intelligent, adaptable, and efficient robotic systems. We can expect over the next five years to see a new generation of robots that are capable of making complex decisions and operating with a higher degree of autonomy. This will be particularly important for applications such as autonomous vehicles, drones, and legged robots that require real-time decision-making in dynamic environments."
Also: Two divergent skills that matter in an AI world: Math and business development
Kevin Dunlap, co-founder and managing partner at the venture capitalist robotics and AI company Calibrate Ventures, has high expectations. "With so much money and expertise flowing into AI, robot development will rapidly speed up in the next five years. This will create a leapfrog effect. One of the biggest changes we will see in the next five years will be robots that can follow real-time instructions. Instead of an autonomous robot having to learn the layout of a building by studying a map, which is common today, they will be able to navigate any new environment on the fly by following instructions such as: 'enter the door, follow the hallway, take the second door on the left, and then proceed to the third loading door from the left and park there.'"
This will be a sea change. Dunlap continued, "We will be able to drop robots into any situation, and they will work without a lot of pre-programming. And, on a higher level, we will see a lot more human-robot interaction. People are getting used to communicating with AI chatbots, so it's a natural next step to interact with robots. We will ask robots to do tasks and correct them if they mess up. Robots, on the other hand, won't just receive our instructions, but will make active suggestions and recommendations based on their own more highly-developed AI brains."
High hopes and skepticism
Selmer Bringsjord, director of the AI & Reasoning Lab at Rensselaer Polytechnic Institute (RPI), is not so optimistic. While acknowledging that "the prospect of making the mind of a robot a chatbot has proved irresistible to many," Bringsjord reported back on his lab's limited success. They toyed with the idea of making GPT-4 the brain of one of their robots but found it to be of "acutely limited value." That's because robots, which are designed to perceive and physically manipulate objects, "have precious little to do with data regarding language." In short, what robots do and what generative AI does are really very different things and they don't work well together.
Also: Partner, helper, or boss? ChatGPT was asked to design a robot and this happened
Still, most people have high hopes for the combination. The proof will be coming soon enough. If all continues to go well with the marriage of AI and robotics, we can expect self-driving cars that will really work -- sorry, Elon, Tesla's not close yet; household assistants to help me stay active around my home as I get older; and, yes, robotic short-order chefs that can flip my burgers. That last scenario, by the way, is -- thanks to Roboburger -- closer than you might think!