For chefs and foodies, big data could be the new secret ingredient
SAN FRANCISCO---Chocolate, vanilla, pine nuts and broccoli.
Collectively, those food stuffs might not sound like the most obvious combination of ingredients to satiate one's palette.
Yet it turns out they do work together in a satisfying enough manner, at least for some taste buds. Who would have thought?
Food and Technology
Chef Watson, that's who, or rather, what -- if you want to get technical about it.
The Chef hat is just one application gaining more mainstream attraction for IBM Watson, a ground-breaking cognitive system so far more synonymous with ambitions for revolutionizing healthcare and education rather than anything as basic as kitchen recipes.
Watson stepped into the national spotlight in 2011 with its winning performance on the decades-old TV game show Jeopardy as an artificially intelligent computer able to recognize and respond to questions posed in natural language rather than tech speak or code.
Watson can do this thanks to the advancement of machine learning, a newer subset of computer science rooted in massive sets of big data.
Big data -- for lack of a better term than the hackneyed buzz phrase -- has been championed over the last few years as the defining edge to get ahead in an increasingly digitized global marketplace.
But the deployment of machine learning takes big data analytics many steps further, theoretically giving computer systems the ability to learn on their own through algorithms based on preexisting data in order to spit out more enlightened assessments and predictions. (Think facial recognition for tagging friends in photos on Facebook or Spotify's suggested songs and playlists based on listener preferences and behavior.)
Just a few years after the birth of IBM Watson, machine learning and cognitive systems are finally making mainstream waves thanks to quantifiable insights that many entrepreneurs and business titans are banking on to sprout new fountains of revenue.
Not surprisingly, machine learning picked up steam first in the industry that gave rise to it - technology -- from cloud services providers like Amazon Web Services and Google as well as IBM to consumer-friendly wunderkinds such as Pinterest and Airbnb.
Now tech giants like IBM followed by a handful of budding startups are experimenting with cognitive technology all along the food chain from growers to chefs to home cooks.
THE FOODIE FRONTIER
With 25 employees, the two-year old engineering startup Fyusion sits comfortably in a sun-drenched open floor plan office space on the fourth floor of a rather generic office building in San Francisco's rapidly gentrifying Dogpatch district.
Only a short bus ride away from downtown or a bit longer from Silicon Valley on the nearby commuter rail, it's an increasingly familiar setting these days in what some critics might peg as peak-tech (or pre-bubble popping) culture in the City by the Bay.
But as Fyusion CEO and co-founder Radu Rusu ran through his company's demo reel on a bright June afternoon in a small conference room, the technology being designed here isn't as cookie cutter as some other developments at local incubator showcases.
Rusu outlined some of the challenges presented by machine learning -- many of which are being solved through another hot flavor in tech these days: robotics.
"We started thinking something was missing, and it actually came from science fiction as robotics and science fiction mix very well," Rusu quipped, playing through an excerpt depicting young Captain Kirk navigating a location through both time and space during a scene in the reboot sequel, Star Trek: Into Darkness.
Adapting that to the real world, the idea is to deploy a robotic device and direct the embedded camera lens through a gyroscope for building 3D-formatted photos, stitching together a visual graph end users can navigate through it on their own.
As you move around in different angles, Rusu explained, the user can visualize the geometry behind it, which can now be 3D printed, if desired.
"The world is changing from the desktop to mobile," Rusu posited. "Everything on mobile tends to go in the space of interactivity, moving away from passive. Even the feeds are changing."
The fusion, so to speak, of robotics and machine learning pushes engineers and consumers alike away from the "way we have been currently doing things," Rusu suggested, meaning snapshots and videos. More modern ways of capturing visual data can unearth more accurate results faster than ever before, he continued.
"As a technologist, it makes me think about data differently," Rusu reflected. "There's no frame anymore. We use to think about frames per second. It's all a continuous space. In terms of the consumer, it allows me to be very interactive."
Rusu theorized machine learning has been progressing forward slowly because we don't have all of the right data sources just yet. A big challenge, he said, is taking this technology, incubating it and pitching it as a comprehensible product to consumers.
One primary (if not the primary) source of data that can be spun for machine learning has become the smartphone. When it comes to food, it's not hard to guess which activity people often conduct with these devices while dining: sharing photos of their food from restaurants.
"The food industry is so much about visual appearance, especially higher-end restaurants. But any restaurant wants to showcase its food in a very nice way," Rusu observed.
At a very basic level, Rusu proposed Fyusion's app, Fyuse, can be used to assign labels to scenes or dishes (i.e. steak and potatoes) at an eatery, which can then operate as "physical Post-Its."
"The nice thing is that once you click on those, you can be taken to the website for purchasing," Rusu suggested, adding this inspired his team about the "massive implications" for e-commerce, fashion and retailers.
Fyusion has begun to work its way into haute cuisine, starting with a booth at the inaugural Bite Silicon Valley conference, a three-day food and tech festival at Levi's Stadium in Santa Clara last month, after making some food industry connections, which Rusu revealed eventually went all the way up to renowned chef and restauranteur Michael Mina.
Thus, Fyusion showcased Mina's personal profile on Fyusion and clips (such as the one below) for his restaurant Bourbon Steak at the football stadium.
Fyusion has some more partnerships with restaurants and chains in the pipeline -- although none of them could be named at the time this feature was published.
"The opportunity right now is to figure out how menus are going to look in the future," Rusu articulated, clarifying this could apply both while in brick-and-mortar locations as well as through other platforms like Yelp or Foursquare.
For the time being, Rusu affirmed Fyusion is keeping focused on enhancing organizations through these methods right now.
For the long term, Rusu predicted Fyusion will be able to leverage algorithms for making better sense of the world through these automatic and intelligent actions with the end goal of breaking through on visual search.
THE SEED STAGE
Although the seed stage might have another connotation (or a few) in the tech-flushed San Francisco Bay Area, NoukaTech takes the value of seedlings and harvesting much more literally.
Founded in 2013 in partnership with a research and development team at the University of Pennsylvania, the early stage startup has pulled together a mishmash of robots, sensors and machine learning for crunching analytics to boost growers' revenue and net sales.
Starting small across just four states (albeit major agricultural hubs being California, Washington, Oregon and Pennsylvania), NoukaTech is initially targeting specialty crops (i.e. apples and oranges or grapes for wineries) rather than staple crops, like corn.
NoukaTech has already applied machine learning to the problem of fruit detection in dense orchard canopies.
That problem, quite simply, is that years of previous data accounting for fruit allotments and resources aren't accurate anymore in the wake of climate change, according to NoukaTech CEO and founder Regina Gindin, who chatted with me before giving a quick fire presentation at FoodBytes 2.0, an afternoon showcase of elevator pitch-style demos vaguely reminiscent of Y Combinator Demo Day or TechCrunch Disrupt.
"People are pushing technology, not solutions," Gindin said frankly while recalling past conversations with individual growers and associations.
Still NoukaTech, which is heavily self-funded between Penn's R&D team and Gindin according to the company president herself, is completely centered around some very advanced homegrown technology.
NoukaTech engineered supervised-learning algorithms to segment fruit pixels in images captured by low-flying aerial robots made by a variety of manufactures, including drones developed at Penn.
These algorithms allow professional researchers to label sample fruit and vines in training image sets, resulting in a set of positive (fruit) and negative (non-fruit details such as leaves) examples. The algorithm takes those inputs and learns sets of criteria so it can identify fruit in future, unlabeled data.
Think of it as the farming equivalent of face tagging.
One of the initial benefits to this model is reducing double counting of fruit in orchards. Although the results aren't presented in real-time just yet, analysis can be served up as soon as within 24 hours of a drone flight through the orchards, which is instantly pushing its findings to the cloud.
With more accurate and up-to-date counts, fruit and vegetable growers could rewrite their own short-term and long-term business plans.
NoukaTech is already drafting revisions for future models, which could incorporate data from other sensors, such as LiDAR (3D canopy measurements) or multi-spectral images (estimates of plant vigor).
NoukaTech is also growing its customer base, picking up some well-known establishments such as citrus growers Booth Ranch as well as E&J Gallo Winery, both in drought-stricken California.
WATSON STEPS UP TO THE PLATE
Despite a recent mini-media blitz for the Chef Watson app, the casual tech user still might only recognize IBM Watson as the computer that won Jeopardy! a few years back.
"There was a group of people who looked at it as a tremendous accomplishment," recalled Steve Abrams, director of Watson Life within the IBM Watson Group, in an interview with ZDNet. "But it was just answering questions. You have to understand the text and find answers quickly. Often the answer is already there. But what about things that aren't yet known?"
Once IBM decided to explore that path, Abrams explained, the idea evolved around honing in on areas of popular human interest.
There are fewer things in this world that interest people more than food.
Food presented itself as a solid foundation for building and showcasing a system to the public about discovering new insights through advanced computing.
Thus, the culinary arts were chosen both because of that role food plays in society and it is an area clearly everyone understands, Abrams noted.
"Food and health really do go hand in hand," Abrams said. "The trick is going to be to learn about the individual. The more it can learn about you as an individual, the more it can learn and customize things for you."
Abrams elaborated that if Chef Watson can simply learn a user's likes and dislikes, then like any health or fitness app keeping track of exercise and medical records, Watson can offer a much more personalized experience.
Chef Watson, which was just pushed from beta to public access in June, sources its ideas from more than 10,000 original recipes provided by the beloved food glossy Bon Appétit.
"The partnership with Bon Appétit was to take this technology to the public in a much more tangible manner, bringing it down to earth with a consumer-facing culinary brand," Abrams said.
With that gourmand treasure trove, Chef Watson has essentially put itself through culinary school, learning what ingredients are commonly used together (i.e. peanut butter and chocolate), what goes into different types of dishes (i.e. the basic components of sandwiches, soups and salads), and food stuffs and spices inherit to specific cuisines worldwide (i.e. olive oil and Pecorino Romano cheese in Italy).
Chef Watson is meant to bring his own flair to the kitchen and build on those culinary basics, adding to them through learned knowledge about the underlying chemistry of food, the chemical compounds associated with popular pairings and taste.
Watson eventually uses that information to make predictions about combinations that could work well even if they've never been done before.
Abrams cited Chef Watson can deliver upwards of "quintillions of possible combinations," which are scored on a variety of criteria, before presenting the top search results Watson thinks might be most well received.
"Ultimately, what was interesting to us about what Watson can do is it can see patterns and also point out things we were too set in our thinking not to think about," said Stacey Rivera, digital director for Bon Appétit, emphasizing the magazine staff never viewed Watson as a source for just making recipes but rather a game changer expanding culinary thinking.
In one instance, Rivera described a punch recipe in which Watson suggested adding walnuts. Rivera rebuffed the idea, pointing out people could choke.
But then a few weeks later when some Bon Appétit staffers were out together at a restaurant, one editor saw a cocktail on the menu with walnut syrup, prompting the editor to exclaim, "Watson was right again!"
As Rivera explained, Watson right that the flavor of the nut would go well, so maybe just not the literal interpretation of nuts. It is this kind of exploration, Rivera suggested, that could push Bon Appétit's readership -- most of whom she described as experimental to begin with -- to think differently about food.
"Most people have a sense of how to cook, not what to cook," Rivera observed. "There are plenty of people who need a recipe, and there are also plenty of people in need of inspiration."
Chef Watson is mean to spark creativity, Abrams agreed, offering groupings of ingredients that on the page might not seem like they'll go down well together but actually work in practice. Abrams recommended a grouping of chicken, mushrooms and garlic -- all seemingly commonplace -- until you top them off with strawberries.
"In all of this, it's really showcasing how Watson, a cognitive system, can help you discover something that has never been seen before," Abrams opined. "As a professional, you're faced with a tremendous information, an information overload, you can't remember it all. Watson is supposed to find the threads and present them as starting points."
Abrams outlined two different audiences and parallel tracks for Chef Watson.
The first would be professional chefs and up-and-comers, such as students at the Institute of Culinary Education, which also contributed to the development of Chef Watson.
Abrams indicated these cooks might not need as many bells and whistles and features and functions when it comes to a web-based app, but rather they would be interested to intuit why Watson is making a suggestion for combining a certain set of ingredients.
"The response from chefs has been tremendous. They love the way it pushes their creativity and makes them rethink their cuisine," Abrams boasted, adding feedback from the food industry motivated IBM to publish a proper cookbook, Cognitive Cooking with Chef Watson.
The second track consists of the home cooks and food bloggers -- including the foodies sharing their creations via Instagram and Tumblr, a practice in itself forming the beating heart of the digital community IBM wants to establish and grow over time.
Looking even farther down the line, Abrams said the Chef Watson team is interested in forging more commercial partnerships with cooking school chefs, restaurant chains and other food industry professionals from consumer product manufacturers to grocery stores.
Nevertheless, one cannot ignore the bigger picture here that Chef Watson is just one app and component of a much larger platform and scheme here. Abrams listed possibilities from unrealized connections and pharmaceutical discoveries in literature about proteins fighting against cancer or prompting new argument starting points in legal briefs based on relevant court cases.
Abrams concluded, "The main thing here is to get people to think, 'If Watson can do this for me in the kitchen, imagine what it can do in my industry.'"
Still, not everyone is convinced that machine learning -- or any buzzy tech trend garnering attraction in Silicon Valley and similar startup communities around the country -- will have any more lasting impact in the food world than a flash-in-the-pan gimmick.
"It may be an interesting demo but probably won't change our eating culture much," said Ted Schadler, vice president and a principal analyst at Forrester Research, about the smarter search engine for inspiring recipes.
Yet Rivera stressed repeatedly that Watson is not just a recipe app, but rather a mechanism for turning content into data, which in turn can open the door to new flavor combinations, products and even restaurants.
Over the last year while Chef Watson was in beta, Rivera revealed what proved to Bon Appétit that Watson had broader appeal was that people were using it to manage dietary constraints and solve daily food problems, including overcoming food waste.
In cutting back on food waste at home, Rivera advocated Chef Watson can make suggestions with even just one ingredient while accommodating dietary constraints.
"People think they have an understanding of what being gluten-free means, but not everyone has the knowledge Watson comes with," Rivera posited. "If you have a good sense of food science, which Watson has, it's not about cutting food out but making choices."
Schadler acknowledged and elaborated on other sectors of the food chain where cognitive technology could make a difference -- starting back at the seed level.
For example, the industry analyst suggested using crop yield data on micro plots from different years with different weather and fertilizer and seed combinations. Such data could help an individual farmer, a fertilizer company, or even a multinational agrochemical conglomerate like Monsanto to all better optimize yields.
From there, Schadler continued, those cognitive patterns could be extended throughout the food supply chain from demand to inventory to distribution to even spoilage.
"This example reveals the opportunity and the complexity of getting it done," Schadler conceded. "But it's inevitable that things like this will happen."