Can iPhone Meal Snap photo app magically tell what's in your meal? We put it to the test.

Can iPhone Meal Snap photo app magically tell what's in your meal? We put it to the test.

Summary: Meal Snap is accurate enough to amaze your friends, and give you a rough idea what you're about to eat.

SHARE:

I decided to use today's blog as an excuse to try two things I've been wanting to try.

The first excuse was to buy DailyBurn's Meal Snap from Apple's App Store for my iPhone 4. The app has been getting a lot of press lately. It lets you take a picture of the meal you're about to consume, tells you what's in it, and estimates the number of calories it contains.

The second excuse was to run out to McDonald's and grab a bowl of their relatively new Fruit & Maple Oatmeal. I've been wondering if fast food can really be healthy, and how it stacks up against normal oatmeal. After all, since we're all always on the go, it'd be so nice to find a healthy breakfast or snack option available from a fast food joint.

I worked up a healthy appetite running out to the car, driving through the drive through, and jogging back through the door with my bag of McDonald's Oatmeal.

The experiment

After downloading the Meal Snap app right to my iPhone 4, I snapped a picture of the McDonald's Oatmeal right in its container. I figured that, as a first try, I'd make it easy for the program, in case there was any kind of label recognition going on.

After snapping the picture, the app offered me the opportunity to add a caption, and to choose the meal it was for. I did so, calling it "McDonald Oatmeal," and naming it "Lunch."

The app displayed a witty message explaining that it might take a minute to get back to me with my data, so I went on to prepare the next phase of the experiment.

I simply moved the McDonald's Oatmeal to a plain white bowl. By this time, the app had already gotten back to me with the data on the first meal. But I'll reveal the results in a minute. I snapped a picture of that same oatmeal in its new habitat, called it "Same Oatmeal in Bowl," and notated that it was an "Afternoon Snack".

Then, I went over to my cabinet, took out my trusty old favorite oatmeal, the Quaker Multi Grain Hot Cereal, measured out 1/2 cup, threw in 3/4 cup of water and 1/4 cup of frozen blueberries, and microwaved it for a minute and a half. The data from the second meal was in, but I'll save the big reveal for the end.

I snapped a picture of my beloved hearty, warm breakfast in a happy, earthy orange bowl, and waited for the app to get back to me. I apparently neglected to give this bowl a title. I chose "Dinner" as the meal, so that it would show up under the other two bowls on my iPhone screen.

Next: The results...drumroll please... »

« Previous: The experiment

The results...drumroll please...

The first McDonald's oatmeal in the McDonald's cup -- somehow recorded as "Mix Mcdonalds oatmeal" because I'm still pretty bad at typing with my thumbs -- was estimated to be 195-293 calories. That's not a bad guess! The nutritional information on the other side of the cup actually says that it contains 290 calories.

Interestingly enough, the same McDonald's oatmeal in a white ceramic bowl was estimated to be 250-375 calories. The app was still right. 290 is definitely in that range. It's funny that moving it to a bowl added up to 85 calories. Did the app recognize the McDonald's container? Was the size of the ceramic bowl hard to figure out, thereby making the quantity harder to guess?

Weirdest, though, was the third bowl of multi grain oatmeal I'd prepared for myself. Since I'd forgotten to title it, and had saved it under the dinner category, Meal Snap decided it was a bowl of fried rice with meat. The app, therefore, estimated that the bowl contained between 351-527 calories.

The truth is, the cereal prepared with water is 130 calories, and the added blueberries (at 15 calories) bring it to 145 calories. I normally just throw in a packet of Sweet'N Low and some cinnamon, which doesn't add any calories to the mix.

I decided to try again, and give the app another chance. I snapped the homemade cereal again, called it "Whole Grain Oatmeal" and saved it as "Breakfast." This time, apparently taking its cues from the fact that I set it as breakfast and/or had the word oatmeal in the title, it came back with a much more accurate caloric estimate of 105-158. That is pretty neat.

That was a really interesting experiment

I'm rather impressed with the app. Not as impressed as some of its reviewers, who seem to believe that magical elves are beaming information back on rainbows from the cloud (or possibly that the elves are so tiny they actually live inside the iPhone). I do have to hope there are no children in China chained to computers performing complex caloric computations. I mean, I sincerely doubt that's the case, but I do wonder how the "magical meal logging" (their words, not mine) is happening.

I haven't been able to dig up anything about how the app does its cool trick. It's hard to believe something so complex could be done without live human intelligence on the other end for each item. Kudos to the programmers at DailyBurn for pulling this one off. It's really neat.

According to my husband, who is a computer scientist, it's relatively easy for AI (artificial intelligence) to tell the difference between a bowl of cereal and a cat. But where it's needed to tell the difference between one bowl of oatmeal with blueberries and another bowl of rice with meat, AI leaves a little bit to be desired.

With a good database and little help from naming cues, Meal Snap is accurate enough to amaze your friends, and give you a rough idea what you're about to eat. I have to say that just for entertainment value alone, it's worth the $2.99.

Next: I get all nursey on you »

« Previous: The results...drumroll please...

Now, I get all nursey on you

For people trying to lose weight, and make the necessary lifestyle changes to do so, a food journal is a commonly recommended, useful tool. It helps bring awareness to the act of eating, and provides actual data for later review. It helps you figure out what food choices are helping you meet your goals, and where your bad habits are. But a lot of people say that keeping this kind of journal can be a drag. Maybe Meal Snap can help with that.

I see the Meal Snap app as a good way to keep an on-the-fly pictorial record, along with some quick notes, about what you're eating during the day. Then, if you want, you can come home later and do a better job analyzing the information and/or adding it to a more traditional food journal.

Of course, it's also important to keep in mind that calories are not the only thing to pay attention to when making food choices. I'm not a nutritionist, but I did have to pursue a course of study in nutrition during nursing school. Also, I am an American woman, which means I've read a lot of diet books.

I've made the mistake of demonizing various nutrients on and off throughout my food-consuming career. I've since learned to look at the total picture and make more balanced, healthy lifestyle choices. I don't always make the perfect choices.

I must admit that I'm carrying a couple of extra pounds. Actually, most Americans are. That scares me, because I've seen some unfortunate things in my clinical experience. I know that, even putting unfair standards of beauty and unhealthy obsession with weight aside, too much extra weight really does hurt our health.

Because of this, I think it's important to point out that what the Meal Snap app can't do for you is provide information about fat, carbs, protein, fiber, vitamins, and more.

For that, it's necessary to look at labels and use your powers of better judgment, taking into account what is nutritionally important to your total health picture. Of course, if you're feeling lost about how to make the food choices that are right for you, your doctor, perhaps with the aid of a qualified nutritionist, is the best resource.

For example, the oatmeal with blueberries made in my microwave has 145 calories, 1 fat gram, 33.5 grams of carbs, and 6.5 grams of fiber. I always enjoy this breakfast. It sticks to my ribs, warms me up, and puts me in a good mood. Sometimes I even stir a little chocolate protein powder into it for an added protein boost, but I didn't do that this time.

According to the nutritional information on McDonald's oatmeal, it has 4.5 grams of fat, which seems like more fat than oatmeal should have (probably because of the light cream where skim milk might have done just as well). It has 5 grams of fiber (which is pretty good), and 57 grams of carbs (again, that seems kind of high). And it is yummy. I'll give it that.

I agree with Jennifer LaRue Huget's assessment that it's good to see McDonald's offering something that's somewhat "nutritionally sound and palatable". There are certainly scarier breakfast choices available.

I'm not about endless drudgery in the kitchen. I know that's not practical for many of us. But light foods, cooked at home from relatively fresh, wholesome ingredients our grandmas would have recognized are usually better for us than the hyper-salient fare, assembled in chain-restaurant kitchens, and engineered to pack the fat, salt, and sugar punch that'll have us racing back for more.

If you've downloaded the Meal Snap app, or you simply love oatmeal, share your experiences in the TalkBacks below. Have a great weekend!

Topics: iPhone, Hardware, Mobility, Smartphones

About

Denise Amrich is a Registered Nurse, the health care advisor for the U.S. Strategic Perspective Institute, and a mentor for the Virtual Campus at Florida's Brevard Community College.


Nothing in this article is meant to be a substitute for medical advice, and shouldn't be considered as such. If you are in need of medical help, please see your doctor.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.

Talkback

14 comments
Log in or register to join the discussion
  • Message has been deleted.

    loverboys
  • RE: Can iPhone Meal Snap photo app magically tell what's in your meal? We put it to the test.

    I've been using it for a few weeks based on extremely positive reviews on the web. My reality is that the app isn't always very accurate, sometimes wildly inaccurate! But the more descriptive my wording the better the caloric count becomes. In other words, I'm doing most of the work here not the app analysis. I hope version 1.1 provides a bit more info on the actual meal, such as a breakdown of the foods involved and perhaps even recommendations for alternatives (ala "eat this, not that" format).
    penguy
  • Good writing; thanks for the article

    The subject.
    DDERSSS
  • Obesity 101

    Nice toy, but a waste of time unless it measures fructose. Weight is not about caloric intake vs calories burned.

    Search YouTube for "Sugar: The Bitter Truth"
    kenift
  • Message has been deleted.

    lehmermeghan
  • This may set dieters up for failure.

    Those that are serious about losing weight should have a pretty good idea what they are eating to begin with. However, I expect some will come to rely on this app to do the homework for them, and with the displayed level of inaccuracy, are going to be very frustrated that they are not losing weight. As someone that is successfully losing weight on a strict reduced calorie intake (down 64lbs since November), I know that 50 calories here and 100 calories there over the course of a day unintentionally consumed due to inaccurate reporting would be very detrimental to my goals.
    JackLR
  • 1 app too many

    This is really getting ridiculous, Another totally useless app
    gbouchard99@...
    • RE: Can iPhone Meal Snap photo app magically tell what's in your meal? We put it to the test.

      @gbouchard99@... not really, the use of high quality cameras like on the iPhone 4 and the fact "Apps" are still a young industry, it's best to support these developers so they continue to improve.

      given a year or two of millions of people using Apps like this, will build huge databases, and with image recognition becoming so accurate, Apps like "Meal Snap" will be perfected.
      Pederson
      • Garbage...

        @Pederson At the very best it guesses the food type and guesses how it was prepared, guesses ingredients that may not be obvious and most importantly guesses the size of the serving.
        So sorry, but too many guesses will lead to erroneous determinations. Better off that YOU do some of the work and identify healthy foods vs. unhealthy foods.
        Bradish@...
    • RE: Can iPhone Meal Snap photo app magically tell what's in your meal? We put it to the test.

      @gbouchard99@...

      Well, it may not be all that good in it's current incarnation, but it is interesting that they are trying.

      Sooner or later something like this will work just fine, and this is just the first crack.
      SlithyTove
  • Message has been deleted.

    gamblndano
  • Thoughts

    Thoughts:

    -It appears to combine image recognition, text matching, and which foods are most common for which meals. So for the most accurate recognition of your food, you probably want to use all three every time.

    -Image recognition is frankly pretty poor for most AI. It can do some things well, others very poorly.

    -AI experts do tend to overstate the capabilities and accuracy of current AI. Probably to create the sense of progress in a field that frankly isn't doing much right now. IMO current AI is actually quite poor except in specialized cases.

    -Another thing to keep in mind is even when good AI is available, there's no saying that the author of an app will use it. I'd say most iPhone apps with some form of "AI" have something very basic.

    -Even perfect image recognition has its limits. It can't see underneath the surface of foods and it can't recognize things not in its database. It will most likely fail to recognize hand made foods.

    I'd say most of the "accuracy" of this app likely comes from the image descriptions and meal time, rather than the image recognition.
    CobraA1
  • RE: Can iPhone Meal Snap photo app magically tell what's in your meal? We put it to the test.

    The magic "photo recognition" software here is actually just a person sitting behind a computer looking at your pictures. The program uses an Amazon service that farms the pictures out and waits for a human being - who gets paid about 3 cents per picture - to tell them what's in it. Look up Amazon's Mechanical Turk platform.
    shonmm03
  • bzocibw 17 mbt

    plhxhz,zpyxwpar45, hgttg.
    bdfwekrwe5301-24379013214580303250156439603070