Ford moves toward facial recognition and gesture UI in the car

Ford moves toward facial recognition and gesture UI in the car

Summary: In its on-going work to innovate UI in motor vehicles, Ford has jointly announced a research project with Intel that could bring facial recognition and gestures to the next generation of automobiles.

TOPICS: Innovation
Image: Ford

Ford and Intel unveiled a joint research project on Wednesday called Mobile Interior Imaging, or "Project Mobii," to utilize interior cameras, facial recognition software, and data analytics to create a more personalized interaction between driver and vehicle.

"Mobii is a great example of culture of experimentation. I don't know what the user interface of the future is going to be. I know that increasingly, I've got sensors, cameras, technology,"  said Don Butler, who has been the executive director of connected vehicles and services for Ford since January. "So [we said] let's try to mash some stuff together and see what happens," Butler told ZDNet/TechRepublic.

Project Mobii is still in its experimental phase, but researchers are looking to use it to make the car's user interface experience more seamless and intuitive.

One use for the system: driver authentication. Once a driver enters the vehicle, a front-facing camera snaps a photo. If the driver is recognized, their personalized data -- such as contacts and music -- syncs to the vehicle. If they are not recognized, a photo is sent to the primary owner of the vehicle, and the owner can set restrictions or permissions from a smartphone. Also, if a child gets into a parent's vehicle, it could allow the parent to adjust music volume, speed limits, and phone access from afar.

The system could also be used to detect natural gestures and simple voice commands to adjust the systems and controls in the car. For example, speaking a voice command to change the temperature or waving a hand to open the sunroof instead of pressing a button.

Intel anthropologists and engineers collaborated with Ford to develop facial recognition software and perceptual computing technology for the system.

"This was born out of a visit to Intel and exposure to things they were working on in terms of pattern recognition, image and gesture recognition," Butler said, emphasizing the importance of partnerships in this field. "You literally have to create the future together."

Ford already uses exterior vehicle cameras to alert drivers if they are drifting into another lane or getting too close to another vehicle or object. The company also leverages smart devices and communication capability of them to connect the vehicle to the outside world.

Butler said customer trust remains the most important factor throughout these innovations -- all of the data Ford collects, he said, is still owned by the customer, though Ford will leverage the data to improve customer experience. However, Butler emphasized that Ford wants to be good stewards of customer data.

"The primary way we can enhance that experience is by leveraging the data," Butler said. "The ability to do that is reliant on the trust of the customer."

In a separate announcement the same day, Ford also revealed plans for SYNC 911 Assist, which will be featured on the 2015 Mustang. The system will offer more direct route to emergency assistance by connecting a mobile phone and improve the overall safety and connectivity of the vehicle. Upon entering the vehicle, the driver must give consent that their location and data can be sent to those services.

It's interesting to note that Butler previously helped pioneer OnStar, a similar service for General Motors.

Both projects are part of Ford's larger platform of innovation to better connect people with their vehicle whether they are inside it or away from it.

"We are playing around, experimenting with what might be possible," Butler said "[We'll] expose customers to it, get some feedback, and let it guide our thinking going forward."

Also see

Topic: Innovation

Lyndsey Gilpin

About Lyndsey Gilpin

Lyndsey Gilpin is a Staff Writer for TechRepublic. She writes about the people behind some of tech's most creative innovations and in-depth features on innovation and sustainability.

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Ford is off the deep end with car electronics.

    Adding gimmicky electronics (face recognition, gestures???) to cars not only increases the price and complexity (reliability), but also adds a serious distraction factor for the driver. At present, Ford's vehicular UI is downright terrible. Cars don't need (and shouldn't have) 10-key keypads on the dash, Ford. They really need to concentrate on improving the UI of their present systems before they add any new stuff to the mix.
    • Hard buttons are better

      Actually a 10-key keypad is much better than what a lot of companies are doing. It's all the inconsistent on-screen stuff that's a real distraction. I want a button for each primary function that's always in the same place and always does the same thing. Imagine if the gas and brake pedal changed position depending on system mode.
      Buster Friendly
      • Did you just leak the plans

        for the new Microsoft car? ;-(}
        • Android car

          I think that would be the Android car. That's the worst for inconsistent UI.
          Buster Friendly
  • Basically an XBox One

    Facial recognition and automatically configuring to your setup is right off the Xbox One. Personally I don't want my car gadgeted up like my entertainment system. I much prefer hard buttons with consistent function so I can pay attention to driving and not what screen my system is on. The only one I really like is voice select out of your MP3 library as that keeps you from having to flip through screens of songs to find what you want.
    Buster Friendly
  • Simple voice commands ...

    ... would be more intuitive, and less intrusive - you set a name for your car, then talk to it ...

    "Jim, reset the satnav for the zoo"
    "Jim, turn the temperature down two degrees."
    "Jim, if you play The Eagles one more time, them park up, give me 10 seconds to get out, then explode"

    The article does not mention which OS; or whether anything syncs to phones. Even if Ford haven't a clue what to do between a car and a phone (a likely scenario) they STILL should sync., for future proofing. Oh, and they could use phone data for security?

    I can see Intel's desire to get inside ... but Ford should be talking to software people with a clue, not Intel.
    • ooopps ..

      I missed the bit about SYNC 911 Assist; apologies.

      But that looks like a partial / 3rd party set up; not ideal. They should work with a mobile OS, preferably a 21st century one.
    • Better to use a button

      If you're in a position to use a button, such as the steering wheel control, it's much better than using a preamble like a particular word. That kind of signaling has a much higher failure rate than just pushing a button to tell the system you're going to input a command.
      Buster Friendly
    • Yes!

      I don't need to be waving my hands around in order to talk to my car. I'd much rather TALK. And Sync's voice recognition in my 2013 Edge sucks. E.g.
      "Call Joan on Cell."
      "Say 1 after tone to call Joan on cell."
      "Say 2 after tone to call Joan at home."
      "Say 3 after tone to call John on cell."
      "Say 4 after tone to call John at home.... (beep)"
      " b**ch"
      Why not *listen* for my choice while rattling off the options? Better yet, improve the voice reco.
      • It's "at"

        You say "Call at ". There's tutorial videos on youtube.
        Buster Friendly
        • Well, that didn't work

          That was supposed to be "Call (person) at (number)". It doesn't like greater than/less than in posts
          Buster Friendly
        • "Call Joan AT Cell"

          Well, OK... but maybe they should concentrate on making the voice interface a little more intelligent before tackling the all-important problem of ordering pizza while driving. ( I shouldn't need a tutorial on how to speak.
          • Not how it works

            You do need to learn how to use all tools properly. Don't complain a screw doesn't stay in if you beat it in with a hammer.
            Buster Friendly

    Where do we get a BASIC CAR?
    One with no frills, no thrills. Just a heater and maybe A/C?
    All these gadgets is getting really reDICKulous!
    Basic car, basic transportation basic high fuel mileage.
    We can install our IPods for music. Air wave radio has become so clogged with adverts it's sickening! .
    • Still can

      Just don't get the electronics package. Options have a good margin, so I don't think they'll make these gadgets standard equipment. Sirius is the way to go for radio. I use my own recordings too but who wants to only hear what they've already heard?
      Buster Friendly
  • User convenience

    It would be nice to have the car recognize my face, adjust the seat, to my last setting, set the mirrors, turn off/on the radio to my favorite station, or none, and set the A/C controls.
    • I like the key better

      I like using the key or fob for the better. That's simple and very reliable.
      Buster Friendly
  • NSA is happy

    Just another way for them to follow us.
    • Yea

      Yeam because those bold metal plates with an ID number on the front of the back makes you completely anonymous.
      Buster Friendly