The future of shopping: When psychology and emotion meet analytics

The future of shopping: When psychology and emotion meet analytics

Summary: The best part about the retail sector is that it combines four fun areas: Business, technology, and human behavior and psychology. Here's a tour of what may be coming to a store near you.


The future of shopping is going to look a lot more analytical in the near future and add a good bit of video to track emotions and engagement.

Welcome to the future of retail, which is quickly moving beyond somewhat silly questions about whether tablets will run on Android, iOS, or Windows, and becoming much more focused on actual applications and sales.

The best part about the retail sector is that it combines four fun areas: Business, technology, and human behavior and psychology. Here's a brief tour of technologies that range in maturity from those that are implemented today to ones that'll take awhile to be adopted.

DON'T MISS: Google Glass Corporate Policy template from Tech Pro Research

Many of the aforementioned technologies were pulled together by Intel, which has a large retail technology unit as well as an Internet of Things division. The two areas are increasingly merging. Intel realized long ago that analytics and the Internet of Things is going to drive a lot of server and processor sales.

Emotion tracking meets retail

At Intel's booth at the National Retail Federation annual retail powwow in New York City, one of the more popular demonstrations revolved around Emotient, a startup based in San Diego. In a nutshell, Emotient captures your facial expressions, gauges your emotions, and turns that data into actionable items for a retailer.

For instance, if a consumer walks up to a display and looks frustrated, retailers will know that something needs to be tweaked. Joy would prod the retailer to buy more of that good. Stores could send help and personnel to folks depending on their emotional response to an item.

Emotient captures whether a shopper is on the grumpy side.

Marian Bartlett, founder and lead scientist at Emotient, said her company's analytics software — based on artificial intelligence and pattern recognition — is being used for research as well as product testing, say an emotional response to a fragrance. Emotient launched its products in June and has pilots in fast food, automotive, and health care. In a shopping context, Emotient "aims to measure emotion and tell the store managers that someone is confused in aisle 12", said Bartlett.

My take: Emotient could have a killer app for retail, but may freak people out. The data is anonymous, but facial recognition has a bit of a Minority Report feel to it. Personally, I doubt millennials will care. Others may fret about privacy; the key for retailers and consumer product good companies will be to be transparent and dangle carrots so shoppers won't sweat revealing how they really feel about something.

Mirror, mirror

Intel partner MemoMi also demonstrated a memory mirror that allows you to try on multiple outfits, compare them, and then share with peers. The mirror is controlled by hand gestures. The biggest difference with the MemoMi mirror is that it actually seemed like it would fit into a normal shopping flow. Similar displays at previous NRF shows were a bit clunky.

My take: The magic mirror approach could work and is getting closer to being rolled out at retailers. Any retailer wanting to boost engagement via multiple channels would be interested. Magic mirrors are getting there.

A look at how virtual mirrors will alter the clothes selection process.

Kinect in retail

Microsoft displays often revolve around how Kinect could be used in retail. The software giant also highlighted a large retail Surface tabletop display that integrated personalization and the shopping experience.

My take: Microsoft has a good chance of turning Kinect into a business tech staple. The biggest challenge is that these large touch displays and virtual mirrors haven't gone mainstream. Fortunately for Microsoft, most retailers are Windows shops, so it'll have plenty of opportunities to broaden its reach.

Upscale vending machines that provide an experience

Signifi, a privately held company from Canada, was outlining "automated retail spot shops". As far as technology goes, Signifi is more of an integrator. CEO Shamira Jaffer said the company's spot shops sit "in the middle of the website and store". Jaffer's aim is to combine big screens, social media, interactive features, and ambiance to create an experience that boosts sales and serves as a marketing vehicle, too. "These don't feel like vending machines," said Jaffer. "We work with the retailer to design them."

Call it automated retailing.

signifi automated retail store 5
This isn't your average Best Buy kiosk.

Indeed, Signifi has been talking to Rolex about the possibilities for spot shops. Rolex? Sure, the company wants to brand and wants more outlets with less overhead. Signifi will provide hardware, software, and support. The retailer handles inventory and stocking the machines. Another perk: The spot shops can take returns, too. "We provide a retail store in a box," she said.

My take: These spot shops could proliferate in the US, a country that so far only has three. There are more spot shops in Europe. BMW is one brand testing the concept, and Jaffer said that locations for spot shops go beyond airports and transportation hubs. Hospitals, which have a bevy of folks waiting around, and universities are also good locations for high-end brands and spot shops.

Video analytics meets store staffing

Scopix, a company demonstrating at the Intel booth, scans a store via video, rates whether a customer is engaged, and then provides real-time data so a retailer can close a sale. The Scopix technology is also used for queue management and predictive analytics to find patterns. One use case would be that the Scopix system could ping a mobile device on a store floor so employees could keep customers happy.

You're being watched to see if you're interested.

My take: Scopix has interesting technology, but it's unclear how it stands out. Video surveillance and analytics tools were everywhere at the NRF conference.

We'll give you discounts to watch our TV commercial demonstrated technology that used audio cues to combine mobile applications from TV networks with commercials and direct response. Here's how it would work — and has worked in a few pilots. A TV watcher is sitting on the couch with a tablet and an app from Fox (or CBS, parent of ZDNet). The app and tablet listen to the TV ad and call up products such as an outfit worn by an actress. For watching the ad, consumers would get flipped a coupon, say a 20 percent discount for a store visit and 10 percent deal for an online sale. Intel provides the ad-serving technology.

Actv8 also announced a deal with NCR to bring its personalized proximity platform to kiosks and multiple industries.

My take: Actv8 could find an audience from media giants given that the technology also works for on-demand commercials. Actv8's technology could track engagement with TV ads, maintain rates, and even add revenue shares for sales.

Topics: Big Data, Emerging Tech, Enterprise Software, Hardware

Kick off your day with ZDNet's daily email newsletter. It's the freshest tech news and opinion, served hot. Get it.


Log in or register to join the discussion
  • Detecting emotion in what people say, in addition to how they look

    The article emphasized the gestural side of emotion detection, so the non-verbal side. The verbal side (what people say and how they say it) is another piece of the puzzle. In Computational Linguistics, there's a sub-field called Sentiment Analysis, so this area was recognized to be important.

    Our company, North Side Inc. ( works on the semantic side of the puzzle, so making sense of what a person says. We recently started a public Alpha of the world's first video game that uses Natural Language Understanding technology (it's called Bot Colony, at, to enable a person to speak with the characters and explore the story in a non-linear way.
    We have plans for our robotic characters to exhibit quasi-emotions (react to a verbal stimulus in the same way a person would react, taking into consideration the personality of the characters. Just getting basic understanding of day-to-day English took more than 10 years of work, but we're now past the hump - so hope to ship the emotion/personality functionality is later this year.
    Eugene Joseph
  • Looks Like We All Will Be Wearing Burkas Soon

    ... with all these technology advancements. And latex gloves, probably, too. And some orthopedic inserts to fool the gait recognition systems.
  • We already have the technology...!

    As a retail consumer psychologist the technology debate fascinates me. We already have "emotion detectors", "semantic analysts" and a host of other psychological analysis systems in place in retail. They are called "people". Even though I applaud the technological advances and can see their application, the human brain inside every sales assistant already does the tasks these technologies are seeking to replace. And those brains do it quicker and more accurately. The biggest problem with retail is it is seeking how to replace people with lesser technology. These technological advances are, from a psychological perspective, a retrograde step.

    Whilst I can see their application when sales assistants are not available, perhaps the real technological advance required is to place sales assistants anywhere, perhaps using holographic techniques. That way retailers would be investing in two technologies at the same time - one of which is exceedingly more advanced than any technology currently being developed.
    Graham Jones - Internet Psychologist