"People do things in doorways you wouldn't expect."
That's how Andrew Farah, a co-founder and chief executive of startup Density of San Francisco describes the peculiar problem he is trying to solve. His company, formed six years ago, has received $74 million in venture capital funding from Kleiner Perkins and other prominent firms, including a Series C round of $51 million just last week, to surveil the workplace.
Density designs and builds small metal boxes packed with sensors and chips and software that sit over a doorway. The infrared sensors pass data to machine learning algorithms that tell how many people are coming into or out of a room. Based on that information, a company knows what a room's occupancy is and can plan for how many people to allow back into a facility. A company can also figure out whether they can slash their properties to save money.
That means observing some strange things people do at the interstices between spaces. "People bump into each other, they stand there taking phone calls, they carry in boxes and bags and bicycles and carts, and sometimes they carry one another, or hats or plates," Farah told ZDNet in a phone call.
"Human behavior is so multivariate," said Farah.
To sort out what's coming and what's going, without confusing a plate or a hat for another person, Density has developed its own machine-learning algorithms that employ convolutional neural networks, or CNNs, the workhorse of modern image-recognition programs and numerous other applications of artificial intelligence.
The entire machine learning task operates on the device, a small silver box that sort-of resembles Apple's Mac Mini.
On the device, an infrared depth sensor streams what's called a depth map of the threshold scene to a processor that has dedicated AI acceleration circuitry.
"The algorithm itself includes but isn't limited to a convolutional neural network specifically tuned for people-counting via a stream of depth data," according to Casey Kelso, Density's head of R&D and algorithm development.
The depth data consists of shapes of objects in the scene, and is tuned to distinguish between people and other objects, and track movement into and out of the space, Kelso told ZDNet.
The box's sensors don't capture any personally identifiable information, just shapes, Kelso noted. Once analysis of the scene is completed, the sensor sends a plus-1, or a negative-1 up to Density's cloud-based server, representing a presence of a person crossing the threshold plus their direction of movement. The server can then map that presence to a floor plan.
"This is where things get interesting from a customer perspective, as we can report analytics for how a room, floor, building, campus, or even how space is used at the city level," said Kelso.
There are a variety of reasons companies want such information, and they are turning to Density at an increasing clip, said Farah.
There's always been a need to have data about the use of office space, things like tracking how many employees badge into a floor.
But there's a new urgency. "Post-pandemic, it's just two things, keeping people safe, and avoiding leases and consolidating or rationalizing space to preserve cash for payroll," said Farah.
Some of Density's customers are essential facilities, things kept open during the pandemic, such as meat processing plants and distribution and fulfillment centers. Some are companies that are gradually sending workers back to offices. Both kinds of clients need to plan and monitor on a regular basis how many people will be in an area.
"They may have a space such as a cafeteria that is allowable for fifty people at a time," to preserve distance, explained Farah.
"But if they have 300 people in the facility, they need to know in real time whether it's safe for an individual to go into the cafeteria." An alert coming from the sensor and going up to the cloud can trigger a message that gets beamed down to a flat-screen monitor outside the lunch room to let employees know if it's currently safe to enter, or if they should wait.
At the same time, Farah pointed out, 41% of all U.S. office space is going unused, according to CBRE Group, the biggest commercial real estate services and investment firm in the world. That's 4.5 billion square feet of empty space that's worth upwards of a trillion dollars.
Think of it as the anti-We Work situation: too much inventory, excess liquidity in a world of stay-at-home and high unemployment.
"In an 'up' market, everyone knows the excess," explained Farah. "In a 'down' market, everyone looks for ways to reduce cash burn."
Density's sensor network can "surface where there is under-utilization," he said. With that data, companies can decide not to renew a lease, or pause new capital projects, and consolidate facilities.
"Many companies are shedding 50% of their office space overnight and turning half the team into a remote team," Farah told ZDNet.
"All new capital projects are on hold, and they are not renewing upcoming new leases," he said of the average client posture these days.
Take the example of Apple, a company with 145,000 people. "They went from in-person to wholly distributed overnight, that's just crazy."
Business at Density grew by 495% in 75 days this year, sped up by the pandemic, Farah told ZDNet, without giving dollar amounts. The pricing model is based on a one-time fee of $895 for the set-up of each sensor, and then a fee per sensor per year of $795 for the cloud data feed.
The company is selling thousands of sensor units to some customers for multiple sites in multiple countries. One customer of Density's, a 125-year-old-firm that has over 80,000 employees, has 60,000 of them working from home for the first time in the company's history, across buildings in 60 different countries.
Overnight, the executives who manage facilities have moved from being a cost center that reports into the finance department to "being in the strategic center of corporate decision making," Farah observed. "They are the experts on safety," he said, and probably now the experts on slashing costs, too.
It's taken a while to get to this point. Density was founded as what sounds like one of those classic engineering larks, the kind of thing that comes from smart people whose brains have cycles to spare.
"When we started, we wanted to know how busy our favorite coffee shop was," in Syracuse, New York, he said. They were building a system to monitor the line at the cafe remotely, recalled Farah. Farah, a creative writing major who previously ran a consultancy, described his own entrepreneurial trajectory cheekily as avoiding getting a real job for over a decade.
What started as a lark became an intense multi-year odyssey. The work of productizing sensor technology took three-and-a-half years of wrong turns before it was ready to ship. "We thought it would be a weekend project," he recalled. "We made our first really bad technology in 2015," he said. "It wasn't good enough, so we continued to iterate."
Today, final test and assembly for the sensor box takes place in the company's Syracuse factory, where perhaps fifteen staffers work. The site was granted essential facility status. The rest of Density's staff, totaling 50, are spread throughout fifteen U.S. states and five countries.
The headquarters office in San Francisco goes unused at the moment. The entire hardware engineering team operates from their homes. Density had been working in this decentralized, virtual fashion for years before the pandemic. "It was no different on the Monday than on the Friday" that California went into lockdown, said Farah. "Friday, all of us were on a team Zoom meeting; we'd been doing this kind of thing for years."
Despite the early adaptivity of his staff, Farah is aware that the changes that are propelling his business at the moment come with a cost. "I prefer being in person," said Farah. "I find it really enjoyable, personally."
"We will continue to have offices," he said. "I don't believe in this concept that either everyone's coming back or no one," he said. "Life tends to be more complex."
When asked about the late Steve Jobs's contention that serendipity, the chance encounter of people face to face in the physical world, is not easily captured in a digital world, Farah conceded the point. Nevertheless, new ways of brainstorming can take shape, he replied.
Something has shifted, Farah told ZDNet. "Some things are discovered." He cited the example of one of his model companies, payments processor Stripe, whose system of documentation he adores. In a new world of decentralized, virtual teams, very good documentation may make up for some of the serendipity lost.
"In the age before digital communications, you could bump into people and have enormous creativity come out of that," he said.
"With really good documentation, the ideas of two people can be distributed to one hundred employees."
Two of Farah's favorite books on innovation are Where Good Ideas Come From: The Natural History of Innovation, by Steven Johnson, and How Innovation Works, by Matt Ridley. Both of those books make clear that In a virtual world, inspiration may come about in new ways.
"You are going to get ideas, not people, to collide," he said.