When man meets machine: Using AI to improve productivity of factory workers

Rewind a bit and imagine that a monitor emits a warning to the worker just as he’s making the mistake. He can correct it, complete the action and move on. “Think of it as the spell check function on Word. If you make a mistake, the software highlights it by showing you a red squiggly line. You correct it and move on. After that, no one in the world knows you made that mistake,” says Prasad Akella, the man behind the world’s first collaborative robots or cobots made when he worked at General Motors in the 1990s.

That’s exactly what Drishti, an American startup with an R&D base in Bengaluru founded by Akella in 2016, is doing. A camera over each station on a factory floor uses computer vision to capture workers’ movements. The steady stream of video that is generated is piped into Drishti’s patented deep-learning software and oversees every floor person’s work. It can capture missteps and alert the worker in “near real-time” (within two-three seconds) and measure, among other things, how long a worker takes per task so that bottlenecks, if any, can be eliminated. In effect, Drishti uses technology to augment workers rather than replace them.

To put it differently: Just as Google mines text to derive insights, and Apple’s Siri mines voice, Drishti is mining video streams, explains Akella. “It’s never been done before because it’s a hard problem to solve. But the opportunity is huge.”

Huge because nothing much has changed in manufacturing since the time Henry Ford combined moving assembly lines with mass labour to make cars cheaper and quicker in the early 1900s. In fact, people still use stop watches as they did in the past—albeit on their phones—to log cycle times of workers and stations in a factory, says Akella. Moreover, five to 10 data points, at most, are used to track output. “Companies are making decisions worth millions of dollars based on five measurements. Imagine the inaccuracy that would be there,” he says.

Despite the much-touted rise of robots that were expected to automate factories, manual workers are still central to manufacturing. Considers the statistics: There are roughly 2.5 million robots in the world today versus 350 million workers on factory floors. A survey of more than 100 leading US manufacturers by AT Kearney and Drishti found that human beings still perform 72 percent of manufacturing tasks and produce three times the value that machines do.

“Robots have not yet been able to replace humans because they’re just not as smart. We’re many decades away from that happening. Till then the opportunity lies in focusing on those 350 million people,” says Akella.

Drishti was incubated at the Stanford Research Institute (SRI), a 75-year-old institution responsible for advances such as the colour television, the computer mouse and the 911 emergency call system. Akella spent a year at SRI. During that time Ashish Gupta, founder of price comparison platform Junglee.com, which was acquired by Amazon in 1998, as well as co-founder of Helion Ventures, and Krishnendu Chaudhury, an ex-Googler and computer vision authority, joined as co-founders of Drishti. The trio was able to rope in two large customers while still at SRI to test Drishti’s initial offering. “Before we had a team, before we had a dollar in funding, these customers saw the value of what we were building and helped by giving us practical feedback and guidance,” says Akella.

One of the early points of contention was whether to use the Internet of Things (IoT) to track workers’ actions on the factory floor rather than cameras. “IoT is also a valid way of solving this problem,” says Chaudhury. Valid but with shortcomings, he quips, because IoT requires a tracking device, like a wristband, to be put on a worker which makes it intrusive. Cameras that use computer vision, on the other hand, aren’t so. Besides cameras can provide a comprehensive view of the factory floor, unlike a wristband that captures just a sliver of that. “Prasad and I went back and forth on this before converging on our ideas,” says Chaudhury who was part of Google’s machine intelligence team at its campus in California, before returning to Bengaluru to run Flipkart’s deep learning projects.

Once the plan of action was charted, Chaudhury, who serves as CTO, dove right in. No one had previously used AI-powered computer vision to extract process data from video streams—also called action recognition—in the manufacturing domain.

While computer vision technology has been around for 50 years it merely looked at single frames, like a door that is ajar. By looking at a single image it’s impossible to tell whether the door is opening or closing, explains Chaudhury. “But if I show you the video, you would be able to tell me. In other words, you have to make multiple observations in time, that makes it 3D in nature, rather than a simple 2D image,” he says. To crunch this 3D data, Chaudhury and his team of engineers—all based in Bengaluru—had to build neural networks from scratch. Off-the-shelf versions didn’t cut it. Three patents have been granted to Drishti for this pioneering work while another 12 are under review with the US Patent and Trademark Office, a government agency.

Furthermore, Drishti has also developed a technology that minimises the human input needed to train their systems. “Neural networks learn examples. Instead of writing a programme, you show examples to the neural network and it learns what to do. Those examples need to be created by human beings and that is the most painful part of neural networks or deep learning. Painful because it’s boring and repetitive,” says Chaudhury. His team has reduced the human input needed by a factor of 100 over the last three years.

All this is translating into real gains for customers. Take the case of Hella, a German car-parts supplier, which used Drishti’s technology to analyse and fix problems in a high-volume assembly line. It led to a seven percent rise in throughput.

“Drishti’s technology helps us support our lean practices with data and use the information we get from the cameras and AI to train our operators, improving their productivity and job satisfaction at the same time. It’s a win-win for the company and our employees,” says Michael Hammoud, vice president of operations, Hella.  

Besides automotive, Drishti has customers in the medical device and electronics verticals. “These are top-end manufacturers whose defect rates are single digits parts per million. That’s ridiculously good manufacturing,” says Akella. “But with our technology—giving people immediate insights on what went wrong and why on a manual assembly line—our customers have seen a reduction in those defect rates by 50 percent, which in turn leads to a 10-20 percent improvement in capacity. That’s a journey that can take years; we can deliver that in weeks. That’s the power of what we do.”

While Drishti’s 20-odd customers, according to a recent report by S&P Global Market Intelligence, are largely concentrated in North America at present, Drishti is looking to expand its geographic footprint in Europe, as well as the Asia-Pacific region, including Japan and Korea. “Drishti’s current priority is growth,” note the authors of the report.

To that extent the company has reworked its pricing model—customers pay a price per stream per year—to be much more affordable, starting at roughly one-fifteenth of its original cost. Drishti chose not to share the specifics.

Industry is taking note of Drishti’s work: The World Economic Forum named Drishti ‘Technology Pioneer’ in 2019 and Nvidia named it the ‘Top AI Company’ in North America in 2020. Investors too are upbeat. Drishti’s latest $25 million Series B fundraise in June 2020 was led by Sozo Ventures with participation from Alpha Intelligence Capital, Toyota AI Ventures, Micron Ventures, Presidio Ventures, Hella Ventures, as well as existing investors Emergence Capital, Benhamou Global Ventures and Andreessen Horowitz.

“The team’s passion to use this technology to improve productivity, quality and training for manufacturers around the globe is what initially attracted us, and their proven ability to execute is what continues to drive our excitement for the future of Drishti,” says Martin Casado, general partner, Andreessen Horowitz.

So what will a factory of the future look like? Says Akella, “Just like you have power going to every station, we’ll have video instrumentation on every line in every station in every plant in the world.”