THE goal of automation has always been efficiency. What if artificial intelligence (AI) sees humanity itself as the thing to be optimised?
When Conor Sprouls, a call centre customer service representative, talks to a customer on the phone, he keeps one eye on the bottom-right corner of his screen. There, in a little blue box, AI tells him how he’s doing. Talking too fast? The program flashes an icon of a speedometer, indicating that he should slow down.
Sound sleepy? The software displays an “energy cue,” with a picture of a coffee cup. Not empathetic enough? A heart icon pops up. For decades, people have fearfully imagined armies of hyper-efficient robots invading offices and factories, gobbling up jobs previously done by human beings. But in all of the worry about the potential of AI to replace rank-and-file workers, we may have overlooked the possibility that it will replace the bosses, too.
Sprouls and his colleagues still have plenty of human supervisors. But the software on their screens – made by Cogito, a Boston AI company – has become a kind of adjunct manager, always watching them. At the end of every call, Sprouls’s Cogito notifications are tallied and added to a statistics dashboard that his supervisor can view. If he hides the Cogito window by minimising it, the program notifies his supervisor.
Cogito is one of several AI programs used in call centres and other workplaces. The goal, according to Joshua Feast, Cogito’s chief executive, is to make workers more effective by giving them real-time feedback. “There is variability in human performance,” he said. “We can infer from the way people are speaking with each other whether things are going well or not.” The goal of automation has always been efficiency, but in this new kind of workplace, AI sees humanity itself as the thing to be optimised.
But using AI to manage workers in conventional, 9-5 jobs has been more controversial. Critics have accused companies of using algorithms for managerial tasks, saying that automated systems can dehumanise and unfairly punish employees. While it is clear why executives would want AI that can track everything their workers do, it is less clear why workers would.
“It is surreal to think that any company could fire their own workers without any human involvement,” said Marc Perrone, the president of United Food and Commercial Workers International Union, which represents food and retail workers. In the gig economy, management by algorithm has also been a source of tension between workers and the platforms that connect them with customers. This year, drivers for on-demand delivery companies protested over a method of calculating their pay, using an algorithm, that put customer tips towards guaranteed minimum wages – a practice that was nearly invisible to drivers because of the way the platform obscures the details of worker pay.
Some staff seemed to view their Cogito software as a mild annoyance at worst. Several said they liked getting pop-up notifications during their calls, although some said they had struggled to figure out how to get the “empathy” notification to stop appearing. Cogito says the AI analyses subtle differences in tone between the worker and the caller and encourages the worker to try to mirror the customer’s mood.
Still, there is a creepy sci-fi vibe to a situation in which AI surveils human workers and tells them how to relate to other human beings. It is reminiscent of the “workplace gamification” trend that swept through corporate America a decade ago, when companies used psychological tricks borrowed from video games, like badges and leader boards, to try to spur workers to perform better.
Phil Libin, the chief executive of All Turtles, an AI start-up studio, recoiled in horror about the call centre experience. “That is a dystopian hellscape,” he said. “Why would anyone want to build this world where you’re being judged by an opaque, black-box computer?”
Defenders of workplace AI might argue that these systems are not meant to be overbearing. Instead, they’re meant to make workers better by reminding them to thank the customer, to empathise with the frustrated claimant on Line 1 or to avoid slacking off on the job. But as more AI enters the workplace, executives will have to resist the temptation to use it to tighten their grip on their workers and subject them to constant surveillance and analysis. If that happens, it won’t be the robots staging an uprising.
