AI + bossware = …

Humans under AI supervision teach robots to replace them

Hello, Control, Spy, Delete readers! Apologies for showing up late in your inboxes this week – I’ve been fighting a cold and spent too much time trying to gather my thoughts.

Which, actually, brings us close to the topic that drew some significant attention this week: weak humans being replaced by infallible machines. We’re in the very beginning of this trend, but we can already observe how and where it’s going.

First of all, companies desperately trying to adapt to the AI hype want to minimize the human workforce (those annoying humans always demanding something, like a healthy workplace, sick leaves, lunch breaks…) and maximize AI-driven automation. But the machines still don’t have enough data about how exactly work tasks should be performed to successfully replace humans. So they need to watch them work – every action, every mouse click, every piece of data used and produced.

This leads to a new trend: workers find themselves under increased digital surveillance at the workplace while every step they make in the digital realm instructs the AI that is planned to ultimately replace them.

This creates a double stress for workers who, on the one hand, have to work under a never ending, all-seeing surveillance regime, and on the other hand, do it knowing that with every action, they are moving closer to getting replaced.

According to the April report by the International Labor Organization, titled AI Systems at Work: A Changing Psychosocial Work Environment, the AI-fueled race for efficiency itself is a huge stress factor, exposing workers to cognitive overload and increased expectations: “automated management leading to increasing number of tasks in shorter working hours; unachievable or unreasonable key performance indicators.”

At the same time, under AI supervision, humans see their professional autonomy shrink, along with their sense of purpose at work. Excessive monitoring leads to erosion of trust and human connection at the workplace. In short, human workers become increasingly powerless and miserable under the all-seeing AI gaze.

It’s a hard pill to swallow for most of us, because the leaders of this big AI overhaul seem pretty content with the direction in which they are leading us, and the governments of the world, along with big-pocket investors, reward their approach. Anthropic’s head of product Cat Wu told TechCrunch this week that the company is envisioning a future in which AI understands humans’ work so well it proactively automates work processes before even being asked to.

There is a lot to think about here, and well beyond the scope of this newsletter, too, but one simple outcome I can think of is this: workplace surveillance will get more omnipresent and intense, and, if the course doesn’t get reversed, employees will face a loss of privacy we’ve never seen before.

Make your own conclusions – and let’s get into this week’s stories.

Don’t forget to share this newsletter with your privacy-minded friends and colleagues if you’re enjoying it!

Discontent at Meta

Meta, which is planning to lay off almost 10% of its staff, has recently notified its U.S. employees that the corporation will install tracking software on their computers, which will capture mouse movements, clicks, and ​keystrokes. All that data will help train Meta’s AI models on “how people actually complete everyday tasks using computers.” The staff, predictably, was not amazed.

The New York Times got word from inside the company that “many workers immediately revolted”. People have been upset about the destruction of their privacy, calling the new practice “antisocial and callous,” the NYT reports. When asked how employees could opt-out of the AI surveillance mode, Meta’s chief technology officer Andrew Bosworth replied there was no such option.

Meta has never been famous for its respect for privacy in general, but now, the company’s own staff is finding themselves under unprecedentedly granular surveillance – another collateral damage in the battle for AI dominance, am I right?

And how many more to bite the dust.

Canada drivers lead the fight

In the meantime, some Canadian bus drivers are successfully pushing back against AI workplace surveillance. The local trade union for school bus drivers ratified a new three-year collective agreement last week, which puts in place certain protections against AI-driven surveillance, the union announced.

According to the new agreement, “automated alerts, risk scores, and AI-generated driver ratings cannot be the sole basis for discipline” – an investigation and review of video footage must precede any disciplinary actions. The new rules also prohibit storage of drivers’ biometric data and establish limits for video storage and access.

The union represents school bus drivers in Toronto, and the agreement followed the transportation organizations’ switch to a new AI-powered video surveillance platform called Samsara.

Human Resources Director magazine provides an overview of the new AI surveillance tools employed by some of Canada’s major employers, including the Canadian government. For example, Amazon’s Canadian branch is now monitoring its fulfillment center workers with AI.

Telecom companies Bell, Rogers and Telus are using AI for performance monitoring in their call centers, with Telus also using an “AI accent-modification tool on offshore agents.”

Walmart Canada is using AI to monitor employees’ messages on the corporate platform. Loblaw, a food and pharmacy retailer, is using Axon cameras (the same ones police use) with “AI-enabled video features.”

Automation vs. human dignity

AI entering the job market will lead not only to increased workplace surveillance and the following loss of income by humans, but also to growing inequality inside the workforce, writes Nazrul Islam, University of East London professor of business, in his op-ed for The Guardian.

“Employers are using AI to empower some workers while subjecting others to more intensive, inhumane forms of oversight. It is creating new opportunities at the top of the labour market while tightening control lower down,” Islam writes.

While low-level workers are subjected to all sorts of invasive surveillance, the top management is holding the strings. The actual efficiency is not as important as the virtual, future, promised one: “We’re already seeing this at companies including Amazon, as its software engineers say they’re being surveilled and pressured to use AI to achieve more productivity, even when it counterintuitively slows them down.”

This creates a “widening gap in skills, autonomy and wellbeing between those who get to work with AI and those who are finding themselves managed by it,” Islam writes. “That matters because work is not just about income. It is also about dignity, trust and control.”

As companies are increasingly redirecting corporate resources towards AI-related goals, little goes toward training and adapting personnel to this tectonic shift, which makes this new inequality even more gaping.

“If better-paid workers are trained to use AI while lower-paid workers are simply exposed to it through surveillance and automated management, then this will not be a story of shared progress. It will be a story of deepening imbalance,” Islam warns.

But the managers still needed today may be replaced tomorrow, too, pulling the thread all the way up to the company’s leadership. Jack Dorsey, founder of Twitter and CEO of fintech company Block, who recently let go of almost half of its staff, is dreaming of a corporate structure in which all the remaining 6,000 employees report directly to him. With the help of AI, of course.

But if we want this AI transformation to happen for the good of all humans, not only the richest ones, the introduction of AI must be subject to open and transparent discussions within companies, and “workers need a voice in how these technologies are introduced,” Islam suggests.

Reasonable ideas all through. Now, what do we need to make corporate bosses listen? Any thoughts?..

Listening to your ER visit

An unrelated, but equally disturbing story from 404 Media: Mayo Clinic, a large U.S. hospital chain, is recording patients’ conversations with nurses and turning the recordings into AI-generated reports.

The clinics have printed notifications about “Ambient Listening,” as the company calls it, and patients can opt out. However, the notifications are not very conspicuous and can easily be overlooked, especially when a patient comes to the emergency room and has little awareness of the “extra” surveillance in the room.

“I did not talk to any staff about it, because we were in the middle of an emergency, which is part of my whole issue with this being an opt-out thing in the emergency department. So many people might not notice this or even be healthy enough to read or notice it,” one patient told 404 Media.

In July 2024, Mayo Clinic announced it was working with AI company Abridge on “a generative AI ambient documentation workflow for nurses” – the same company is also working with Johns Hopkins Medicine.

Unlike a human note taker, an AI system has access to all patients’ data all the time – and the way this data can be accessed, shared and used is governed by the technological platform in charge of the product.

You would think that in this wave of AI “optimization,” there would be at least some areas of human privacy that won’t be touched just because of how vulnerable people in certain situations can be.

But human dignity has no calculable monetary value – and human data very much does.

Palantir in Britain

In the meantime, across the ocean, another privacy situation is unfolding with Palantir, the powerful American data and AI company and one of the major contractors of the U.S. immigration authorities.

The Financial Times has found that the UK healthcare authority, NHS England, has granted several contractors, including Palantir, “unlimited access” to identifiable patient information as they work on the agency’s new data platform.

In 2023, Palantir won a £330 million contract to build the Federated Data Platform, which connects all NHS England data into a single system. Inside it, there is a database named the National Data Integration Tenant, where patients’ identifiable information is held before it gets anonymized and shared with other systems.

The NHS staff admits there is a “risk of loss of public confidence” when it comes to “safeguarding patient data and ensuring appropriate use and access to it” with the new practice, according to the agency’s documents FT reviewed. That risk did not stop the agency from providing Palantir with access.

Which tells you something about priorities in this AI revolution – globally.

***

And that’s all from me for this week, guys.

Stay vigilant!

Anna