Containment issue

An end-to-end encryption promise is not enough anymore

Happy Friday, privacy-minded readers! Dropping another week’s worth of digital surveillance stories into your mailboxes.

This week’s TechCrunch story about Stardust, a female health tracking app, made me think about the architecture of data privacy in the digital age. The gold standard of data privacy is end-to-end encryption, which allows data to be transferred without revealing its content to any third party and keeping everything sensitive on the user’s device.

But how can a non-technical user know whether the apps they use are end-to-end encrypted or not? One famous example is the still unresolved question of WhatsApp’s privacy: the app says it’s end-to-end encrypted, but an unfinished investigation by a U.S. government agency, revealed by Bloomberg, shut down before any results were publicly shared, implied that Meta can read users’ WhatsApp messages.

So our privacy essentially relies on the willingness and capacity of app creators to contain our data inside our devices – and their honesty about it. The “container” is constantly online, and whether it’s secure or not is ultimately a matter of trust.

The question becomes even more timely as humans are increasingly surrounding themselves with sensing machines – from health tracking gadgets and smart speakers to humanoid robots already on their way to our homes (check out this story by The New Yorker for the current state of humanoid robotics).

Any device is an AI device now, which makes the issue of containment a thorny one: to learn and expand, AI needs massive amounts of data, so restricting it to each individual user’s device severely curtails the capabilities of AI. Plus, AI systems are becoming more complex and harder to control by the hour, and they can crawl outside the boundaries defined for them.

Does it mean we are bound to see the existing digital privacy architecture crumble under the onslaught of inventive algorithms, and if it’s possible to build a new one, what would it look like?

That’s a question for someone with much more technical knowledge than me, so if you know a person who might have ideas – forward this newsletter to them!

And in the meantime, let’s dive into this week’s stories.

Support and share this newsletter if you’re enjoying it!

Biometrics briefing

• U.S. Congressman Josh Gottheimer introduced bipartisan legislation requiring online betting platforms and prediction markets to use facial recognition technology to verify users’ ages. – News 12

• California discount retailer Grocery Outlet has started using facial recognition at store entrances to prevent shoplifting. – Mission Local

• The town of Hudson, Massachusetts, will equip public schools with an upgraded security system, including facial recognition. – Community Advocate

• In the UK, over 125 retail chains use a live facial recognition system that will soon begin alerting police when someone previously caught shoplifting enters a store. – Biometric Update

• In Japan, Tokyo’s Ikebukuro subway station has launched a facial recognition ticket gate system. – International Business Times

• Police in Tokyo will use street cameras with facial recognition to search for missing people. – South China Morning Post

A Flock update

Flock, an AI license plate reader maker, made headlines this week again. 404 Media has found that the firm’s surveillance cameras sold under the brand name Condor (designed to work in combination with license plate readers) allow searching for people, in addition to cars.

Reporters reviewed data on police requests in Flock’s system and found searches for people wearing specific clothing, having tattoos, or carrying items like backpacks or skateboards. The analytic capabilities of AI turn hours of video footage into a searchable database, allowing user to look up any object matching a description.

For example, police requests included descriptions like “american flag shirt,” “male with tattoos,” “male with brown hair,” “woman blue shirt,” “man wearing a black t-shirt and shorts,” among others, according to 404 Media.

In the past, police officers would have to review footage from street cameras manually to find specific people and things. Today, the new generation of AI video surveillance systems allows for constant real-time, as well as retrospective, search inside a vast video archive.

In the meantime, Flock is getting pushback in some parts of America. The Los Angeles Police Department (LAPD) announced it will not renew its contract with the company. The decision followed an audit that showed frequent cases of innocent people being flagged by the system, 404 Media writes.

According to the report, the LAPD improperly investigated 161 people because their vehicles were flagged as stolen but in fact were not. In many cases, the database was not updated in a timely manner, so cars remained on a hot list after a stolen vehicle had already been recovered or turned out not to be stolen.The LAPD database relies on the information from other police departments, which might be slow to update their hot lists.

During the two-month audit period, 5,911 license plates were tracked, but for 4,575 of them, no action was taken. 337 stolen cars were recovered and 74 arrests made thanks to the license plate data, according to the audit. This means that people who actually did nothing wrong might still be tracked by the police, and when such a vehicle is stopped the police considers it a “high-risk” stop. The audit suggested suspending the deployment of automated license plate reading tools.

Eyes in the sky

More from the world of spying things: police and fire departments in the U.S. are increasingly using drones to spot illegal fireworks. The footage is used to punish people who launch the fireworks; for example, the Sacramento Fire Department in California issued a $100,000 fine for illegal fireworks this July with the help of drones, Ars Technica reports. Video filmed by the drones helped investigators to identify the location using Google Maps, the firefighters said.

In the meantime, a leak from the San Francisco Police Department offered a glimpse into how police in the U.S. are using drones for surveillance, Wired writes. Videos from SFPD Skydio drones were discovered by security researchers Sam Curry and Maik Robert.

The leak includes 60 videos from 20 drone flights, featuring hundreds of people and vehicles, according to Wired. The footage shows police chasing and detaining people, but also more peaceful and innocuous scenes. For example, in one case, a drone followed two young men in a car, one of whom was marked as a “suspicious person in a vehicle” in the police report – they arrived at a basketball court and started playing, after which the drone left. Another drone filmed a person sitting on the roof of a building wearing headphones.

The practice is not unique to America – across the world, in India, the Railway Protection Force has started using drones with cameras and speakers to stop people from throwing stones at trains and prevent situations in which trains run over cattle, The Hindu writes.

In court for bossware

The AI workplace surveillance tool that Meta had previously launched to track employees has backfired again: now, 26 workers are suing the company, saying AI was involved in the May round of layoffs, The Guardian reports.

Meta let go of 8,000 people in May. Shortly before that, the company deployed the so-called Model Capability Initiative, which was designed to capture every action employees take on their work computers to train Meta’s AI models.

Now, the employees claim Meta used AI algorithms to pick who should be let go based on criteria that punished people who took medical leaves and employees with disabilities.

As a result, one of the plaintiffs was laid off two days before giving birth during her approved pre-birth pregnancy leave; another one said his rating was lowered because he took a medical leave following an injury. At the same time, AI tracking was launched in a way that barely gave employees any way to opt out, the lawsuit reads. The program was canceled in late June after a security flaw exposed the monitoring data to everyone within the company.

The plaintiffs are now asking the court to approve an independent audit of Meta’s AI tools and stop the layoffs.

OpenAI’s listening machine is coming

Bloomberg has learned details of OpenAI’s mysterious upcoming smart home device. The screenless smart speaker, developed by a team of former Apple designers, will have a camera and other sensors that can gather information about the user’s surroundings. The device will be able to control smart-home appliances, play media, answer questions using OpenAI’s artificial intelligence capabilities and even respond to its user’s messages, sources told Bloomberg.

OpenAI is planning to make the device highly personalized (and probably addictive), as it is expected to gain a deep understanding of its owner’s life and even anticipate their needs, connecting with users “on a humanlike level.” The device is designed to be portable, so that users can carry it around the house. According to an earlier report by The Information, the gadget will also have a camera with facial recognition capabilities.

The smart speaker will be equipped with an advanced version of the ChatGPT Voice Mode that will allow it to listen and talk at the same time, making the experience even closer to a human conversation, Bloomberg reports.

In the meantime, Apple, which has lost some designer and engineering talent to OpenAI, is suing the ChatGPT creator and alleging the departed employees took some confidential information about Apple’s future products and engineering know-how with them. Apple itself is also planning to launch a new generation of home devices armed with AI soon.

Health data for sale

Period tracking app Stardust is sharing users’ health information with an analytics company RudderStack, TechCrunch reports. Data like users’ birthdates, birth control methods, reproductive goals, and specific symptoms shared with the app were among the data sent to RudderStack, according to the latest research by Mozilla.

More importantly, the data was tied to unique user identifiers, which are used to replace people’s actual names but are not a perfect privacy protection, according to experts. RudderStack is advertising itself as “built for a world where humans and autonomous agents work together to collect, unify, and activate customer data.”

Stardust, which gained popularity after the U.S. Supreme Court overturned the constitutional right to seek an abortion, claimed to be end-to-end encrypted, but the app’s network traffic shows this is not true, according to TechCrunch’s own analysis. Mozilla security researcher Shoshana Wodinsky also analyzed five other female health apps but only Stardust was sharing sensitive health data with a third-party company.

Data generated by human bodies is the next frontier for AI, opening new opportunities for research, but also for surveillance and control. It’s hard to predict who may get access to such data once it’s accumulated in a cloud beyond users’ reach, but a potential list includes anyone from marketing agencies to health insurance companies to law enforcement.

A famous example is fitness app Strava, data from which has already been used to track people and even military objects. However, data on jogging routes is something people themselves are willing to share. Sensitive health details are a completely different story, but also a promising material for surveillance.

Surveillance export: from Spain to India

The more capable a surveillance technology is, the bigger power it puts in the hands of whoever is using it, and the bigger the risk of abuse. For that reason, countries around the world that treat human rights seriously are working on regulations that limit and restrict the scope of technological surveillance.

However, technology is global and laws and regulations are local: once a powerful surveillance tool is created it’s hard to predict where in the world and how it will be deployed. Take the Spanish tech company Herta Security.

Over the past five years, the firm has received no less than 3 million euros in research funding from the EU and Spanish national programmes, including 2.36 million euro for a project called FUTURE. Herta developed technology to analyse crowd behaviour and “identify abnormal activities and potential threats in large gatherings, public events, and high-traffic areas,” as well as “identify suspects and terrorists,” Investigate Europe writes.

However, since February 2025, the EU has maintained strict rules for AI surveillance systems. The EU Artificial Intelligence Act forbids deployment of real-time remote biometric identification systems in public spaces for law enforcement purposes, unless they are used for a targeted search for missing persons and trafficking victims, for preventing violent crimes and terrorism, or to identify a suspect in a serious crime.

This would likely make Herta’s product illegal in the EU, where it was created, but the company has successfully exported its technology to other countries, especially India, Investigate Europe reports. Herta’s facial recognition technology has been deployed in hundreds of railway stations, a prison in Delhi, a pilgrimage site in Ayodhya, and other places – overall more than 4,000 cameras across India, journalists found.

The proliferation of surveillance technology was at least partially funded by the Nirbhaya Fund. The fund was created by the government as a response to alarming rates of sexual violence, following the infamous bus gang rape in 2012 in Delhi, when five men attacked a young woman and her male companion, beat up the man and raped the woman, who later died of her injuries. The fund was supposed to enhance the safety of women and girls.

According to the official data, half of all the money was spent on surveillance and policing, and about 31 per cent went to direct victim support programs. And this seems to be the universal response for crime issues around the world: when it comes to distributing resources, surveillance trumps community support and social policies.

Whether this is what we want must be up to citizens to decide – otherwise we end up in a surveillance panopticon that does nothing to make our lives better.

***

And that’s a wrap for this week, folks.

Stay vigilant!

Anna