While sitting across my kitchen table from a friend doing her own remote work, she happened to mutter, “I think my boss is watching me.”
She didn’t know I was in the middle of annotating the startling report published last week by a coalition of research and labor organizations, titled Bossware Comes To California.
Her boss was innocuously and visibly looking on as she clicked between spreadsheets; what the bossware report describes is the discreet systemic implementation of AI-powered surveillance across sectors of labor. It names dozens of platforms designed to collect and digest data about individual workers, often discriminatorily, and often without employees’ knowledge or consent.
Workplace surveillance and automation has been an issue for decades, but the implementation of AI bossware systems has accelerated employers’ ability to intensify work, increase control over workers, and extract more value from their labor. Bossware, according to the report, can look like physical devices — GPS trackers, cameras, microphones, wearable or handheld devices — or electronic monitoring software collecting data from a worker’s performance to their “biometric information, such as fingerprints, facial features, eye movement, gait, or voice characteristics.” Some bossware systems continue to track and gather data on employees outside of work hours.
California recently passed four first-of-their-kind laws establishing beginner guardrails for automation in labor. SB 947 banned employers from using AI to unilaterally discipline or fire employees with no human oversight or corroboration. SB 951 required expanding research and the information available to workers when AI causes layoffs. AB 1331 banned AI surveillance in workplace bathrooms.
And AB 1883, the “Stop AI Strip Searches Act,” banned bosses from using AI surveillance tools to track, recognize, or predict the emotional states of workers, or collecting their neural data.
All of those measures should absolutely be in place, although the report emphatically states they are not comprehensive enough. The details, however, should make you pause and consider why they were necessary.
The Bossware report lists evidence compiled from the firsthand experience of twenty-five California workers across many industries, including healthcare, warehousing, software engineering, janitorial services, sanitation, and hospitality. The takeaway: Bossware is widespread across industries, especially surveillance that collects biometric data.
The Los Angeles Alliance for a New Economy (LAANE) is a part of the coalition that published the report. LAANE’s Executive Director, Víctor Sánchez, agreed that, of the report’s revelations, the secretive tracking of workers’ emotional states stands out.
“I feel like, honestly, it’s one of the most scary things that we kind of learned in doing this report,” he told me. “I think my mind goes immediately to,, what allows a technology to be able to make a determination on someone’s emotions. And what that takes is facial recognition, eye scanning, these things that are terribly intrusive. And so I think, for one, it’s scary to think what that tech could be used for in the future.”
The future of work is the thematic concern; the tangible ones — how the well-established encoded bias of AI is violating workers’ civil rights, how the use of bossware erodes worker power and enables union busting, how algorithmic management pushes workers to perform at unrealistic or impossible standards that endangers their mental and physical health — are glimpses of an anti-worker future. Automating human discernment and complex work will eventually deskill the human workforce and reshape labor entirely.
“This idea that workers themselves are being pulled out of decision making roles where they can exercise judgment — where they’re not analyzing, diagnosing or trying to assess critical problems — Instead, those processes are being automated,” Sánchez told me. “Largely an excuse to be able to pay workers less.”
He said that the conversation of how AI and automation could change the workforce is no longer a conversation; it’s happening, and the data employers are collecting from their workers is helping accelerate it. “The harms are being realized now,” Sánchez emphasized. “ I think the purpose of these tools is to deskill, to move towards this workerless future, regardless of whether or not the tech is actually good enough to do what they want it to do. I think for all intents and purposes, employers are deploying this technology because they see it as an important stepping stone to get to where they want to go.”
The discrimination enabled by uncritical use of biased AI systems in the workforce is another alarm sounded in the report. Especially where biometric information and facial recognition technology is concerned, racial bias in AI detection systems is well-established. Bossware isn’t even close to an exception. Workers of color and immigrant workers are disproportionately targeted for discriminatory and predatory employer practices bossware systems enable. ‘Surveillance wages,’ or employers’ use of predictive analytics to identify the lowest pay a worker would be willing to accept, is only one example.
“These tools are imperfect, and so oftentimes when they’re treated as gospel, the outcomes are thought to be perfect when they’re not,” Sánchez expanded. “We’ve got some examples in the report that we uplift, of workers being terminated, [and] having no course of action because it’s not transparent who it is that they can speak to. And so these become tools of furthering inequities in the workplace.”
When it comes to a course of action and call for regulation, the report says plainly that they don’t expect federal regulation of bossware in the near future, but discourages nihilism. The four California laws aren’t a fix, but they are a win, and an established regulatory baseline. The report lays out four new policy points as immediate goals.
1. Employers, not algorithms, should be the ones making decisions about workers, and workers must have the right to appeal.
2. Workers should have the right to know when and how bossware is used by their employer, and the right to access and correct their data.
3. Employers should be prohibited from using bossware in ways that are intrusive and harmful.
4. All workers must be protected, and they should have the power to enforce their rights.
LAANE and the coalition behind the report, Sánchez told me, is working on local regulatory frameworks in four major regions of the state, with the hope of any eventual federal regulation having California as an example to build upon.
“And a lot of those have to be, I mean at its core it’s about democratizing the process, right? It’s making sure that people are at the center of how these technological tools are being deployed and not left out on the periphery,” he said. “We think that that can create great momentum, not just at the state level, but across the country, just given the sheer size and scale of some of our regions. LA alone is larger than a lot of states.”
Starting at the local level for something as ubiquitous as workplace AI surveillance regulation seems frustrating, at first. Ideally, this would be shut down at the federal level very soon, and NLRB standards and worker protections could evolve as quickly as AI. Imminent federal progress being unrealistic doesn’t make local efforts futile. Sánchez reiterated that.
“I think city council members, even school districts, right, have an opportunity to look at how AI and tech, or tech that’s AI enabled, is being deployed within their jurisdiction, and look for how they can meaningfully put up practical guardrails to make sure that people are at the center,” he said. “I think, you know, a large part of the challenge with state legislation is, there are a lot of interests that are at play. And so we weren’t able to win everything that we wanted to.
I do think that it’s got to be a joint effort between local and state, because local is really where the rubber meets the road. I mean, you’re seeing a lot of the fights around data centers emerge locally. A lot of the stories that we’re hearing are stories that people are experiencing on the day to day, and that has to ultimately guide the type of change that we’re trying to win, how people are being impacted every day.”
No matter what you do for work, and even if you are unaware of it happening, it is very possible you are being tracked, data mined, and quantified by bossware powered by imperfect AI systems that may or may not be overseen by humans. As far as what’s possible with our data employers collect — is it worth finding out?
“We’re looking at the deployment of bossware as a kind of laboratory experiment,” Sánchez told me. “Now there’s obviously a clear, longer-term threat in terms of where these companies want to go.”
I know this isn't what you mean, since I work for myself, but I do feel like I'm being surveilled all the time … but I was afraid "they" would notice if I said anything. 😏 Phew. It's not just me.
Apropos of nothing, is ECHELON still a thing?