• If LLMs Have Human-Like Attributes, Then So Does Age of Empires II

• Krebs. Who Runs the Ransomware Group ‘The Gentlemen?’

• A passive collector of intel on Telegram.

• SpaceX IPO.

• Claude Corps.

• Anthropic begged the world to stop AI… then shipped this

The editorial desk barely made it this week as I was at a conference (which you will hear about soon). That said, it is still Friday here in Minneapolis (with 2 minutes to spare), so all’s good. This is the continuation of last week’s post on the CivitAI saga. Note, this post does include some description of nasty stuff, so, buyer beware.

CivitAI prefers to describe itself as a platform, a community, and sometimes a marketplace. These convenient designations are conceived to avoid uncomfortable questions about the content that is shared. They are just a platform after all, and as with any ecosystem, there are just a few bad apples; it would be unrealistic for CivitAI to catch these perverts. So the story goes for all those who seek refuge behind the mighty Section 230––the same bulwark that shields social media companies of user-generated content liability. Regardless, CivitAI takes content moderation seriously and as a consequence has implemented numerous safety features that would prevent an innocent visitor from stumbling upon lewd imagery. The homepage dissuades passersby by displaying images that I imagine adorn the walls of AirBnb rentals marketed for bachelor retreats in Miami and Palm Springs. And, after creating an account, they provide a useful set of filters that allows you to block all the indecent content.

The ability to describe any image or video you want has seemingly not resulted in the advancement of noble goals. Instead, the most popular adapters––each boasting hundreds of thousands of downloads––showcase instructions that look like this:

Words between the brackets indicate customization possibilities.

If you were to run the instructions above, you would most likely receive an “artificial” woman. A depiction of someone who looks like a real person but does not actually exist. This provides little comfort. In a world where there are online tools that take Instagram handles, retrieve all pictures, and automatically create sexually explicit media and corresponding adapters, then the fact that we have “a deepfake nude epidemic” is not merely explained but entirely logical. Boys and men are sacrificing maidens at robotic altars in rituals of pleasure.

Pesky observers quickly caught wind of this practice. The chickens––along with two large condors––had at last come home to roost and the reports that CivitAI was enabling the creation of non-consensual explicit images (i.e., “deepfake pornography”) and “content that could be categorized as child pornography” entered the public conversation. The bad air emanating from Boise could no longer be ignored. Early in 2025, a crackdown seemed imminent: the UK began enforcing its Online Safety Act (OSA) in March and the TAKE IT DOWN Act cleared the US Congress in April––both laws covered deepfakes. In the background, payment processors had begun pressuring platforms like CivitAI.

By the end of April, CivitAI announced a sweeping change in their policies, highlighting some of the content that existed in the platform in its fourth year of operation:

We’ve updated Section 9.6 of our Terms of Service (ToS) to explicitly prohibit content depicting:

• Incest, including sexual activity between immediate or close biological family members.

• Self-harm, including depictions of anorexia or bulimia.

• Content that promotes hate, harm, or extremist ideologies.

• Bodily excretions, and related content;

• Urine

• Menstruation

• Smegma

• Diapers

Additionally, content in the following categories which depicts sexual activity or context that insinuates, or portrays, sexual intent (X, XXX) is explicitly prohibited;

• Firearms aimed at or pointed towards individuals.

• Vomit.

• Depiction of illegal substances or regulated products (e.g. narcotics, pharmaceuticals).

Content depicting sexual activity while in a mind-altered state is prohibited, including;

• Being drunk, drugged, under hypnosis, or mind control.

From: https://civitai.com/articles/13632/policy-and-content-adjustments

Remarkably, CivitAI misjudged the political climate and did too little in the two departments that mattered the most, opting for simple interface (UI) filters. Sexual images of real people were allowed to stay but would simply be hidden. And, content with child/minor themes were allowed to stay for the R category which they describe includes “adult themes and situations, partial nudity, bikinis, big bulges, sexual situations, and graphic violence.” Yikes.

cont.

We make certain types of content less visible across the platform depending on the content you have enabled in your browsing settings:

• Real people and celebrities: Content tagged with real person names (like “Tom Cruise”) or flagged as POI (real-person) resources will be hidden from feeds.

• Minor content: Content tagged with child/minor themes is filtered out of feeds whenever mature content (X, or XXX) is enabled.

By May 18th, CivitAI realized the error in their ways.

cont.

Update 5/18: Content filtering rules were extended — minor-tagged content is now hidden whenever mature content (R, X, or XXX) is enabled, not only X/XXX.

Soon after, they also reeled back their policy on depictions of real people, albeit a bit more begrudgingly.

We are removing models and images depicting real-world individuals from the platform. These resources and images will be available to the uploader for a short period of time before being removed.

We know this will be frustrating for many creators and users. We’ve spoken at length about the value of likeness content, and this decision wasn’t made lightly. But we’re now facing an increasingly strict regulatory landscape - one evolving rapidly across multiple countries.

From: https://civitai.com/articles/15022/policy-update-removal-of-real-person-likeness-content

Needless to say, these concessions were not deemed enough by the payment providers. On May 22nd, three days after President Trump signed the TAKE IT DOWN Act, payment processors rescinded support for CivitAI. Capital punishment. Without the ability to collect money from users, you will have a hard time delivering shareholder value. CivitAI is not the first company to face this dilemma. In 2020, Visa, Mastercard, and Discover cut ties with PornHub after an investigation which stemmed from a piece on the NY Times titled “The Children of PornHub.” The legal imbroglio dragged on all the way to 2022, when payment processors proceeded to go the extra mile and also cut ties with ad arm of Pornhub owner MindGeek. Until this day, the only two payment options for a premium membership are ACH transfers and Bitcoin.

Ahh crypto… the magical wand bypasses all sanctions and regulations. The bane of a regulator’s existence. Within a few days, CivitAI was relegated to also accept ACH transfers and cryptocurrencies. “If it wasn’t for crypto, these fascists would not let us have any fun on the Internet. To hell with these moralistic dictators and their concerns. Debauchery for the masses!” Cryptocurrencies have now existed for 14+ years. 10 of those have been in the light of popular culture with massive funding cycles, substantial development, and even two countries adopting cryptocurrencies as legal tender. If the cryptocurrency dream is alive and well, then it should be ezpz to swap out those suckers payment processors and rely on crypto. Users on CivitAI, by virtue of being consumers of a relatively niche AI product, are presumably tech-savvy and able to whip out the future of finance to avoid Visa and Mastercard’s sanctions. Or are they?

Next week, we will wrap up the last bit of this story and talk about what modern CivitAI looks like and what their trial and tribulations with various payment processors tell us about the effectiveness of financial interventions. Hasta la proxima chicos.

Adjacent Reading

• Exploring the Use of Abusive Generative AI Models on Civitai

• A Marketplace for AI-Generated Adult Content and Deepfakes

• “Unlimited Realm of Exploration and Experimentation”: Methods and Motivations of AI-Generated Sexual Content Creators

• Deepfakes on Demand: the rise of accessible non-consensual deepfake image generators

• The Atlantic. The Biggest Tell That Something Was Written by AI.

• WIRED. xAI Asks Court to Strip Alleged Grok Deepfake Nudes Victims of Anonymity.

• Krebs. Hackers Used Meta’s AI Support Bot to Seize Instagram Accounts

• 404. Companies Are Using Reddit to Manipulate ChatGPT and Google AI Search

In the summer of 1862, a traveler coming east from the Oregon Trail would see long stretches of sagebrush and volcanic rock, broken by narrow ribbons of water descending from the mountains. It was here that a few prospectors moved slowly through the valleys, stopping to pan gravel from creeks. They were looking for the same thing that men had been looking for before their time and even now in the present day, and finding it about as often as everyone else. It was in country like this that George Grimes and his companions began following the traces that would eventually lead them into the Boise Basin. Within months thousands of men would be traveling the same route.

A hundred and sixty years later, Justin Maier struck gold again. Seldom, if ever, does Boise appear associated with a technology company, less so with an enterprise dedicated to a concept that we cannot seem to define––Artificial Intelligence. For those “in the know”, Micron Technology might ring a bell, an enterprise that as many others has become a prime beneficiary of the modern gold rush by being in the industry of semiconductors. Micron’s computer memory chips are underneath many of the machines that today enable millions of people to write vapid sentences online, among other tasks. The Old West’s most famous adage was that prospecting was for gamblers; the business-savvy sold tools, shovels, pickaxes, and the like. Last week, Micron became the latest company to breach a trillion dollar market capitalization.

Justin Maier did not start Micron but had a similar idea to his Boise-based predecessors. If the electricity flowing through the runes carved in silicon rocks could generate language, then why not images and videos? A picture used to be worth a thousand words but today with far less you can transmute your imagination into digital pixels, and Justin’s CivitAI pioneered the modern alchemical workshop for mortals who do not know the `curl` or `npm` incantations. CivitAI has done an admirable job in breaking down the huge traditional barriers that separated noobs from wizards. The firm Andreessen Horowitz agrees. In 2023, after CivitAI surpassed 3 million users––only a year after launch, it secured venture capital funding.

Access to revolutionary technology has always been met earnestly by humanity. The Internet made vast troves of information available to the populace. Free encyclopedias and courseware from places like MIT were now suddenly available to barrio urchins and the promise of upward mobility was now attainable. People used the Internet to acquire knowledge, expand their cultural horizons, and create bridges of understanding with fellow humans from distant places. It stands to reason, then, that people will use these new artificial intelligence tools for the same noble goals. In exchange for making the world just a bit warmer each time, our prompts allow us to create better scientific illustrations, more effective didactic materials, and a completely new creative canvas for artistic exploration. Photography, with a single finger click, betrayed a generation of virtuosos who could achieve photorealism with brush strokes. AI-generated images promise to do the same and CivitAI is at the forefront of this creative revolution.

The computational generation of images first involves creating a visual encyclopedia––a massive collection of appropriately described images. Through the observation of millions of images including objects, scenes, faces, bodies etc., a computer is able to seemingly “understand” what a “dog running in a park” should look like. This procedure is called training a base model, and once your model is trained you can ask it to conjure anything your heart desires. Collecting that large set of images, however, is a pain, and training a model from scratch is tremendously expensive; the whole ordeal is often beyond mere mortals’ grasp. Some astute humans realized that sometimes you just need to tweak the model a little bit. To sway the model towards creating images of dobermanns rather than labradors, you just need to finetune the model a bit by feeding it a collection of dobermann pictures, a small adjustment; this procedure is much cheaper and accessible. The result is called an adapter, and the go-to platform where you can easily create, monetize, and showcase your adapters is called CivitAI.

A small segment of deviant readers may have consumed the paragraph above and contemplated the titillating prurient possibilities. Fortunately, CivitAI has implemented numerous safety features that would prevent an innocent visitor from stumbling upon lewd imagery. The homepage dissuades passersby by resembling those image-sharing websites of old that you would browse for desktop wallpapers. If a user needed to create an account on the website (e.g., to comment on a thought-provoking piece of art), the website provided users with a fine-grained filter covering PG, PG13, R, X, and XXX. Yet, despite their arduous efforts to contain the amount and intensity of obscene content on their website, CivitAI lost the ability to process payments through Visa, Mastercard, and the like.

Next week, we will find out why, and what they did in response. Hasta luego.

Adjacent Reading

• None yet, just next week’s post!

• 404. People Are Selling Kills of Marathon’s Hardest Boss on eBay

• Krebs. Netherlands Seizes 800 Servers, Arrests 2 for Aiding Cyberattacks

• The Pope talks about AI?

• CMU Startup Gray Swan featured on Forbes.

I dropped you all into the water and you kinda have just been swimming. This is good news and reflects well on your intellectual prowess. My editorial desk, however, tells me that we are ten, eleven posts deep and I haven’t yet bothered to explain what this blog is about, what we do here, why a goose, and what’s this gonzo thing about anyway.

I started lecturing about Telegram, VPNs, and deepfake pornography with little context. No warning shots. No indication that I would sneak up behind you and just push you into the midst of all of this. And you went along with it. I do owe you an explanation. Swimming is fun. But after reading this post, it will be like… snorkeling. The muddy waters will be clarified, revealing colorful fish every which way.

Last week, I poked fun at “Internet researchers.” I was riding a fever high, some nasty affliction, and kept awake and moderately lucid by a combination of Sudafed and Claritin. I closed my laptop and hit “POST”. After that was done, a kind reader reached out to me and made me realize that I came across as insensitive. My jokes landed crassly. Some of you may know immediately why while others may be completely puzzled. I amended the previous post and published an errata alongside with it.

Most importantly, though, I realized that to even explain what the issue with the previous post was, and to properly convey my present annoyance and gripes with Internet researchers, I have to begin by explaining what INTERNET RESEARCH EVEN IS, and what I’m doing here. So let’s get to it.

Intro

I am interested in all things shady on the Internet. The Internet to me is like a city, composed of netizens, neighborhoods, stores, etc. Within this city, there are all sorts of seedy, questionable things happening. All sorts of vices are peddled: drugs, pornography, escorts, counterfeit items, all sorts of illegal goods and services. Apart from vices, there are all sorts of questionable things that take place: misinformers, propagandists, astroturfing, pirates, hackers, pyramid schemes, pumps, dumps, market manipulation, fake influence, fake reputation, fake followers, bots for all purposes and grifts. I love observing and understanding all these behaviors.

Understanding these deviant behaviors is important because we often want to prevent them from happening. We typically want to prevent these behaviors from happening because they are harmful. For example, we want to prevent people from being able to launch coordinated propaganda campaigns because it compromises electoral integrity. Someone funds this research because it matches their mandate or agenda. And, these things ultimately may affect your life. Here’s how.

What

Internet research is the label I give to the investigation of behaviors that occur, well, on the Internet. A journalist reports on things happening: “there’s a deepfake epidemic”, “men are buying spyware”, “new states and countries now require you to verify your age before accessing social media or porn sites.” An academic researcher (a.k.a. a scientist) tries to answer more fundamental questions in ways that are reproducible and generalizable: “how big is this deepfake epidemic? why drives it?”, “how should we design phones knowing that your partner may plant spyware on them?”, and “what makes people abide to age verification and not bypass it?”

Things, of course, have gotten increasingly blurry. Investigative reporters often go so deep into a specific topic that they uncover scientific truths. On the other hand, researchers often work on projects that amount to a form of academic journalism. That is, they report on a thing that happens with little interest to why it happens or how prevalent it is. In short: journalists report on things happening and academics try to understand how and why things happen and whether they are different from the body of knowledge that already exists.

Why

Understanding a thing allows you to make that thing happen more or less, depending on what outcomes you care about. A journalist may report that young people are voting at lower rates, which is a problem for civic participation. A scientist may have uncovered that young people will vote at higher rates if they see other young people voting on Snapchat. Local governments, caring about civic participation, may then launch campaigns on Snapchat to drive people to the booth.

Knowing how tweaking one variable affects another variable is useful. This is called a causal relationship. Exposure to germs leads to disease. A better diet leads to healthier outcomes. These are things reminiscent of science classes like biology or physics. But they also extend to the Internet. We may uncover that people with more social media time usage have worse mental health outcomes. Or, we may uncover something more specific, such as the fact that higher consumption of short form video content leads to greater levels of anxiety.

Another type of question we care about is how much of a thing is happening. In a more traditional scientific field, this could be something like: "how much CO2 is there in the planet right now? And, how much does this change week over week, or year over year.” This is called a measurement. On the Internet, we may be interested in measuring how much of all Telegram bot activity is illegal. Or, how many illegal drugs are sold over the Internet. In fancier terms, this is measuring the prevalence of a phenomenon. Measuring a thing is useful to know whether we should do something about it.

For what and for whom

Too many reasons to name and too many stakeholders. For example, some medical research is useful so you can develop better treatments so that people suffer less or live longer. These outcomes can be useful, health insurers, pharmaceutical companies, hospitals, politicians, governments, military, society at large, and/or even the researchers themselves. Depending on what the research is, some combination of these groups may want to fund the research.

Internet research is equally complex. The Internet is broad and full of activities. There’s many entities that care about what goes on there. Broadly, though, funding comes from industry (e.g., tech companies), government/military, and others (e.g., weird philanthropic organizations). Meta may want to know how to not give teenagers body dysmorphia through their algorithms. The IRS may want to know how much of all crypto transactions is evading taxes. And, an organization like Open Society may want to reduce antisemitism in online communities.

Each of these groups may allocate some amount of funds for causes they care about. Scientists with the expertise write proposals for the work that they want to do. If selected, the scientists receive the money and do the work.

How

Formally, this is the “methodology” question. How do you do science? In the simplest form, how do you figure out that social media use worsens mental health? For the most part, and whenever possible, we apply the same conventional techniques we would apply for “regular science.” If you want to know whether a drug is effective at making your hair grow, you do a randomized controlled trial. You randomly split a group of people, you give the drug to one group, and you give a placebo (e.g., a sugar pill) to the other group. You then measure whether there was meaningful difference in hair length, density, fragility, etc.

With Internet stuff, you would ideally like to do the same. You make one group of people use a lot more social media and the other group you make do something else that counts as a good placebo. You may quickly realize though that things get a little je ne sais quoi TRICKY. Each person’s social media feed is different. Is it comparable across each person? What’s a good placebo? How long should you do this for the effects to take place? How do you measure mental health outcomes? No need to answer these questions here. You just need a taste of what these questions and problems look like.

There’s many techniques we can use to study things. The Internet is a rich medium, full of data, online chatter about myriad topics, plenty of traces of human psyche laying around to be inspected and perused. Like a TV with Animal Planet on, curious academics like myself get to observe peoples’ interactions in the wild.

Here, in the infinite scroll of the digital wetlands, we witness something truly extraordinary. A male has ventured beneath a female’s photograph––that most perilous of territories––and left his mark.

He has chosen three words. This is deliberate. Too many and he appears eager; too few and he risks invisibility among the dozens of identical displays competing for her attention in this very thread.

She has not responded. He has liked his own comment. We watch. We wait. Somewhere, in another tab, he is refreshing.

What am I, myself, currently working on

You have heard me talk a lot about Telegram, deepfake pornography, and vices. Surprise, this is because these are typically the things that I work on. I have studied people faking online influence, online fraud, markets of drugs, hacking forums, cryptocurrencies, scams, DAOs, questionable digital goods, piracy, etc. I am currently trying to figure out what these age verification laws are accomplishing and whether this TAKE IT DOWN Act thing is doing what it is supposed to do. Next up: problematic pornography consumption and what drives that behavior.

Summary

My goal with this publication is to give you an insight on the sketchy things that happen on the Internet. Not only because it is fun and good weekend dinner conversational fodder, but because the things I study are often things that will either affect you directly (e.g., being targeted by a scam) or they will affect you indirectly (e.g., through legislation designed to prevent something). For example, age verification laws are being rolled out around the US and other parts of the world. Soon (if not already), you will need to provide some sort of verification that you are old enough to access certain sites. You may find this troubling, annoying, or both. Knowing why these things exist will probably not make them any less annoying, but you might feel compelled to civically complain about them to some legislator.

This post was a quick overview. Broad vague strokes devoid of specificity. Send me an email if there’s a topic that interests you. Or, even easier, drop a comment. Why not? I might just write the next week’s episode fully based on what you ask for.

Adjacent Reading

• KDNuggets. Causation or Correlation: Explaining Hill Criteria using xkcd

• WIRED. You Can Get Some of Your Nudes Removed From the Internet Under a New Law (guest appearance from Dr. Gonzo)

• WIRED. Your iPhone Gets Stolen. Then the Hacking Begins.

• 404. How Deepfakes Tore a High School Apart.

• 404. Streamer Realtime Deepfakes Himself into Mr. Beast, Says He Loves ‘Touching Little Boys’.

• Cool sites: radio.kwsx.online ; synesthesia.club ; theshfl.com

Back in the day, I was a script kiddie. I would frequently send my friend F (who reads my s’tack; thanks!) malware through MSN messenger. One time I got his printer to waste some paper from the comfort of my house.

When I was in undergrad, I did a consulting internship at a big consulting firm. Mr. Robot season 1 aired that year, rekindling all my hacking dreams. So, I got the partner in charge of the “cyber” division to take me around for a tour and show me their labs, tools, and so on. Unfortunately, I did not find guys in hoodies typing quickly on screens. Instead, I saw a conference room with 4 guys projecting an Excel spreadsheet and doing PCI compliance. Essentially, if a company like Comcast wanted to create a page on their website that had a credit card payment widget, these guys had to make sure that you had a lock icon when visiting that page.

This was certainly not the cybersecurity ninja 1337 h4x0r vibe I was expecting. I was hoping for a bit more cargo shorts, sandals, uncovered legs and white socks stretched up mid-shin. Instead, much to my dismay, one of these Excel enjoyers would make partner and advise major companies on cybersecurity strategy; a tragic prospect that may explain why every couple of months there are major credential leaks.

It is a generally understood truth that the most legit security researchers are not wearing suits, but rather people with anime pfps and questionable (to put it kindly) social behaviors. These are people who are breaking stuff with a passion. I once worked with a kid (a bonafide youngster) who just arrived from China to Penn State. He was a top 10 CTF(a type of hacking competition) player in China. The professor hosting him had to repeatedly remind him that he could not go around probing Penn State systems or the FBI would swiftly become involved.

Another generally understood truth of the computer security field is that some of the people that have pushed the field the farthest often engage in shady and illicit activities. This is what the field has come to call “black hat” hackers, hence the name of one of the biggest computer security conferences. However, Black Hat (the conference), despite its name, failed to fully capture (and appeal to) the rogue-anarcho-cypherphunk types. So, alongside came DEFCON, a conference that built its early credibility from the fact that it attracted, tolerated, and (kinda used to) embrace criminals and miscreants from around the world. You could pay cash at the door for increased anonymity, you were advised to bring a burner phone, and you should definitely not connect to the wi-fi.

Throughout the years, Black Hat became the sanitized corporate version and DEFCON its grimy and nefarious twin. The cybersecurity community understands you need both, the sperries and the tevas. Or, to put it more scientifically, you need people who have an emic command of adversarial culture, rather than just an etic one, like PCI compliance auditors.

Somehow, the same mindset has not always extended to other areas of computer security, like research on online deviant behavior. To understand certain phenomena, you need immersion. Ethnography, after all, is the study from the point of view of the subject. A software security researcher does this context switch without noticing. When they practice an “adversarial mindset” what they are really doing is roleplaying what an attacker would do. They put on their black hat and carry out what is ultimately an empathetic exercise.

This means that if you work on, say, manosphere radicalization1, you need to understand what mogging is, along with femoids, sigmas, chads, gymcels, etc. But you need to understand them not merely through definitions but through their “cultural” context: what you are gravely missing is emic competence. Knowing that a “femoid” is a dehumanizing term for women is etic. But understanding the mix of contempt, grievance, and in-group signaling that comes with the term is emic. Thinking that pepe is just a “symbol” of the alt-right is etic (and borderline ignorant). If you see pepes––and memes, more broadly––as artifacts that store communities’ memories (especially in a website where threads and posts are ephemeral), then you can see them a bit more as a form of neo-oral tradition. Neat, huh?

Some of the most meaningful and rich research starts with deep qualitative understanding. I know this statement will have my colleagues who are more attuned to the humanities snapping their fingers. What they may not want to hear is that they are often missing emic-ness.

My best guess for why researchers working on questionable online communities fail to cultivate an emic understanding is because they seem oblivious to the fact that they are, still, studying humans. I can understand this, especially because some people online are engaging in heinous activities. In an earlier version of this post, I poked fun at some computer science papers that do research on sketchy online activities based on how they write about these communities and how they describe their ethical considerations. The reason I did this is because some researchers write as if they want maximal distance between them and their study subjects.

The consequence of this, however, is that researchers end up treating these online communities like radioactive substances to be handled with the smallest of exposures––long metal prongs and a hazmat suit. They look at some 4chan boards as biohazardous, their fumes almost escaping the screen. To be emic would expose you to a fungus that threatens to corrupt your psyche.

I often use 4chan as an example because it’s recognizable, has been around for a while, and yes, there’s a ton of mean, offensive, and malicious behaviors. But that website is not a monolith, a hive mind, a unified body with the same agenda. It’s composed of boards with slightly different cultures and norms, seeking different things. Behind all the name calling, slurs, offensive language/actions there are people, behaviors, and beliefs. With the right approach, it’s a place that provides you with a unique vantage point to observe what people do when given a podium and anonymity online. Except that, that vantage point is less so far and distant, and moreso in the trenches.

Ultimately, the case for an emic perspective when researching sketchy communities online––beyond the ability to understand the lingo, the context, the memes––is that you can quite simply do better science. In an earlier post, we talked about the ideologues who are preserving deepfake porn models. By understanding that some people are not money-motivated but ideologically-motivated, then you know that threatening Visa and Mastercard, in your quest to prevent people from creating deepfake pornography, will only get you so far. Without an emic lens, it is hard to not think that the bacteria people who are part of online cesspools communities (the subjects of observation) are bad, malevolent entities for which we need to develop antibiotics and chemotherapy: swift, deadly, and merciless eradication.

This is not to excuse anybody’s behavior online. Understanding does not mean condoning. Humanization does not mean removing the blame nor the condemnation of their actions. Empathy is just the quest of abandoning our preconceptions and trying to truly see their viewpoint, whoever “they” or “theirs” refers to. It is important to note that this often not easy and sometimes not even feasible. Some online communities require a serious amount of safeguards for your mental health’s sake (think, for example, people who produce and share CSAM online). In many other communities, however, an emic understanding is paramount.

This blend of emic and etic perspectives is something that cybersecurity folks have understood well. You need people that can think like hackers and criminals, or even better, people who were criminal themselves (your emic). And you need your etic people, the ones that have all those acronyms next to their Linkedin names. ‘Til this day, DEFCON begins right after Black Hat ends––Vegas, Aug 6th.––yin and yang.

Adjacent Reading

While I haven’t read all 3 of these books, people generally seem to believe that these people blend emic and etic perspectives well.

• Coleman. our weirdness is free.

• Phillips. This is why we can’t have nice things.

• Coleman. Hacker, Hoaxer, Whistleblower, Spy.

• Greenberg. THIS MACHINE KILLS SECRETS: How WikiLeakers, Cypherpunks and Hacktivists Aim to Free the World’s Information.

Errata

• 05/26/26. I modified paragraphs 11-14. The earlier version of this post didn’t account well for the wide range of online communities that exist. So, when read from different lenses, I came across as an insensitive ****** to researchers who have to look at terrible stuff, CSAM and the like. Hopefully, this is now fixed.

What do I tell my readers? Will they understand? The pressure, the deadlines…

Complete blockage. Dire constipation of thoughts. No funny openers, no hooks, no attention grabs. I stared at the bottom of the screen for answers: the orange splat is always there for those who come for help. If I prayed the right words it could give me the answers. Some answers at least, or some semblance of answers. The clock continued to tick. My subscribers were waiting.

Would they know? Were they actually reading? Surely, they are not reading each line I write.

I may just be able to sneak this past them, inconspicuous, but dirty. This is just what they do at the Link: artificial words made up in some datacenter somewhere, completely devoid of soul. It’s too early to sacrifice their trust. But is it? A swarm of agents could get me that sponsorship, that affiliate deal. Yes, maybe a few external advisory seats and recurring consulting gigs, expert witness, talking head, pundit. After I close the deal on my new apartment in some metropolitan city paid in hard cold cash directly from Substack HQ. Would I care? Let my agents manage my views and engagement and likes and follows. Passive income.

Certainly, this was all an attractive proposition. But I would have to do so carefully. One wrong move and the jig is up, the whole house of cards would come crashing down. They would uncover my orchestrated scheme of agents designed to maximally extract attention through words like “seedy” and “gonzo” and some secret truth about the Internet’s underbelly that only I knew and no one else. They would tar me and feather me and I would be relegated to a life of fundraising from companies that know no better: a69z, big tree capital, Ychromosome.

No, I must be careful and start with something simple, a summary of sorts…

Hello and happy Friday! It was hard to squeeze some time to write this week because I was working on my first ever journalistic assignment: read my piece––for Tech Policy Press––about the TAKE IT DOWN Act here. Like, comment, share, and subscribe to them if you are interested in, you guessed it, tech policy. Great people over there.

Coincidentally, I’ve been tied up this week working on revisions for the deepfake pornography study which I discuss in the piece above. The deadline is Saturday morning and I only have a few dangerously asinine reviewers’ comments left to tackle. Because of this, and due to the fact that we have just crossed 10 posts, I figured it would be a good time for a recap of everything we have tackled so far.

See you next week!

#001 // The enforcement of vices on the Internet. (Mar 6, 2026)

Drawing on research documenting deepfake pornography services openly listed on Fiverr, the post uses the history of online drug markets as a framework for thinking about how vice enforcement plays out on the internet. Each crackdown tends to push activity to a more fragmented, harder-to-police layer rather than eliminating it. The conclusion: enforcement creates friction but rarely eradication, and the same arc that played out with drugs is likely to repeat with AI-generated nonconsensual imagery.

#003 // The ideologues preserving deepfake porn. (Mar 13, 2026)

Financial levers––cutting off payment processors, defunding platforms––are the go-to enforcement tool for curbing deepfake pornography, and they work on purely money-motivated actors. But a significant subset of the community is ideologically motivated, treating censorship resistance as a value in itself, and will maintain and distribute the tools for free regardless of financial consequences. The post uses the piracy movement as a parallel: ideology, not profit, is what keeps the hardest-to-kill corners of the internet alive.

🏆 Winner: Highest open rate.

#002 // The performance was always the product. (Mar 20, 2026)

AI influencers and fake OnlyFans personas are being treated as a new phenomenon, but “fake” accounts built to sell adult content were already thriving well before generative AI––they just required more human labor. The post traces cases like AyannaSoBlack, an AI-generated persona that went viral, to show that the fraud is less novel than the production cost dropping to near-zero. The real shift AI brings is scale and deniability, not invention.

#004 // The Telegram Ecosystem, Part I. (Mar 27, 2026)

Telegram is not the dark web––it’s something stranger, larger, and far harder to regulate, structured less like a platform and more like a city with no zoning laws. The post breaks down Telegram’s unique architecture (channels, groups, bots, mini-apps) and why it has become the go-to infrastructure for both legitimate communities and underground markets. The sheer scale and permissiveness make it a genuinely novel governance problem.

🏆 Winner: Most views, most likes, most comments.

#005 // AI can now find zero-days. (Apr 3, 2026)

AI researcher Nicholas Carlini demonstrated live that Claude can find zero-day vulnerabilities in major codebases like the Linux kernel, sparking widespread alarm. The post pushes back on the doomsday narrative, arguing the balance between attackers and defenders is more symmetric than people think. For most threat actors, the real bottleneck was never finding vulnerabilities––it was the legal risk of exploiting them.

#006 // The Telegram Ecosystem, Part II. (Apr 10, 2026)

Continuing the series, this installment focuses on the methodological challenge of studying Telegram’s darker corners: advertisements for illicit content don’t equal verified supply, and treating them as equivalent systematically overstates the problem. It draws on the RentAHitman.com analogy––a seemingly alarming site that, on inspection, is mostly bait––to argue that sustained fieldwork is necessary to distinguish scammers from genuine bad actors. The policy implication is that inflated threat estimates lead to poorly calibrated responses.

#007 // The dark web is dying. (Apr 17, 2026)

The dark web (Tor-based hidden services) was once the premier home of illicit online activity, but it has steadily lost ground to more convenient alternatives like Telegram. The post explains how Tor works, traces the evolution of dark web markets from their early days through repeated law enforcement takedowns, and argues that inconvenience––not policing––is what’s truly killing it. The underground didn’t disappear; it just moved somewhere easier to use.

#008 // Chameleon Channels. (Apr 24, 2026)

Social media accounts are a valuable commodity with a booming underground economy of buying, selling, and repurposing them––sometimes pivoting overnight from, say, a COVID vaccine info account to a partisan political outlet. The post explores how platforms are poorly equipped to handle this identity-switching, since followers have no say when an account they trusted transforms into something entirely different. The underlying driver is simple economics: captured audiences are worth real money.

#009 // Do you need a VPN? (May 1, 2026)

Commercial VPNs do exactly one thing: route your traffic through their servers so intermediaries can’t see what you’re browsing, which also lets you spoof your location. The post warns against the affiliate-link-poisoned “best VPN” comparison guides littering the internet, recommending Mullvad for genuine privacy needs, iCloud Private Relay for Apple users, and Tor for anyone who wants true anonymity. Free VPNs are a hard no––their only viable business model is selling your data.

#010 // The takedown scam (May 8, 2026)

The post recounts receiving a suspiciously formal DMCA takedown notice at Redoux (the author’s fragrance brand), demanding a backlink credit to “Ice Baths Australia” under threat of legal action––complete with a fabricated law firm and fake lawyer. Investigation revealed it was an SEO scam: bad actors send fake copyright notices to small website owners who, out of fear, comply and add the requested backlink, boosting the scammer’s search ranking. It’s a clever, low-risk exploit of how few people actually know DMCA procedures or bother to verify the sender.

???

• Big week for scams. Scam Inc. a podcast by The Economist won a Peabody Award

• Almost simultaneously, “Scam Center Simulator” premiered on Steam.

• And this undercover procurement of scam software was shared by 404 folks.

Every now and then I come across a nifty little scam that makes me smile. Some of them are fascinating demonstrations that human nature is full of ingenuity, especially when it comes to making money, and even more so online.

I co-own Redoux, a fragrance brand. I manage the operational portion of it, which these days primarily involves the tech stack: the website and business email. As such, I’m the recipient of numerous daily emails corresponding to taxes, bills, trade shows, and charlatans. Rude marketers. A barrage of ChatGPT pitches. Worse, many of them Copilot. Of course, I also get the occasional ruffian. We’ve had numerous attempts––including a few successful ones––of people buying stuff from us and then issuing a chargeback.

Recently, I checked my inbox and found this bastard staring back at me:

Redoux has had some run-ins with the law. In fact, we once got something akin to a cease and desist for matters regarding copyright. An entirely unfortunate misunderstanding that only was cleared after an in-person dialogue with a suit. The mere prospect still managed to positively shock us for a few days. Pointing a lawsuit to a small business is like pointing a Magnum 44 to a small animal. And in a city like New York, no quarter is given; they will have you for breakfast.

DMCAs, copyright, and take downs had been in my mind, especially the morning I found that email. I had been (and continue to be) working on matters that concern the removal of nonconsensual online content, either through the newly passed TAKE IT DOWN Act or more ancient mechanisms like the DMCA. That same week, I was negotiating a time slot with a firm that helped adult content creators remove content that was pirated using DMCA. My first instinct was thus to take the email with seriousness.

The blog they called out on the email was made by an SEO marketing firm we had hired back in the day to improve Redoux’s search ranking. Their suggestion for us was to write blogs about general matters of interest in our niche (i.e., whether you should use bottled soap or a bar). The idea is that you want to fill in the information gap for search terms that have decent monthly traction. If people are looking at authoritative advice that would finally settle the matter in the longstanding rivalry between bars and bottles, then perhaps they may just linger long enough in your website such that they might buy something on their way out.

Chez Redoux someone was put in charge of cranking out those blogs. This is pre-GPT era, so I believe that someone had an English major. My first suspicion was that they had gotten lazy. They copy-pasted some text, perhaps an image, with no attribution, all because they needed to attend some scene-y design event in some forsaken place in Brooklyn. Their lack of citation manners was now costing me a DMCA TAKEDOWN. I was annoyed. But it was odd… a reverse image search did not reveal any obvious owner. The purported website looked wack and with no trace of the reported image.

The lawyers’ website was equally odd, like misplaced plaster wall in a motel room. Dubious provenance. Phoenix address, 4th floor? Not possible. Arizona is flat, sprawled, and hot. Building upwards only gets you closer to the sun, expand towards the ocean at all costs. The house of cards was falling down quickly. No semblance of an office where the address pointed to. No lawyer by the name of Noah Carter. All bogus images, generated by some computer somewhere.

The reason? Also just SEO. To rank higher on search you not only need to produce content that fills keyword voids. You need backlinks: higher ranking websites who deemed you worthy of a a hyperlink. The NY Times wrote about you? Must mean you are legit. These reputable backlinks tell Mr. Google to place you higher. The first page is an absolute must, the top spot is the most coveted. But not everybody will do this in an honest God-fearing way. Players play. A poor website administrator somewhere read the takedown notice and complied, adding a link at Noah Carter’s behest.

The scam itself is rather innocuous. Sure, you get a rude jolt in the AM while the caffeine is still penetrating your nervous system. But no bank wire was asked, no cryptocurrency demanded, no fake relative seemingly in inevitable danger––jail, car accident. Those ones in the wrong ears of a senior citizen could cause serious mental damage; exploding veins, heart arrest to follow. Vicious stuff. This backlink scheme is prank territory, mostly harmless. Today, “ice baths in australia” returns the above website within the first 8 results. The scheme might be working. Unfortunately for them, I won’t be falling for it today. Noah Carter, wherever you are, better luck next time.

Adjacent Reading

• Funnily relevant: Princeton privacy study halts GDPR/CCPA research over ethics concerns and industry blowback.

  • Hacker News thread.

• Relevant for today’s post: “Technical wizardry used to combat illegal immigration also funnels the personal data and whereabouts of U.S. citizens to federal agents.”

• Why university domains are serving porn websites.

• Science. “The Misinformation Accelerator”

• A ton of screenshots from a celebrity’s phone were floating online.

• Chinese Hackers Spied On Cuban Embassy As US Prepared Blockade. My guess is that they did not use Mythos.

Today’s post is special. Gonzo Lab’s very first commissioned piece. A long time reader timidly posed the question “what VPN do you recommend?” Over here, we pride ourselves in efficient scientific communication. I figured I’d make a quick guide because: the reality of the matter is that choosing a VPN sucks.

Online advice is all over the place. Everything feels like a psyops. There is no way to verify any of their security claims. They might all be owned by the CIA or Peter Thiel or Russia. What does “RAM-only” or “no logs” policy mean. Do you even NEED a VPN???

If your attention span has not yet been obliterated by short-form video content, let me walk you through how you can (try to) prevent Palantir’s surveillance apparatus from inferring what color underwear you are wearing at this exact moment. If your brain has already decomposed, skip to the summary.

What VPNs Do

Commercial VPNs do only one thing: IF the VPN is properly implemented it will route your traffic through their servers so the destination website and intermediaries don’t see what you are browsing. That’s it. Incidentally, because they route your traffic through their servers, you get to spoof your location and access a website as if you are in a different state or country, bypassing geolocation restrictions. Because of this functionality, they can also help bypass censorship both implemented by the destination website or by an Internet Service Provider (ISP).

Does it protect you against malware? No. Does it prevent your data from being leaked online? No. Does it make you anonymous? No.

Pop quiz, what’s the best way to browse the web privately? Answer.

The Technical Bit

When you visit Gonzo Labs, your request travels through your router, your ISP (Verizon, Comcast, whoever), and a few more invisible middlemen before reaching its destination. Think of it like sending a letter through regular mail: each stop reads the address on the envelope.

All intermediaries know you have great taste––since you sent a letter to Substack demanding the epitome of academic journalism on the seedy corners of the Internet. Thanks to modern security standards these pesky intermediaries may not know exactly what you’ve been reading on Substack. But they know you sent a letter to Substack. Eventually, when you leave Substack and open TikTok, they will see that too.

What a VPN does is to act as the recipient of your letter. So instead of your router and ISP seeing that you are contacting Google, Gonzo Labs, Substack, TikTok, etc. they will see you are sending a letter to BigVPN Corp and that BigVPN Corp is replying back. The recipient website will also only see that BigVPN Corp is sending them the letter and not you.

What a VPN Affords You

The main reason you would care about this is if you were afraid that your ISP may be selling your data, or that the Wi-Fi provider at the coffee shop you are working from is selling your data, or that the recipient website is selling your data. These are valid concerns. Advertisers buy your data and know that if you are checking the price of Bitcoin they can advertise to you Finasteride. More concerningly, there is also a big surveillance apparatus (as you can see from the news above) that gobbles your digital crumbs and is ready to weaponize it against you if needed. A VPN could, in theory, make their lives harder.

However, your VPN could also make <their> lives easier. You are essentially entrusting all your browsing traffic to your VPN provider. So if the P in VPN stands for Palantir or Peter Thiel rather than “private” then you are cooked, as the kids would say. By the way, this is part of the reason why schools or companies require you to use a VPN on the machines they issue you: so they can see everything you do.

A big part of choosing a VPN thus comes to a matter of trust. This is also why VPN providers make a fuss about not keeping any logs, for example. The idea is that, if they keep nothing, they have nothing to give to the FBI. The other thing they love to flaunt is the jurisdiction of where they are incorporated. NordVPN claims that Panama is lawless; if it’s good enough for shell companies it should be good enough for VPNs, no? ProtonVPN leverages Swiss ethos. After all, if Switzerland can keep sketchy money in their banks, surely they will keep your sketchy data safe too. Mullvad shields itself under nordic credibility. They also take private cryptocurrencies as payment which is nice to not leave a paper trail.

Unfortunately, a lot of claims from VPNs are hard to verify. There are some external things that you can verify (i.e., whether their implementation is complete ass). For most claims, you just have to take their word for it. We don’t really know whether they keep logs or not or if they feed data directly to Alex Karp’s OpenClaw server. The things that are easily testable are latency and the ability to spoof your location (i.e., performance); the privacy is not.

VPN Advice Floating Online

It came to my attention, when researching VPNs for this blog post, that there’s a lot of trickstery and underhandedness that happens online. The business of VPNs is entirely similar to the business of bogus health supplements: it’s the playground of phony growth marketers.

Most people who spent some time researching VPNs in the 2010s likely came across this famous VPN comparison chart (shown above). This list made a generation of people go for NordVPN. Eventually, the person who created this list stopped updating it. Someone saw an opportunity and hijacked the original URL to now point to a completely phony VPN recommendation website. Similarly, the search “Reddit VPN comparison chart” has been hijacked by threads trying to get you to click their affiliate links, like this one, this one, and this one. I’m showing you these examples so that you are SUSPICIOUS of them, not as an endorsement.

You should be skeptical of these lists because the moment that they are more motivated by money than by giving you actual advice, they put you––dear consumer––at risk. I’m not saying affiliate links = bad. However, they can create misaligned incentives which are exacerbated in industries that are already rife with deceit. In this research, I found this initiative by Consumer Reports very cool. They hired legit academics to do legit research and bring some transparency to the ecosystem (DMs open btw). Wirecutter also has a roundup (although they do make money with affiliate links too). Both CR and Wirecutter recommend Mullvad. Mullvad has not paid me for this review.

Summary

• If you want proper privacy and anonymity. Use Tor.

• If you are tech-savvy, spin up remote virtual machines and make your own VPN.

• If you have Apple devices, iCloud Private Relay (better explained here) is an extremely underrated feature. It’s like an always-on VPN when using Safari.

• Under no circumstances use a free VPN if you want privacy. The only way in which a free VPN makes economic sense is if they sell your data.

• If you want to change your location to access geo-blocked content. Use a paid VPN.

• If you want some mild privacy guarantees:

  • Like hiding from your ISP that you are pirating something (so they don’t send you a scary letter). Use a paid VPN.

• If you use public Wi-Fis a lot. Use a paid VPN.

• If you want to escape advertisers. Use a paid VPN.

  • However, don’t forget that if you use a VPN and log into a website, the website still knows who you are and they can still sell you out :-)

• If you really need a paid VPN, go for Mullvad.

  • They take cold cash and Monero which is nice for privacy.

  • Their client is open source which is nice.

  • And their website’s design seems like it’s made by a company with more technical people than marketers, which bodes well.

Adjacent Reading

• Website: https://vpnalyzer.org/

• Ramesh et al. “All of them claim to be the best”: Multi-perspective study of VPN users and VPN providers

• Ramesh et al. VPNalyzer: Systematic Investigation of the VPN Ecosystem

• Freedom of the Press. An in-depth guide to choosing a VPN

• Channel 5 with Andrew Callaghan released “The Gooniversary”, a story about the radicalization of mas******ion.

• A soldier was charged for using classified data to bet on Polymarket. Hey, if members of Congress can bet on the stock market, why not?

• Wait, Congress members are also insider trading on prediction markets?

President Obama was dubbed the first “social media president.” Obama was the first to have @POTUS on Twitter. Today, the White House not only has a Snapchat but also regularly posts AI slop on TikTok. But, what happens to these accounts after a presidential transition? While this question likely doesn’t impede your sleep, the National Archives and the Office of Digital Strategy have a formal procedure to handle the transition. In a nutshell, if you were following President Biden you found yourself following President Trump after. This is what POTUS means, much to peoples’ surprise. While jarring, this was consensual.

A much more jarring example took place in Paraguay. During the COVID era, many people were following social media accounts that transmitted information on vaccinations. This was one of the primary ways, for example, to find out about leftover vaccines. One such account was called InfoVacunatePY, which had ~22,000 followers on Twitter. After COVID was gone-zo, this account was no longer needed. But, some entrepreneurial mind realized that letting these 22k followers go to waste would be stupid. Why not redirect their attention to something new, something fresh? Overnight, InfoVacunatePY turned into HonorColoradoOk, an account affiliated with Paraguay’s leading conservative political party. This happened just as Paraguay headed into an election year.

It is clear the person that repurposed the account had the incentives to do it: they either made money or gained political goodwill, or both. What is clear is that the people who signed up for COVID vaccination updates did not expect the account to later become one associated with a political party. If this was not noticed by users and later reported by the media, this case would’ve gone unnoticed, as many other accounts do.

It should come to no surprise that there’s a booming underground economy dedicated to all things social media, including the sale of accounts. People sell rare usernames like they used to sell domains. Bots. Shoutouts. Tools to automate your growth. Large accounts, small accounts, verified accounts, accounts based in the US, in India, Brazil, old accounts; various niches, demographics, age groups. What do you need?

Having an online audience is valuable. There’s a reason we are all writing Substacks and LinkedIn posts. As reported by New York mag, a substacker and an influencer were each clearing ~300k/yr. You may wonder how it is possible that many PhD-level computer science jobs pay less than being cute online. The math is quite simple. As an influencer, you amass people who want to consume. As a brand, you want to tap that big fat audience. If 10,000 people buy your soap thanks to an influencer’s post, this is $160k revenue. If you have a PhD, you will quickly realize there’s all these metrics (e.g., follower counts, demographics, engagement, etc.) that you want to optimize to increase your revenue prospects. Intuitively, these audience metrics are what drive the value of the account that commands them.

One of the key drawbacks from influencer-type of monetization deals is that your face/persona is the selling point. There is only one Andrew Huberman, one brand, one main monetization point. On the other hand, you can have many faceless accounts. For these cases, there are also other monetization pathways that make more sense. In longer-form content platforms (e.g., YouTube and podcasts), you optimize for ads. You create content that will gather views and make people stay through the ads. The person who created the OG Lo-Fi girl probably owns an island by now, paid with ad dollars.

The market for social media accounts, until very recently, was primarily composed of faceless accounts (synthetic influencers have begun to change this, but that’s a post for another day). These would be for example, channel dedicated to narrating sports facts ($1,500), AI slop ($680), or music tracks ($4,000). Not only are you looking for content that is long and that people typically have in the background, but you are also looking for automation. Your goal is to create passive income machines: channels that run on an unholy amalgamation of web scrapers, ChatGPT, Nano Banana, and ElevenLabs.

But what about short-form content? YouTube Shorts purportedly pays between $30-70 for a million views. For an average watch time of 3.5 seconds per view, the market rate for ~1000 hours of human attention is dystopically low. This math only makes sense if you are running a content creation sweat shop: thousands of people across warehouses churning out new permutations of cat videos with the hope of hitting your dopamine receptors just long enough that you decide to give it a like, perhaps even a comment.

Luckily for the taskmasters, new forms of monetization have emerged for short- and long-form content platforms. Let’s say you are a memes account. You could get paid to post political content in the guise of a meme, like Iran’s LEGO videos (meme information warfare as they call it). You could advertise an online gambling platform (e.g., the accounts with the “1win” logo). You could launch a cryptocurrency. You could advertise some pharma products, like peptides or supplements. Or, you could sell your account to someone who is interested in doing any of the above.

In the Paraguay case, you know that most people following the account with vaccination updates for Paraguayans are people who could vote in the upcoming elections. Getting your message to these 22,000 people, at the very least, is worth as much as a digital billboard. For more universal types of goods and services like gambling and adult content you do not need to have a carefully manicured audience; you just need middle-aged male attention. You can get that by creating hundreds of accounts that post thirst traps and sports content. When you reach a nice follower size, you cash out. Someone takes this account and changes the content. For the account followers, YouTube/IG/TikTok will not notify you, your feed will just change.

Not all people who are buying accounts are using them for societally-detrimental purposes. Across the accounts that I looked at for our paper on repurposed accounts, there were many cases of vloggers, musicians, wannabe influencers, etc. Some people (and even companies) simply want the veneer of clout afforded by a high follower count. This works because a lot of people still naively think many followers/likes/comments = good. The people who haven’t yet subscribed to Gonzo Labs will likely reconsider their choice once we are in the 4 to 5 digit territory. Such is human nature.

There are no mitigations, only incentives. For laborers in the developing world, there are just enough monetization pathways that churning out faceless content beats a minimum wage––better to harvest attention than corn or whatever. For content platforms, indifference is a sound business strategy: more content, more watch time, more ads served, more revenue. Doesn’t matter who made it or why. And for viewers, the incentive is the scroll itself: an endless, frictionless supply of content optimized to hold attention just long enough. In that state of scrolling stupor the changes are imperceptible by design. As for the content, nobody asks where it comes from or why it exists. Like cheap electronics and fashion, most of the time we don’t wanna know.

All the above and more is available in our paper that’s set to appear at USENIX Security 2026: “Chameleon Channels: Measuring YouTube Accounts Repurposed for Deception and Profit.”

Adjacent Reading

• Click farms

• Grohman et al. “Click farm platforms: An updating of informal work in Brazil and Colombia”

• Photographer steps inside Vietnam’s shadowy ‘click farms’

• The world of Asian influencer farms: A dystopian reality

• Inside the Daily Life of a Live Streaming Star in China | NY Times Op-Docs

• WTF is clipping? The low-lift creator strategy grabbing advertisers’ attention

• WTF is behind the explosion of faceless creators?

• Elmas et al. Misleading Repurposing on Twitter

Recent Happenings

• Vegetable and fruit AI slop emerged as a potential replacement to Italian brainrot. Based on how often they appear on my feed, they might be losing the race. It seems like we will not be getting vegetable slop theme parties.

• Iran has been actively engaging in meme warfare. Surprisingly, they went for LEGOs instead of Studio Ghibli. Either way, the White House is definitely not unprepared for this.

• A huge marketplace for scam services and potentially even human trafficking (as per WIRED) seems to be operating with no repercussions on Telegram. The market is easy to find, although it’s hard to verify what products actually get traction. We will continue to monitor the situation.

Now, back to our regularly scheduled programming.

I’ve been referencing Tor and the dArK w3b throughout my posts without ever dedicating a piece to it. This is that piece. What it is, who made it, what’s on it, and why it’s fading.

When you use the Internet everyday, you are using the “surface web” or the “clear web.” These are essentially all the websites that are indexed and discoverable by Google. Throughout the years, websites like Facebook and Instagram began to host more and more Internet content. Companies also began hosting their own Intranet networks. This content is typically not easily accessed externally. If you want to find an Instagram post, Google will not help you much. While I don’t like the name, these portions of the Internet which are not externally accessible, have been dubbed the “deep web.” Both the surface web and the deep web operate largely with the same underlying protocols.

The “dark web” is slightly different. People take the “dark web” to generally mean Tor hidden services (or Onion services) which are accessible through a Tor browser. Tor stands for The Onion Router because the onion analogy inspired the design of the protocol. Just like other Internet protocols, Tor came out of a military research lab.

When I say protocol, what I mean is a set of rules. Safari, Firefox, Chrome, etc. all follow a set of rules that allow them to get the content of websites. When you visit a website, it’s like you are sending a letter. You put the letter into the mail with a given address. The post office routes this letter to the destination. It’s delivered. They write a letter back with their response. The post office routes the letter… you get the point. This is how you get a website. We have protocols to send letters, packages, freight, etc. Other countries follow similar rules and so we are able to send mail internationally as well. We have protocols that let us transmit information both in the physical and the digital world.

How Tor works, roughly

The problem is that if you are doing something that requires privacy, such as sharing private state secrets or accessing nsfw pony cartoons, you may not want the post office to know that you are sending a letter to “ponyland xxx.” Tor to the rescue. To follow the Tor protocol, what you would do is:

1. Put your letter addressed to ponyland xxx in an envelope addressed to Alice (she’ll be the last recipient before ponyland).

2. Then you will put your envelope addressed to Alice in an envelope addressed to Bob.

3. And then you put everything into an outermost envelope addressed to Charlie (the first stop).

So, your letter is first sent to Charlie. Charlie knows you sent them a letter and Charlie knows to then send the letter to Bob. Bob opens the letter from Charlie and sees that it is a letter for Alice. Alice opens the letter and sees that it is a letter for ponyland xxx and sends it. In this way:

1. Charlie knows you sent them a letter and that the next destination was Bob.

2. Bob knows they received a letter from Charlie and that the next destination was Alice.

3. Alice knows they received a letter from Bob and that the next destination was ponyland xxx.

4. Ponyland only knows that Alice sent them a letter, and they return a response.

No single relay knows both who you are and where the letter is going. Charlie knows you but not the destination, Alice knows the destination but not you, Bob is in the middle and knows neither end. The response travels back through the same chain in reverse.

This idea of envelopes that you peel out is what makes this The Onion Routing protocol. Even though I left a lot of technicalities, what you need to remember is that: done well, Tor is the most private way we have to browse the Internet.

How is this different from a VPN? A VPN is like using FedEx instead of USPS––FedEx doesn’t tell the post office where your letter is going, but FedEx itself knows everything: who you are and where you’re sending mail. With Tor, no single party has both pieces.

You can use Tor to browse regular websites, as well as “hidden services.” Hidden services are websites that hide their own address the same way you hide yours. Neither side knows where the other actually is. Hidden services are what you use if you’re setting up sketchy websites so the police don’t come after you as an operator. And you use a Tor browser to access them so you evade the law as a consumer.

What goes on the Tor network, anyway

Tor’s privacy affordances made it an ideal candidate to host sites where illegal happenings took place. After a series of failed attempts on the clear web, more tech savvy users saw Tor as a viable alternative to evade law enforcement. And then, with Bitcoin’s arrival, people saw that they had a way to privately host websites, privately navigate to said websites, and a (seemingly) private way to transact online: so people began selling drugs.

If Telegram is the Wallet City of Kowloon, Tor hidden services are underground tunnels, like the ones used in the Vietnam War: large networks of websites that are hard to find, some of which have links to others, but a lot which are isolated from each other.

Whenever I used to describe my PhD work to people, more often than not people thought of weapons and hitmen, sometimes drugs. In reality, it’s just mostly about drugs. I’d venture and guess that drugs are the primary use case, followed by illegal digital goods/services (e.g., stolen data, hacking or spam services, etc.), and lastly unregulated media, mainly porn and gore.

This isn’t to say other stuff doesn’t exist; weapons and CSAM are both there. But a few things keep the dark web from becoming the lawless free-for-all people imagine. There’s the technical hump, first to access the sites and then to pay for things. There’s the clear web alternative. Most people don’t actually need Tor to find unregulated porn, fake IDs, or counterfeit goods. There’s the police, who run a major bust every couple of years and pursue the more extreme stuff aggressively enough that most sites have rules against it (better to piss off one federal agency than five). And there’s the common fear that the sketchy website you’re trying to buy drugs from is also going to give your computer a ~virus~.

It is important to mention that the Tor network has important benign uses. It allows people that really need privacy (e.g., dissidents, journalists in conflictive zones, etc.) a way to navigate the Internet. It also allows people to circumvent censorship. Unfortunately, while Tor helps hide the content you are browsing from prying eyes like your Internet service provider, it does not hide the fact that you are using Tor. The fact that “you have something to hide” in itself can attract scrutiny from law enforcement. Also, a lot of the Internet becomes less usable when you use privacy-preserving technologies; you may get infinite CAPTCHAs just trying to reach Google. This has ultimately prevented this tool from flourishing.

All of this has resulted in a network that––all things considered––is not very big. A recent paper found only ~25k unique sites. A quick skim of /r/onions reveals little activity and the same handful of websites that are worth visiting. Circa 2015 era, there were about 20ish darkweb markets operating at a given point in time. By the 2020s, you’d be pressed to find 5.

In 2026, the dark web seems to be dying.

Tor requires a dedicated browser that doesn’t run on your phone, loads slowly, and breaks on half the sites you try to reach. Many online guides will tell you to not even open it unless you are using a separate operating system. Paying for things in a private way has also gotten harder, even with cryptocurrencies, because so many crypto exchanges have implemented rules that require you to verify your identity. All of this makes it a major hassle. In a world that is increasingly mobile-first and short-attention-spanned, VPNs and Telegram are winning.

Usability alone doesn’t explain it, though. In recent projects involving the misuse of AI, both to generate deepfake pornography and to generate malware, I had little luck finding leads in the dark web. It turns out the clear web was permissive enough. The enforcement pressure that pushed illicit activity onto Tor in the first place has slackened, or at least become inconsistent enough that operators are willing to take the bet. Why offer deepfake services on the darkweb when you can do it on Fiverr? Why go to Tor for illegal goods and services when you can go to Telegram?

What’s left on Tor is what can’t be tolerated anywhere else. Which these days is not much.

Adjacent Reading

1. Jardine et al. “The potential harms of the Tor anonymity network cluster disproportionately in free countries.”

2. Leonidou et al. “A Qualitative Analysis of Illicit Arms Trafficking on Darknet Marketplaces”

3. Soska and Christin. “Measuring the Longitudinal Evolution of the Online Anonymous Marketplace Ecosystem”

Yesterday, WIRED published a piece based on a new report by AI Forensics documenting abuse on Telegram. The headline: men are buying hacking tools to use against their wives and friends. The report behind it is a serious piece of work: 57 pages, 2.8 million messages analyzed across 16 Italian and Spanish-language Telegram groups, collected over six weeks. AI Forensics is a credible organization doing important research.

If you read Part I of this series, nothing in the report will surprise you. Of course Telegram hosts groups where people share non-consensual images. Of course bots play a central role in automating and scaling operations. Of course the groups regenerate hours after being shut down. Telegram has the infrastructure for all of these things. The report documents real harm happening to real people, and the finding that most victims are ordinary women targeted by people they know––partners, ex-partners, acquaintances, colleagues, and classmates––is consistent with everything the academic literature has established about image-based sexual abuse for the past decade.

Where the report runs into trouble is not in what it found, but in what it claims to have found.

Consider the hacking services. The report documents over 18,000 references to spying or surveillance tools, and 441 references to hacking services. Screenshots show users advertising “professional hacking on commission,” promising access to phones and social media accounts. The presentation is alarming. It is also, to anyone who has spent meaningful time in these spaces, mostly theater.

The people offering to hack your wife’s iPhone for $50 are not sitting on zero-day exploit chains. They are not former NSA operators moonlighting on Telegram. They are, overwhelmingly, scammers who can take your money because you will have nobody to complain to. The misspellings are a tell. The pricing is a tell. The vagueness of the technical claims is a tell. Real offensive cyber capabilities are not sold in Telegram groups with open enrollment. At the very least, you would hope your hacking contractor uses Signal.

This does not mean spyware is absent from Telegram. Commercial stalkerware is real, genuinely harmful, and I would not be surprised if there are groups where you can find this software and help deploying it. But the report does not clearly distinguish between a scammer posting a menu of hacking services he cannot deliver and a channel distributing actual surveillance software. The screenshot gets the same evidentiary treatment either way.

The same problem appears with the report’s most alarming claims about child sexual abuse material (CSAM). The report documents folders and offers that claim to contain CSAM. The claims are real, the groups exist, the advertisements exist, the folders are labeled exactly as described. But a lot of what gets advertised in these spaces is also bait: marketing designed to extract payments. The folders can say whatever but this is no guarantee that the content exists or that the advertiser can fulfill the promise. This is not speculation on my part; it is a pattern that anyone who has done sustained fieldwork in underground Telegram communities recognizes. Scammers impersonate the flashy content (usually the worse stuff) because the flashy content commands the highest prices and the most desperate buyers.

I want to be precise about what I am and am not saying. I am not saying CSAM does not exist on Telegram. It does. I am not saying these groups are harmless. The advertisements themselves normalize the content they describe, and some percentage of what circulates is genuine. I am saying that a methodology that treats advertisements as equivalent to verified content will systematically misrepresent the scale and nature of what is actually being transacted. The same thing happened with darkweb marketplaces. People saw that weapons and hits were advertised on these markets, thought it was a common thing, and fell for it.

Here is the distinction that matters: some evidence on Telegram is self-authenticating, and some is not.

When someone posts a non-consensual intimate image in a group, the posting is the harm. The image is real, the victim is real, and the damage is done the moment the file is shared. There is no gap between the advertisement and the product. A screenshot of this is sufficient evidence because the evidence and the harm are the same object.

When someone posts a listing advertising hacking services or CSAM access, the posting is not the harm––or at least, not the harm being claimed. The harm would require the service to be real and the content to be genuine. A screenshot of a listing is evidence of demand. It is evidence that someone believes there is a market for this. It is not evidence of capability or supply.

The report’s strongest findings (the non-consensual image sharing, the nudification bots, the doxxing) are all cases where the posting is the harm. These findings are methodologically sound because they do not require the researcher to verify anything beyond what is visible. The report’s weakest findings (the hacking tools, the CSAM claims, the spyware advertisements) are all cases where the posting is an advertisement for something that may or may not exist behind it. The methodology does not change between these two categories, but it should.

This brings me to a tension I cannot resolve but it’s pervasive in this type of research.

The report’s unreliable evidence points at real problems. Telegram likely hosts spyware distribution. Telegram does host CSAM. The fact that most of the specific listings documented in this report are unverified (and possibly scams) does not mean the underlying harms are fictitious. So if a policymaker reads this report and concludes that Telegram needs stricter oversight––which is AI Forensics’ explicit recommendation, calling for Telegram to be designated a Very Large Online Platform under the DSA––they have arrived at a defensible conclusion through partially indefensible evidence.

Similarly, the WIRED headline may be true, but it is not necessarily supported by the evidence it points to. Is that acceptable? Scientifically, no. Practically, I do not know.

Policy built on misunderstood evidence can misallocate enforcement resources. If regulators believe Telegram is primarily a marketplace for sophisticated hacking tools, they could prioritize interventions for a threat that barely exists while ignoring the one that does: the low-barrier, high-volume sharing of non-consensual intimate images by ordinary men targeting ordinary women. But demanding perfect evidence before acting means never acting, because Telegram’s opacity is precisely what makes perfect evidence impossible.

We do not need to prove that Telegram hosts sophisticated spyware operations to have grounds for action or moral panic. The verified findings––that thousands of people are sharing non-consensual images of women they know, at scale, with essentially no friction––should already be enough. We do not need to reach for more.

Adjacent Reading

• Cuevas et al. Measurement by Proxy: On the Accuracy of Online Marketplace Measurements.

• Indicator. The Indicator Guide to Using Telegram in Digital Investigations.

• Campobasso et al. You Can Tell a Cybercriminal by the Company they Keep: A Framework to Infer the Relevance of Underground Communities to the Threat Landscape

• Nucleo. Telegram App Says It Prioritizes Child Safety. Its Bots Tell a Different Story

• Part I of this series: The Telegram Ecosystem.

PS: Thanks to Miranda Wei for sharing the WIRED piece and AI Forensics report.

Essentially, anybody can now prompt a model to find critical software vulnerabilities. According to keyboard pundits, we are about to be in a world of hurt. I’m willing to be a contrarian on this point and argue that nothing will fundamentally change between attackers and defenders. Finding all these vulnerabilities changes little.

Context. In February, Anthropic’s Red Team published a blog post demonstrating how Claude can, on its own, find a ton of vulnerabilities. A week ago, Nicholas gave a talk at Unprompted showing live how Claude found “zero-day vulnerabilities” in Ghost, the open source version of Substack that I didn’t want to bother setting up, and in none other than one of the most publicly scrutinized codebases: the Linux kernel.

A zero-day vulnerability is a fancy way of saying a vulnerability that no one has seen before. A vulnerability in computer security means that some piece of software has a flaw that can be exploited to do bad things. And, because no one has seen it before, defenses do not yet exist. Imagine, for example, you find a flaw in a government’s server that lets you read all the employees’ information. Or imagine you find a flaw in a company’s systems that allows you to lock all computers down and ask for money to unlock them. Or imagine you find a flaw on peoples’ phones that lets you spy on all their texts, calls, emails, etc. Because of all these crazy things that you can do with never-seen-before software vulnerabilities, protecting against them is really important.

Finding 0-day vulnerabilities was really hard, especially if it affected software that is used in a ton of devices, like the Linux kernel or iOS. Security nerds around the world dreamed of the day they would find one of these, not only because security agencies are constantly searching for top hackers, but because you’d immediately achieve tremendous aura that will have people online arguing about how you are not “some” security researcher.

So when Claude found a decently important vulnerability in the Linux kernel (an impressive feat for mere mortals that do not dream of electric sheep), doomsday predictions began. The standard conclusion is that we are entering a period where anybody will be able to find critical vulnerabilities and that we should all be tremendously worried.

I think this is mostly wrong because of two core reasons. First, attackers and defenders’ capabilities are more symmetric than painted in the current narrative. Second, for most people, the thing that prevents you from exploiting computer systems is that you could be sent to jail for a long time.

To explore these arguments, it’s useful to think of the three broad groups of attackers we might face. On one hand, you have what we like to call a “state actor” (e.g. CIA, NSA, Mossad, MI6, you get the point). You can safely assume that defending against these groups is mostly futile unless you are another state actor. If the CIA wants to get into your laptop, they can leverage a wide variety of tricks, one of which may include a zero-day exploit. And by the way, if they needed one before Claude, they could’ve just bought one, for a few million dollars––pennies on their balance sheet. Even simpler, they can also just come and take you and your laptop away.

Then, there are sophisticated for-profit criminal groups (hackers with black hoodies a la Mr. Robot). They are technically proficient but will most likely not implant secret chips at the factories where phones are made. Lastly, you have normoids (a.k.a. unsophisticated actors, such as your average Twitter tech influencer). They are proficient in vibe coding and logging into their router. Congratulations you have just done something called “threat modeling.”

Ok, back to the counterargument: if language models are good at finding vulnerabilities, they are good at the same reasoning that underlies patching them. Offense and defense are not drawing from different capability pools. They are running the same models against the same code. The question is not whether AI helps attackers––it obviously does––but whether it helps them more than it helps defenders. There is no reason to think the answer is yes. If anybody can now use Claude to find vulnerabilities to exploit, then anybody can now use Claude to find vulnerabilities to patch. Saying AI only helps attackers is like saying faster cars only help bank robbers.

The first objection against the claim above is that patching (i.e., fixing vulnerabilities) is harder than finding them. A common motto is “defenders need to be lucky every time, attackers need to be lucky only once.” On top of that, attack verification is cheaper than patch verification. When you are patching software vulnerabilities, you need to make sure you don’t accidentally mess something else up. When creating an exploit, you just need to find one thing that works. But is it that simple?

The cost of verification is also high for a successful attack campaign. Exploitation is but one step in a multi-step operation. You broke into a server, now what? If your goal is to make money, you need to find something valuable, extract it, cash out, etc. And you need to do all of this without fucking up. Because if you fail to cover your tracks and you do not fall under the category of state-sponsored actor, it’s federal prison for you.

This last point deserves more attention than it typically gets. The security conversation almost always focuses on technical capability as the binding constraint on bad behavior. But think about what actually kept script kiddies in check for the last twenty years. It wasn’t that Kali Linux was too hard to use, or that Metasploit required a PhD––two tools you can download and play with right now. It was that the FBI has a long memory, long arms, and federal prosecutors have a fondness for making examples. The casual and the not-so-casual criminal attackers newly empowered by AI still fear the law; 3D-printed guns didn’t make everybody a killer. Non-state attackers also still have to figure out how to not be idiots about attribution, lest they get extradited to the US to pay their dues.

The natural objection is to move the goal posts. “Claude is good at exploitation now but in the future it will handle the whole kill chain: reconnaissance, exploitation, exfiltration, hell, even cashing out!” Maybe. But if that day comes, the same model will also be able to patch bugs without wrecking the rest of the codebase––a lower bar to clear than the aforementioned. The capability argument cuts both ways or it cuts neither.

But all of this assumes we are transitioning from a world that was reasonably secure into one that is exploitable. The truth is the state of software security was already bad for everyone, not just for people facing nation states. Before AI, software was already outdated, unpatched, and vulnerable. People were falling for phishing links, writing passwords on sticky notes, plugging in random USB drives. Even “naïve” teenagers had access to enough tools to target high-ranking US government officials. You didn’t need a zero-day to make a mess. Arguably, there were already more vulnerabilities than anyone knew what to do with.

None of this is to discount the work that Anthropic's Red Team is doing. An autonomous agent finding and exploiting 0-days is something that, when I was an undergrad researcher, seemed so distant. It is truly remarkable. But capability claims without a threat model just result in fearmongering, the type of fodder CISOs and security startups love to feed because peddlers of software security are always incentivized to paint a bleak future; a future only they and their products can fix.

AI can now automatically create zero-days? So what, most people’s password is still their dog’s name.

Adjacent Reading

• The Untold History of America’s Zero-Day Market

• AI Finds Vulns You Can’t With Nicholas Carlini - Security Cryptography Whatever

• Bruce Schneier - New leaks prove it: the NSA is putting us all at risk to be hacked

• Andy Greenberg - The Shadow Brokers Mess Is What Happens When the NSA Hoards Zero-Days

• The Unfalsifiability of Security Claims

Telegram is an application that began as a messaging app built on an encryption promise. Today, Telegram exists in the Internet the way certain neighborhoods exist in real cities––not because anyone planned them, not because a developer filed permits and laid infrastructure, but because the land was cheap and the landlord didn’t ask questions.

Most people know Telegram houses seedy things. Fewer appreciate that it houses them at a scale and complexity that makes the word “seedy” feel quaint. What Pavel Durov built, intentionally or not, is less a messaging app than a sovereign territory with its own currency and its own infrastructure.

The platform claims over a billion users, which is a number that sounds impressive especially when you realize that a sizeable portion of them are not using it to chat with friends and family but rather to sell things that are illegal, gamble, and engage with fringe ideologies. This is not a design flaw; it is a natural progression for an application whose founder describes it as a platform for freedom of speech. This is technically accurate in the same way that the Silk Road was a free market economics experiment and a gun shop can describe itself as a platform for conflict resolution.

Telegram’s architecture is communication-centric: you can send messages that vanish and create channels that broadcast to millions with no friction. But the whole apparatus truly becomes alive thanks to the bots; they are what make it a city rather than a bulletin board. Bots are the city’s infrastructure. Some are bouncers, verifying membership before granting channel access. Some are streamers, broadcasting content continuously to passive audiences. Some are couriers, automating delivery logistics. Some are lawyers, enforcing the rules of private groups without human oversight. And yes, some are cashiers, performing every conceivable commercial function from pizza delivery to fentanyl procurement. Our pre-print on bots (to appear at ICWSM later this year) is out.

Bots, through their Turing completeness, dissolve technical constraints and extend Telegram’s functionality to whatever humans decide to want. The bots are the vending machines. The only question left is the currency.

Money is where app stores and governments can take their cut––the entities that need to look away for this experiment to continue. Telegram has engineered its own economic plumbing accordingly. Inside the app there are Stars, an in-app currency for digital transactions. Alongside it, there is a blockchain, wallets, and a suite of cryptocurrency integrations. For the privacy-conscious, there is an in-app browser pointed at decentralized websites in its very own TON network, and failing that, in-app JavaScript rendering that never requires you to surface into the visible Internet at all.

Arriving at the Walled City of Telegram is easy. Many newcomers have visited its various storefronts by deciphering QR codes posted on street lamps, or by searching for synthetic joy at a music festival.

If you don’t have a plug, finding your way around is tough. There are no sanctioned yellow pages. Google has not mapped this place out. Outside observers, don’t know how deep the tunnels are, nor how many rooms, nor how many people; only what Telegram tells us. There are third and fourth parties that have attempted walking every corridor and knocking on every door. Of course, a lot of residents want to stand out and attract your business so they put up posters and leave flyers through their very own advertising ecosystem and mini app store. But many rooms are closed to the public and many groups prefer their nooks and crannies to stay tucked away. If you know a guy that knows a guy, you might get an intro.

Telegram is self-sufficient and wants for nothing. It has its own currency. Its own blockchain. Its own advertising ecosystem. Its own browser. Its own app store. It is both externally and internally opaque. Those who wish to be found can, and those who wish to stay tucked away will. With all this infrastructure, what did you imagine people would build?

What Telegram has accidentally built, in the process of annoying the FSB and the FBI and the BKA and approximately forty other law enforcement acronyms simultaneously, is the infrastructure that Tor always promised but never delivered: a functional underground economy at consumer scale. Tor was too complicated; too easy to fuck up. It offered mathematical guarantees to people who couldn’t reliably operate a VPN. Telegram offered something more valuable than formally verified security––it offered convenience. The onboarding is an app store download. The payments are automated. You give up Tor’s cryptographic promises for Telegram’s pinky promise that they will not rat you out, and it turns out that for most people, most of the time, that trade is worth making. This is what scaling an illicit economy actually looks like: not dark, not hidden, not technically sophisticated. Just friction, methodically removed.

Adjacent Reading

• BBC. Telegram: ‘The dark web in your pocket’

• Rice Media. The Risks and Rewards of Telegram Sex Work

• Marjanov et al. Stayin’ Alive: How Global Stolen Data Markets Thrive on Telegram (Embargoed Until July 2026).

• Roy et al. DarkGram: A Large-Scale Analysis of Cybercriminal Activity Channels on Telegram

• Marechal, From Russia With Crypto: A Political History of Telegram

• Chou et al. Bots can Snoop: Uncovering and Mitigating Privacy Risks of Bots in Group Chats

AI influencers are here. This was entirely expected. Even before generative AI was a thing, we (meaning us, humans) had been toying with the idea of virtual/artificial personas, in social media and beyond, just look at Gorillaz or Hatsune Miku. But of course, much of the Internet is for prurient uses. So, it seemed like only a matter of time until we saw AI influencers used for explicit content on platforms like OnlyFans or Fanvue. Recent reporting paints these AI sex workers as a new phenomenon. But in reality “fake” accounts selling “fake” adult content were already here.

One of the most notable recent cases of a fake OnlyFans model going viral is AyannaSoBlack (now banned from Instagram). A picture of her and former-WWE-star-turned-actor John Cena at Planet Fitness accumulated hundreds of thousands of digital ogles across social media. Similar to other adult content creators, her profile invited you to know her more intimately for a low price. People were so enthralled by her allure that requests for her adult media quickly surfaced in adult piracy forums. What turned this into a fiasco was the collective realization (and disappointment) that she had been artificially generated by matrix multiplication. The account purposefully intended to deceive users. For example, the model in question would answer in Q&A style that her fascinating dark pigment was due to South Sudanese genetics. As per modern internet etiquette and morality (as well as OnlyFans guidelines): undisclosed AI use is a no-no.

Now, the first thing I want to dispel about this whole thing is that this phenomenon is somehow new. A man pretending to be a woman online for profit the oldest trick in the book.

When I was barely a teenager, I discovered MMORPGs. WoW and Ragnarok were my go-tos. While these games were instrumental in my acquisition of the English language, they also demanded copious amounts of hours doing repetitive tasks to make in-game money and to obtain cool shiny items. However, if toiling around those universes collecting minerals and such was not your calling, you could do what my friend N did: he would create a female character, befriend presumably male players, and try to convert them into patrons. And just like that, N—a then-13-year old from the middle of nowhere South America with a limited yet expanding English vocabulary—adeptly demonstrated why the FTC’s statistics on romance scams make complete sense (and, unfortunately, the stats about online grooming).

If N had not gone to college he would’ve certainly taken part in the many communities who realized that they could pretend to be women online for monetary gain. As I hinted at in the previous paragraph, romance scams are one flavor of this approach. You receive a message from an unknown number who wants to befriend you. They lead you to a Whatsapp or Telegram chat with an attractive person as a profile picture. They then try to convince you to part ways with your money with a variety of techniques, ranging from persuasion to (s)extortion. Scammers are allegedly pulling more than a billion dollars with playbooks like this.

Whereas in a traditional romance scam you build and leverage an emotional bond to extract value out of a victim, a parallel tactic involves convincing the victim to buy your spicy pictures. In 2019, Hutchings and Pastrana published a paper on what is known online as “eWhoring”: tricking people into buying adult content that you did not create. The first thing an aspiring eWhoring entrepreneur needs is the right type of content, or a “pack.” Procurement is the act of finding “packs” of pictures about the person you want to impersonate. Ideally, you want someone with a wide range of pictures (often taken from social media) who is not very known (little followers) and for whom you also have access to nude or sexual media (often obtained nonconsensually). This combination of attributes was challenging to achieve, so people in online forums also specialized in collecting and selling these packs. With a suitable pack you could begin generating leads and closing sales.

For the past 16 years, the playbook has remained relatively consistent. An account follows you on a social media platform (e.g., Instagram), and proceeds to persuade you to buy “premium” content. These sales were more ad-hoc, through something like Venmo, or in more sophisticated cases a landing page with a credit card terminal, similar to those that tried to sell you eBooks.

The advent of OnlyFans (OF) introduced a new paradigm for adult content creation, both for legitimate content creators and otherwise. OF made procurement substantially easier because it centralized a vast repository of explicit images. To create a pack, all you needed to do was to find an OF model, their social media account, and you could begin a copycat of them without them ever finding out. OF’s infrastructure made monetization easier. And lastly, their opaque external visibility made it harder for people to identify copycats: a model from Mexico would have a hard time knowing that a teen from the other side of the world was running an account with their stolen pictures.

AI has now similarly re-upended the procurement paradigm. Any person can now be a victim because AI models allow the creation of non-consensual images, explicit or otherwise. I can run an Instagram account using someone else’s legitimate pictures and sell photorealistic explicit media of them on OnlyFans. Or, I can even take it up a notch and run a completely synthetic Instagram and OF account leveraging a pre-trained AI model on their likeness. I can use this AI model to post pictures of them at school, at the beach, posing with John Cena, or performing whatever sexual acts my followers are willing to pay money for. On top of that, you can hire gig workers to help you out. To borrow a word that technoidealists love to use, AI has now democratized victimization.

When AyannaSoBlack was exposed, people were not outraged that they had paid for adult content. They were outraged that the intimacy had been fabricated, as if the intimacy was ever real. On top of that, keyboard doomers treated this phenomenon as some new dark AI-enabled innovation. But a man pretending to be a woman online for profit is, as I noted, a discovery that even a 13-year old boy makes and exploits. Performances are powerful: the best adult content creators know this. They create micro-universes in which their subscribers can dream—for 5, 10, or more dollars a pop. We are just now entering a universe in which this parasocial fantasy, this performed desire, can be executed on an office floor somewhere in Eastern Europe… wait, we were literally already there.

Adjacent Reading

• Inside the Booming AI Pimping Industry

• How AI-generated Influencers Exploit Celebrities to Sell Synthetic Nudes

• Pastrana et al. “Measuring eWhoring”

• OnlyFans Models Are Using AI Impersonators to Keep Up With Their DMs

Notes:

• I use “OnlyFans” as a blanket term for similar platforms, including Fanvue.

Source: https://www.vice.com/da/article/humans-of-the-cd-street-stall-who-still-buys-pirated-discs/

Last week we discussed why there’s a surge of gig workers that will create AI porn for you. To explain this phenomenon, I talked about money and markets. It comes as no surprise that sex and money go hand in hand. And because of this a lot of the interventions that scientists and lawmakers discuss are centered around money. Last year, a group of state Attorney Generals sent a letter to payment processors telling them to do more. Adding financial friction is powerful: you make it unprofitable and annoying to pay for something and people walk away. This is part of the reason why we are not sending each other crypto when splitting dinners.

However, the underlying assumption is that most people are primarily money-motivated (a.k.a. “finacially-motivated rational actors”). Weirdly, some people are not. Some seem to be guided by morals, values, ideologies and such. Our financial enforcement measures would work if it were not for these pesky ideas.

A Short Intro to Piracy

Somehow we landed in a reality where people pay for the right to stream music and movies. In the 2000s this seemed unconceivable, laughable even. Limewire, Ares, Bearshare, ThePirateBay, (I can keep going), provided uninhibited access to any and all digital needs. They still do. But, the hassle of figuring out how to pirate properly, the fear that you might download malware, and the worry that your Internet service provider will send you a threatening letter have pushed people to more convenient options. Piracy, though, is still alive and well, powered by a community of committed enthusiasts.

Sources: https://www.reddit.com/r/Piracy/comments/1k8jtd6/lmao/ [1]

Piracy gets a bad rep. Probably because of the edgy ad campaigns equating piracy to stealing cars and unsuspecting women. Yes, piracy can suck. It can really harm creators––especially smaller ones––of movies, music, books, etc. But piracy also has served a variety of noble goals. For example, if you grew up where I did, you just couldn’t buy certain stuff, even if you wanted to. You wanted to buy a Windows XP license? Good luck. On top of that, prices were tremendously prohibitive for us wretched and proud inhabitants of the developing world. An Adobe CS6 license would cost ~$2,600 according to this Reddit user. That was just about the minimum wage in Paraguay in 2010... for one year of work. Maybe you hold the opinion that kids from the barrios should not access digital goods they cannot afford. But those kids also wanted to play Warcraft 3, watch Full Metal Alchemist, and make beats with Pro Tools. Thank you pirates for your service.

Source: https://www.vice.com/da/article/humans-of-the-cd-street-stall-who-still-buys-pirated-discs/

A lot of pirating groups have a strong code of conduct and ethos. These groups, which roughly trace back to early hacker/technolibertarian cultures are of course not a monolith. Id est, there is no unified cultural umbrella or dogma that governs all self-declared members of the aforementioned ideologies. However, there are a few generally accepted concepts: free access to information, anti-corporate sentiment (stick it to the man but not to indie creators), archival (preservation of content), sharing as reciprocity, and technical prestige (”I pirate faster, better, quicker”). These values also exist in many other communities like open source software and open access.

Source: iykyk.

There is another highly pirated media category: porn.

Porn Piracy

As you would expect, porn is pirated because of paywalls and because studios thought that $25 dollars for movie access at a hotel made sense. When tube sites like PornHub and YouPorn launched in 2006-2007, porn studios saw their revenue collapse. These sites allowed anybody to upload content, so users began uploading pirated scenes en masse. Big Porn tried to fight tooth and nail with lawsuits and takedowns. But, like other media producers, pornographers ultimately adapted and found ways to make people open their wallets. OnlyFans as a whole grosses billions of dollars per year; top earners in the platform make millions. Of course, the same battle that porn studios fought against pirating communities are now being fought by individual content creators. Today, there are several online communities dedicated to procuring, indexing, and distributing paywalled content belonging to OnlyFans models. Where there’s paywall, there’s a leak.

Porn piracy may seem like it is primarily a financial thing. But porn is an attractive candidate for piracy not just because it is often paywalled, but because ‘til this day, porn also pushes censorship boundaries. In the US, dirty films can still be cataloged as obscene if they do not satisfy the Miller test. In Japan, porn is obscene if it displays genitalia (hence why it is blurred). In several countries, porn is outright banned. But boy, people do not like it when music/films/books are banned or censored. Take books as an example. One of the largest shadow libraries traces its roots to Samizdat––an underground movement to resist book censorship in the Eastern Bloc. Today, we still celebrate banned books week. If Sarah Maas books were banned tomorrow, the riots would be violent. When pornographic content is banned, it triggers the same ideological knee jerk reaction: people want to preserve and redistribute the content, they want to fight back, lest their freedoms be trampled.

Source: <redacted>

Porn Preservationists

Last year, HuggingFace and CivitAI suddenly started taking down AI models used to generate deepfake porn (or synthetic non-consensual explicit imagery for long). Other adjacent platforms like SeaArt and TensorArt went farther and even took down all adult-generating content. These actions came as a reaction to a period of heightened scrutiny by lawmakers: the UK began enforcing its Online Safety Act, the US passed the TAKE IT DOWN Act, journalists and scientists began demonstrating the prevalence of deepfake porn in these platforms, and so on. We published a paper on the impact of this regulatory pressure on the ecosystem of deepfake pornography. Now, what we did not capture in that paper was this underground movement that quickly took place after platforms began taking down these models. The key difference, of course, is that the content being preserved here is not Dr. Zhivago, but an AI model that can generate Hermione Granger’s nudes.

Days after the purge, I came across several initiatives oriented at preserving these porn-generating AI models that were getting taken down. It was like a natural catastrophe hit and these netizens were tasked with orchestrating a sort of humanitarian response. Volunteers emerged. Missing model posters were put up. People shared their libraries, others dedicated hard drives for storage, others offered to design databases and build the necessary tooling. And as if the library of Alexandria was in flames again, they mourned the models that had not been saved: lost treasures that future generations would never get to appreciate. They vowed this would never happen again.

Source: <redacted>

Today, one community dedicated to the preservation of deplatformed AI models has over 4.5k members. There’s a version of this story where that number stays small. But the more instructive version is the one where it doesn’t, where the same cultural infrastructure that allowed kids from Ciudad del Este to buy PlayStation games now keeps non-consensual imagery of real women circulating indefinitely, powered by people who genuinely believe they’re on the right side of a censorship fight. That belief is not entirely irrational. Enforcement has consistently struggled against ideologically motivated communities, not because the ideology is correct, but because financial friction is a weak tool against people who aren’t primarily motivated by money. The piracy wars taught us that. Here we go again.

Further Education

• A very pixelated view of Ciudad del Este from back in the day:

• A mall in Ciudad del Este:

• An English-speaking news report about Ciudad del Este.

• An FBI note with the same tone I use in these blogs: https://archives.fbi.gov/archives/news/stories/2004/may/piracy_051704

• Book: “Media Piracy in Emerging Economies”

  • “Relative to local economies a copy of Microsoft Office is 5-10 times higher than in the US or Europe”

  • I found this piece after I finished writing this blog post.

• Paper: “Studying the Online Deepfake Community”

Footnotes

[1] Apparently, the anti piracy ad shown above was not the one pirated, but the universe in which it was is funnier https://torrentfreak.com/sorry-the-you-wouldnt-steal-a-car-anti-piracy-ad-wasnt-pirated-170625/

Source: https://www.nytimes.com/2012/03/18/nyregion/from-roosevelts-new-york-lessons-in-the-futility-of-policing-vice-in-the-city.html

I’ve been looking at how people do bad things on the internet for a while. This has taken me to various communities, forums, server, chats, and other digital hangout spots where I can observe and read peoples’ behavior. This boots-on-the-ground approach helps me get a pulse of why people do what they do, how, and get a sense of what comes next. These are all useful things for policies or so called “scientific interventions.” Essentially, we want to figure out how to stop people from doing bad things, like generating and sharing synthetic nonconsensual images (”deepfake pornography” for the masses). The goal of this blog post is to try to understand our Fiverr findings by thinking about them in the same terms as other vices on the Internet.

We recently published a paper that showed how there are listings on Fiverr (paper; blog post), which is a popular gig work platform, that allows people to hire contractors to create pornographic deepfakes. That is, you can hire a freelancer to help you make AI-generated nudes of someone else on Fiverr, same as you would hire someone to help you make a logo or run online ads. When Mohamed (PhD student who led the Fiverr work) approached me and told me about these services, it felt important to document it, not just because it’s a thing that is happening, but because it tells a broader story of existing enforcement efforts.

Source: Fiverr paper linked above.

The story of drugs online provides a useful way of thinking about enforcement. During my PhD I looked at how and where people sold illicit substances online. “Darkweb marketplaces” (i.e., eBay-like websites that you access through Tor) were already mature enterprises in the 2020s, accounting for billions of dollars in revenue. They had come a long way from the moment that someone had the straightforward idea of selling drugs over this then-new thing called “the Internet.” If we trace the evolution of drug sales online, we can probably get a blueprint of what will play out with deepfake pornography.

First came the people who advertised directly on websites: “canna seeds for sale, we take western union.” Then, came the forums (e.g., “The Hive”). Afterwards, we got the first semblance of markets (e.g., “Adamflowers” then “The Farmer’s Market”). As you may imagine, the DEA, FBI, and adjacent acronyms cracked down on these sites aggressively. Eventually, came the “Silk Road” and the darkweb era facilitated by the emergence of cryptocurrencies. The way in which technological developments (like Tor and cryptocurrencies) ushered new opportunities to sell narcotics and other illegal goods is fascinating. We will tell, however, that story another day. Today, we will talk about the market pressures that pushed these websites to evolve.

In his thesis, “Outsourcing Cybercrime”, Rolf van Wegberg [1] draws from transaction cost economics to describe how cybercriminal markets increasingly become more sophisticated. One way they do so is by becoming more structured. This is most clear when you look at forums that then become marketplaces. For example, before Facebook Marketplace was a thing people used Facebook Groups like “Buy/Sell/Trade New York” to sell things, and before that, people directly posted on their profile that they were selling something. The transition from sales posts in personal profiles, to groups dedicated to buying and selling, and then a dedicated platform of sales illustrates how markets seek structure and sophistication. The same pattern plays out for drug sales: standalone websites advertising drugs, then forums, then marketplaces.

Source: https://www.aljazeera.com/news/2013/11/6/silk-road-illicit-online-market-reopens

Markets are good for vendors and buyers. A market-type-of-website makes it easier to search for products, it introduces reputation systems (e.g., reviews and ratings), standardizes payments, and introduces a mediator who can resolve disputes when things go wrong. There is also an incentive for people to create the market itself. If you are the owner of the market, you take a cut of all the deals. In the Silk Road trial, the FBI said it had seized 144,000 BTC (yes, a hundred and forty four thousand Bitcoins) from Ross Ulbricht. In 2022, the Hydra market was allegedly pulling 1 billion sovereign American dollars. Owning the platform is definitely where the money is at, so when a market disappears, another quickly steps in. The demand for drugs definitely does not go away. And you, as the market owner: all you need to do is avoid the cops and the suits [2].

Source: https://www.nbcnews.com/id/wbna6809149

But before starting a market there are alternatives to explore. Starting a market is tedious. Before LLMs, you needed at least some semblance of coding experience to get it up and running. You then needed to overcome a cold start problem and attract vendors and buyers. So instead, people tested existing markets like eBay. Vendors would advertise the sale of drug “catalogs” or “magazines” while making air quotes––”it would be a shame if someone happened to accidentally drop a vial with four carbon rings, some oxygens, and a few hydrogens originally meant for veterinary use but which gives you superior athletic performance, right?” With less frills, people also just directly sold cannabis on Facebook Marketplace.

If this post is clear, by now you should see the parallels with Fiverr.

As of last year, there were three main places where you could go to get your fix of synthetic nudity. Dedicated websites where you could upload images and perversely manipulate them yourself (e.g., nudifiers). Forums where people share their creations, commission new fakes, and share tips and tutorials (e.g. MrDeepfakes). And these part-forum, part-cloud-infrastructure, part-market platforms where you could train and share AI models, share your images/videos, and earn platform currency that enables you to generate more images/videos (e.g., CivitAI). This last one is not yet a market like eBay but has a lot of market-like characteristics. For example, you earn platform currency based on how much people interact with your model, which in turn makes model providers (think of them as vendors) to try to put out better models (think products) to attract more users (customers), like a market [3]. Given all of this, it is not surprising that people are testing the waters in established markets, like Fiverr. Economics and market dynamics demand more sophistication and structure.

The market for deepfake pornography is still finding its footing. Many peddlers of artificially generated genitalia rely on ad-hoc mechanisms to monetize their creations: they have tip jars on their profile, they create private communities and charge for access, or they rely on affiliate programs to nudification websites and such. Increasingly, though, people are trying their luck in established markets. Fiverr, OnlyFans, Fanvue, and others, have structure and sophistication. They make payments easy. They have brand reputation. They have active user bases. And they are not turning these vendors away. How long do you think anabolic steroid vendors stayed active on eBay before they were shut down?

There are many similarities in the trajectories of online drug sales and synthetically generated body transgressions. The websites, the forums, the economic pressures. These factors help us understand why these services are popping up on gig platforms and mainstream markets. But ultimately there is one crucial difference: making and commissioning deepfakes is pretty cheap. Websites with very powerful models give daily free credits for generation. For users with GPUs, they can generate so much of this stuff and they take commissions for free and with gusto. There is no incentive yet to create a marketplace for this stuff since so much of the volume is gratis!

Source: <redacted>

The low cost of production also means enforcement faces a different challenge here than it did with drugs. You cannot cut off the money if there is no money to cut off. Procuring sexually explicit videos of your colleagues, classmates, or favorite celebrity is cheap, easy to make, and easier to find. In fact, the listings we identified (and which we reported) in our Fiverr study still seem to be there. Chasing down deepfake pornographers seems like hopeless work––for every head you cut off, a new one sprouts. And then, even if you get rid of these websites on the clear web, they can migrate to encrypted chat groups or the ~dark~ web. But hear me out. One, we are a long way from that, and if these services migrate to shadier corners we will at least have a sense that enforcement is actually doing something. Two, we pursue CSAM distributors relentlessly, even despite all the constraints above. And three, come on, detecting these listings on a website like Fiverr is trivial in 2026.

If we ever see these services migrate to the shadier parts of the Internet, and if a market ever pops up exclusively dedicated to the commercialization of deepfake pornography, as bleak as it sounds, we will know we are on the right track.

Related Reading

• Plug and Prey? Measuring the Commoditization of Cybercrime via Online Anonymous Markets

• Deepfakes on Demand: The rise of accessible non-consensual deepfake image generators

• Exploring the Use of Abusive Generative AI Models on Civitai

• Analyzing the AI Nudification Application Ecosystem

Footnotes

[1] Rolf was on my thesis committee and has been a phenomenal mentor. Thanks Rolf!

[2] Fun fact, the IRS is even more interested than the FBI in whether you own the marke: https://home.treasury.gov/news/press-releases/jy0701

[3] It is important to note that CivitAI cleaned house after substantial pressure from payment processors; they introduced cryptocurrencies and a gift-card-type of payment. There was a genuine reluctance to see Visa and Mastercard go away which means that the alternatives are just not as good.