I Smell a Bot: California’s S.B. 1001, Free Speech, and the Future of Bot Regulation

I Smell a Bot:^[1] California’s S.B. 1001, Free Speech, and the Future of Bot Regulation

I. Introduction

Among the most revolutionary changes wrought by wide-spread internet use is the disruption of hierarchies within the marketplace of ideas.^[2] In the age of social media egalitarianism, individual viewpoints are far less constrained by the gatekeeping of traditional media. On one hand, this new reality has benefits—a lack of top-down control allows for more consumer choice and fosters a more diverse range of voices. But there are also downsides—while media was once primarily legitimized by the editorial process, the credibility of content is now tied to a populist notion of virality. As Americans increasingly rely on online peer-to-peer interaction for accessing information and engaging in commerce, an emerging technology, online bots, may be used to exploit trust in online identities, defraud consumers, and stoke political tensions.^[3]

In reaction, California lawmakers passed Senate Bill No. 1001 (“S.B. 1001”), a transparency law that regulates the use of bots but also raises unique First Amendment questions.^[4] The purpose of this Comment is to explore the free speech issues raised by S.B. 1001, evaluate the likelihood that the law survives a constitutional challenge, and suggest ways to improve future bot regulation. Part II will define bots, discuss why their regulation may be necessary, and address the competing approaches to such regulation. Next, Part III will analyze First Amendment issues that may arise should S.B. 1001 be challenged in court. Finally, Part IV will make two recommendations. First, lawmakers could avoid the appearance of expressly targeting political content (reducing the risk of a successful free speech challenge) and retain the democratic benefits of S.B. 1001 by expanding the commercial prong of the statute. Second, lawmakers could better achieve the goal of regulating bots by moving beyond transparency. Overall, this Comment concludes that, while S.B. 1001 may have the effect of chilling some expression, the law has an overall net-positive effect on the marketplace of ideas and will likely survive a First Amendment challenge in its current form. However, this Author believes that a regulatory scheme based on transparency alone may prove inadequate in the long-term.

II. Framing The Issue

A. Defining Bots

In general terms, bots are automation programs.^[5] Whereas robots are hardware programmed to perform automated tasks in physical space, bots are software programmed to perform automated tasks in digital space.^[6] Bots come in different shapes and sizes; for example, web crawlers provide search results,^[7] social media bots create automated posts,^[8] and helper bots guide users toward solutions.^[9] While some bots may interface exclusively with other computer systems, and others may communicate directly with human users, all bots have one thing in common—they act as agents in service of their human controllers.^[10]

Though they may be programmed with artificial intelligence (AI), many bots can perform complex automation without the use of machine learning or other algorithms. For example, chatbots, programs that “engage[] with users by replicating natural language conversations,”^[11] do not require AI to replicate human conversation; many use “pattern matching,” a process whereby a program is given “rules to govern the system’s response to given inputs.”^[12] MIT professor Joseph Weizenbaum, the creator of ELIZA, an early conversation program developed between 1964 and 1966, compares ELIZA’s pattern matching to an actor using a script to “improvise around a certain theme.”^[13] Because chatbots do not require advanced technology to functionally interface with humans, and because today’s technology users are especially comfortable with chat functions, some technology experts foresee chatbots playing a significant role in the future of user interfaces more generally.^[14]

As a broad category, bots may seem an unwieldy technology use case for government regulation. However, S.B. 1001 provides a narrower view of bots, defining them as “automated online account[s] where all or substantially all of the actions [performed] or posts [generated] are not the result of a person.”^[15] This definition does not identify a bot by the nature of its software, but instead by its outward appearance and, implicitly, by its purpose. Though “online accounts” generally serve an identification purpose for system administrators, social media and online shopping accounts in particular have popularized the use of “profiles” for user identification.^[16] Commonly, an online human is identifiable by a screen name, profile picture, short bio, and possibly a few other identifying pieces of information.^[17] The purpose of S.B. 1001 is to regulate accounts that have those basic features and may, at first glance, seem human, but produce automated content.^[18] Importantly, California’s bot definition allows for at least some amount of direct human control.

B. Current Issues and Proposed Regulation

Although bots have existed on the internet since the birth of Internet Relay Chat in 1988,^[19] their proliferation throughout social media is now under the microscope following reports by U.S. intelligence services that foreign agents used social media to influence the 2016 U.S. presidential election.^[20] The present legal and ethical conversation concerning bots is in reaction to their effect on social media, but it has become increasingly clear that bots’ influence may extend to other commercial and political environments.^[21] This Section will discuss problems with unregulated bots and proposed regulations.

1. Why Regulate Bots?

Bots are problematic for the same reason that they are useful—they amplify the power and efficiency of a single person.^[22] Because the human ability to use computer systems is limited by the ability to physically input commands,^[23] bots can be programmed to perform computer-based tasks more quickly and reliably than a human without access to such technology.^[24] Even without the use of advanced AI technology, a single bot’s increased efficiency relative to a group of human users makes bot use lucrative and ripe for abuse.^[25] Specifically, experts express concerns that bots can be used to fraudulently manufacture consensus and defraud consumers.^[26]

Manufactured Consensus. First, research shows that internet communities are effective environments for building consensus among individual users,^[27] and bots can be used to manipulate that effect. The major social networks, in particular, are founded on the idea that organic sharing can have a viral effect, diffusing content to an exponential number of users beyond the original user’s immediate network of friends (on Facebook) or followers (on Twitter).^[28] Because it is possible for a single independent producer to widely distribute content based on shares and trending tags, bots programmed to share specific content can be used “to create a bandwagon effect, to build fake social media trends . . . and even to suppress the opinions of the opposition.”^[29] And this effect is not limited to social media ecosystems; one particularly worrisome example of non-social-media bot influence occurred on a federal website.^[30] During the Federal Communications Commission’s period for public comment on recent net neutrality rulemaking, bots contributed hundreds of thousands of comments in support of positions that had relatively little public support.^[31] In a different vein, the popularity and relative power of commercial review sites such as Rotten Tomatoes and Yelp have made such sites targets for bot influence.^[32] Other sensitive online processes may be targeted by similar efforts.^[33]

Fraudulent Bots. Bots may also be used to defraud consumers looking to purchase the kind of online influence described above. The value of the actual bandwagon effect aside, the mere appearance of organic impressions is a valuable commodity in internet commerce but is often fraudulently produced via bot technology.^[34] In a recent interview, Robert Hertzberg, a California State Senator and author of S.B. 1001, gave three examples of how bots can be used to defraud consumers.^[35] First, services that promise to help users‌ or businesses grow their social media reach may use bot accounts to pad follower or ‌‌subscriber counts.^[36] For example, Instagram has tacitly allowed the buying and selling of followers and likes, though it may now be using its own “good” AI to weed out the fakes.^[37] Second, companies seeking an IPO or individuals seeking employment as influencers may misrepresent the scope of their organic reach by providing follower or subscriber counts padded by bots.^[38] Third, bot-boosted false news stories can be used to increase ad revenue for their hosts.^[39]

As is the case with many regulated technologies, dangers inherent to the technology are magnified by external factors—here, the growth of internet commerce and the commodification of personal data.^[40] As we do more business, store more data, and live more of our lives online,^[41] marketers, governments, and influencers benefit from a treasure trove of information previously unavailable or too difficult to collect in the physical world.^[42] Each link clicked, product placed in a shopping cart, and “follow” is valuable when aggregated.^[43] Because the value of our personal data is so high, the largest social networks and retailers are incentivized to automate user interactions to efficiently collect as much as possible.^[44]

2. California S.B. 1001

On September 28, 2018, Governor Jerry Brown signed Senate Bill 1001,^[45] making California the first U.S. jurisdiction to promulgate a bot law.^[46] The substantive portion of the Bill, which went into effect in July 2019, reads as follows:

It shall be unlawful for any person to use a bot to communicate or interact with another person in California online, with the intent to mislead the other person about its artificial identity for the purpose of knowingly deceiving the person about the content of the communication in order to incentivize a purchase or sale of goods or services in a commercial transaction or to influence a vote in an election. A person using a bot shall not be liable under this section if the person discloses that it is a bot.^[47]

S.B. 1001 represents one possible method for regulating bots: state-compelled bot-user transparency.

First, S.B. 1001 is state legislative action.^[48] Courts have acknowledged the internet as an “instrument of interstate commerce,”^[49] making regulation of its activities, presumably including the use of bots, subject to Congress’s authority under the Commerce Clause.^[50] However, several states have promulgated legislation to control internet communication in the past, specifically targeting sexually explicit materials and “spam.”^[51] Although state-based regulation of the internet raises very interesting federalism questions, a Dormant Commerce Clause analysis is a topic best left for another comment.

Next, S.B. 1001 targets bot “users.” Maybe the most important policy choice for lawmakers is where to focus bot-control efforts—the individual programmers responsible for a bot’s creation, the end users of the software, or the online sites and social networks on which bots operate? The ultimate decision by California lawmakers to target users reflects some of the practical hurdles of regulating the internet more generally. On one hand, a platform-focused approach may require a more coordinated legal effort because many institutional barriers exist to protect online service providers.^[52] On the other hand, focusing on creators and users of bots is more direct; though, achieving proper jurisdiction over individual internet users may be extremely difficult.^[53] Notably, S.B. 1001 makes it unlawful for those who “use” bots but does not create liability for those who create bots that are then used for purposes deemed unlawful under the Bill.^[54]

Finally, S.B. 1001 is a transparency law. S.B. 1001 requires that the disclosure imposed by the law be “clear, conspicuous, and reasonably designed to inform persons with whom the bot communicates . . . .”^[55] Regulating bots through user transparency is enticing to lawmakers; on principle, transparency provides balance between the politically polarized American values of freedom of contract and consumer protection.^[56] But there are limits to transparency’s effectiveness—some commentators argue that “transparency places too much of the burden on users to understand the information being disclosed to them and to take appropriate responsive actions.”^[57] Additionally, transparency regimes with “one-size-fits-all applicability” may cause unavoidable First Amendment challenges.^[58] But despite the benefits and costs of transparency, does society actually benefit from increased transparency alone? In his recent article titled Forthright Code, Paul Ohm suggests imposing a stronger standard that “focus[es] on the actions of the entity developing or using the AI system, rather than [simply] describ[ing] a desirable attribute of the system itself”—forthrightness.^[59] In the context of AI and bots, creating an “obligation of forthrightness”^[60] may even involve establishing “information fiduciar[y]” relationships between the large internet platforms and consumers.^[61]

III. First Amendment Analysis

According to California Senate committee reports, early critics of S.B. 1001 were particularly concerned with how the regulation may stifle free speech.^[62] Since passage, Robert Hertzberg, the Bill’s author, has expressed confidence that the final Bill would pass constitutional muster.^[63] Notwithstanding those changes, S.B. 1001 still presents two surface-level issues: First, the law places limitations on how bots, and their users, by extension, can communicate with California citizens; second, S.B. 1001 compels bots, and their users, again by extension, to speak.^[64] These issues are further complicated by the machine-generated qualities of the regulated conduct and varying levels of bot autonomy.

Because U.S. courts have not yet ruled on a First Amendment bot-speech controversy,^[65] this Comment will consider caselaw from adjacent areas of First Amendment jurisprudence as well as scholarly suggestions of how courts may handle the emerging world of AI and technology law. In order to analyze whether S.B. 1001 would survive a hypothetical First Amendment challenge, this Part will explore three broad questions: First, should the conduct regulated by S.B. 1001 receive First Amendment coverage? Next, if the regulated expression is covered, what kind of scrutiny should the regulation receive? Finally, would S.B. 1001 survive strict scrutiny?

A. Should Conduct Regulated by S.B. 1001 Receive First Amendment Coverage?

When speech-related laws are challenged, courts must first determine whether the regulated expression actually triggers a First Amendment analysis.^[66] As mentioned above, S.B. 1001 regulates two types of expression: a bot communicating with a human and the bot disclosing its bot-ness. Setting aside S.B. 1001’s disclosure requirement, the regulated conduct alone, “us[ing] a bot to communicate or interact with another person in California online,”^[67] will likely trigger First Amendment coverage. Although U.S. courts have interpreted the First Amendment to cover emerging modes of communication,^[68] the Supreme Court has hesitated to create wholly new categories of protected speech,^[69] choosing instead to draw simple analogies between new modes and traditional protected speech.^[70] Experts in the growing field of AI law argue that determining First Amendment coverage for machine-generated speech may require more than a simple analogy;^[71] this Section adapts Tim Wu’s coverage framework for machine speech to determine whether the conduct regulated by S.B. 1001 should be covered under a First Amendment analysis: We must ask, first, whether the right is claimed by a “person;” next, whether the regulated expression is considered “speech;” and finally, whether the regulated expression categorically triggers a First Amendment analysis.^[72]

1. Personhood

Though free speech theory has now incorporated more than just individual persons,^[73] there are limits to who can claim a free speech right.^[74] Unlike strong AI personhood, which may require rethinking about how autonomy functions within free speech,^[75] bot personhood more cleanly comports with established free speech jurisprudence; the Supreme Court has traditionally downplayed strict personhood^[76] and recognized protections for speech through technological vessels.^[77] Ignoring the possibility of fully autonomous bots, when a bot engages a human, the bot’s “behavior” is a direct result of a programmer’s code,^[78] thus, courts are more likely to find the programmer to be the true “speaker” instead of the machine itself.^[79]

However, the fact that S.B. 1001 targets only bot “users” complicates the personhood analysis. The conduct regulated by S.B. 1001 includes two separate expressions, meaning the law could potentially infringe on two separate speech interests, including interests not directly tied to the user. Determining who has an interest in each or both types of expression can get confusing. For example, programmers may have a free speech interest in both the code necessary for the bot to interact with humans and the required disclosure if the bot is programmed to disclose itself; on the other hand, a nonprogrammer bot user may have a free speech interest in any expression that directs the bot^[80] or any incidental text included in a bot profile to meet the disclosure requirement. And what happens if the user is the programmer or if there are multiple users and multiple programmers? Tracing back each expression to its source may be necessary before an individual can make a First Amendment claim.

Another issue of personhood may arise in a future S.B. 1001 challenge if, for example, a party responding to a bot enforcement action challenges the validity of the law under the First Amendment and also claims to not be liable for the semi-autonomous actions of its computer program. If a programmer or user maintains even a scintilla of control over the actions of a bot, are those actions not still in some way the expression of the programmer? Until it is possible to credibly argue that speech can be completely generated by a machine, the bot programmer/‌‌user will be able to claim personhood for the purposes of triggering constitutional protection. Although, if a First Amendment challenge of S.B. 1001 requires a party to admit to some amount of direct control over a bot, that party may risk proving its own unlawful intent to use the bot.

2. Is Bot Expression Speech?

Some of S.B. 1001’s disclosure options (textual disclosure by the user in particular) should simply be treated as traditional textual speech, but a bot’s machine-generated communication poses interesting new questions for courts. Speech, for the purposes of the First Amendment, can be defined more broadly than simple “spoken or written word.”^[81] But, because not all communication is speech,^[82] the Supreme Court has developed ways of determining which communications qualify as speech.^[83] While some relatively new forms of communication, such as videogames, are assumed to be speech because of their similarity to traditional speech,^[84] forms of speech that are not easily compared to traditional forms may be analyzed with the Spence Test, which examines whether “[a]n intent to convey a particularized message was present, and in the surrounding circumstances the likelihood was great that the message would be understood by those who viewed it.”^[85] The Spence Test may be necessary to analyze the kinds of expression covered by S.B. 1001; for example, a bot need only “interact” with a California resident online to trigger the law.^[86] Does a friend request by a bot on Facebook convey a “particularized message?”^[87] Would a person understand a friend request from a profile to mean that the profile is operated by a person? Maybe.

On a more foundational level, the output generated by a bot is the direct result of programming, and software code is physically the same as traditional written expression.^[88] As such, courts may consider bot code, like encryption source code,^[89] to be speech simply because it is a fixed medium of expression.^[90] Though, since “there is independence between source code and program behavior”^[91] and because a bot program’s substantive output (a Twitter post, a chat reply, a comment in a forum) is not always a direct result of the code, bot code may need to be characterized as a hybrid of traditional, functional, and symbolic speech.^[92] Section III.B.2 will consider how communication generated through the “black box” of bots is similar to using cash as “speech-by-proxy.” Depending on the complexity of a particular piece of bot software, it seems likely that a court would consider bot expression to be the speech of whoever has input parameters necessary to generate that expression.

3. Categorical Inclusion/Exclusion

Another way to analyze First Amendment coverage is to look at categories of speech that the Supreme Court has found to fall within or outside of free speech scrutiny.^[93]

Otherwise Unlawful Speech. The First Amendment does not provide absolute protection for all expression; certain categories of speech are considered unprotected.^[94] Perhaps the simplest way for a bot law to avoid a constitutional challenge would be to specifically target otherwise unlawful speech that courts have recognized as regulatable, such as “true threats”^[95] or fraudulent speech.^[96] Of the general ways that S.B. 1001 may chill free speech mentioned above,^[97] the regulation of deceptive influence on commercial transactions likely falls into a categorical exclusion of free speech coverage. For example, S.B. 1001 uses “intent to mislead”^[98] as a qualifier for the targeted bot speech; this both combats possible overbreadth arguments^[99] and attempts to frame the speech as unlawful and unprotected in the first place. Though the inclusion of an intent element does not magically repel First Amendment scrutiny,^[100] it may persuade a reviewing court that the regulated behavior is not worthy of protection.^[101] Even if a court considers a regulation of deceptive commercial speech, for example, ripe for First Amendment analysis, the targeting of such unlawful speech will help prove a government’s interest in promulgating the law.^[102] Unless a state seeks to only touch on the issue of consumer deception^[103] or hate speech,^[104] the inclusion of political purposes will certainly sink any claim of categorical exclusion for otherwise unlawful speech. For this reason, S.B. 1001, which includes a political prong, will not avoid a free speech challenge by categorical exclusion alone.

Functionality. Within the scope of symbolic conduct, the Supreme Court explicitly distinguishes between expressive communication and functional communication.^[105] In the extreme, conduct that is all function and no expression will not receive First Amendment coverage,^[106] but where should the functionality line be drawn, and can an argument be made that programming a bot is more functional than expressive?^[107] Some commentators suggest that certain types of functionality may be dispositive in triggering First Amendment analysis.^[108] For example, Tim Wu offers two ways to analyze the relationship between programmers and machine speech as functional expression: conduits/‌‌carriers and communication tools.^[109]

First, conduits/carriers “handle or transform” information without directly controlling the content of the expression.^[110] For example, television broadcast networks have “editorial discretion” over the content of the broadcast—this triggers First Amendment coverage,^[111] but cellular networks only provide a conduit for expression to move within the service—this does not trigger First Amendment coverage.^[112] Depending on how a bot is programmed and to what extent the bot operates autonomously, an argument that a bot is simply broadcasting with “editorial discretion” may be credible. For example, if a social media bot violates S.B. 1001, but only expresses itself through the content generated by social media users other than the bot’s programmer/‌‌user, that programmer/‌‌user could claim that the bot is merely broadcasting the views of others. Furthermore, any control that the programmer/‌‌user has to shape the tone of the broadcasted content is within the “editorial discretion” of a network operator. On the other hand, a “noneditorial” common carrier argument would only be possible for bots with either complete autonomy or a large number of programmers/‌‌users.

The second way to analyze functionality in bot expression is to characterize it as a mere communicative tool.^[113] The Ninth Circuit has declined to apply First Amendment analysis to maps and navigational charts,^[114] and at least one other court has avoided the constitutional question of whether Google Maps directions could be considered protected speech.^[115] Bots, in their most common form, are essentially tools for extending human access to retrievable information,^[116] so it is plausible to frame the programming of a bot, not as expression, but as the design of a tool that functionally provides information to users.^[117] For example, if a bot user were to unlawfully use a telemedicine helper bot to obtain medical data from human users, California could argue that the intentional deception falls under S.B. 1001, while the bot expression was intended to be functional in its capacity to elicit responses from the human user. However, because S.B. 1001 was written in such a way to avoid sweeping up bot users who have no intention to deceive, California would only be able to use a functionality argument against a small sliver of bots—those that are both functional and misleadingly human.

Other commentators suggest that because software source/‌‌object code “yields both a highly functional result and some measure of fixed expression,”^[118] similar to the burning of a draft card,^[119] the existence of functional elements should not preclude the finding of a speech interest. After all, the presence of a “language” within a bot’s programming ensures that a court will consider its final expression partially based on fixed speech^[120] and minimally expressive.^[121] If S.B. 1001 is challenged by a bot that in some way aggregates human expression or collects data under the guise of a functional service, California could be successful in avoiding First Amendment coverage.

B. Does S.B. 1001 Touch Protected Speech?

Assuming California is unsuccessful at classifying the conduct regulated by S.B. 1001 as outside of First Amendment coverage, the next question is whether S.B. 1001 touches protected speech—the answer will determine the level of scrutiny the law should receive.^[122] Beginning with United States v. Carolene Products Co.,^[123] “strict scrutiny,”^[124] the Court’s highest level of scrutiny, has come to symbolize the “jurisprudential distinction between ordinary rights” and those “liberties entitled to more stringent judicial protection.”^[125] Thus, it is rare that a law restricting free speech, a fundamental American right, survives this scrutiny.^[126] To survive strict scrutiny, a state must both carefully draft its legislation and make strategic arguments before the court.^[127] In our hypothetical challenge, the level of scrutiny applied by the court may be based on whether S.B. 1001 is found to regulate content-based speech, political speech, or compelled speech.

1. Content-Based Speech

S.B. 1001 provides boundaries for how bots interact with humans, but some may argue that it actually controls the content of bot expression. When legislation touches speech, courts will “distinguish between content-based and content-neutral regulations of speech.”^[128] For example, there is difference between regulating what words can be printed on a sign and where that sign can be located; both regulations touch speech, but only one directly controls the content. On this issue, the U.S. Supreme Court’s First Amendment jurisprudence is clear; content-based regulation of speech is “presumptively invalid.”^[129] Because courts judge content-based laws under strict scrutiny^[130] but content-neutral laws under a lesser standard,^[131] the ultimate fate of legislation can hinge on the ability of a state to portray its legislation as content-neutral.^[132] Drawing the distinction between content-based and content-neutral is not always clear,^[133] but the essential question is “whether the government has adopted a regulation of speech because of [agreement or] disagreement with the message it conveys.”^[134]

California’s strongest argument is that S.B. 1001 targets a particular modality rather than a particular viewpoint.^[135] Restricting the ability of bots to pose as humans online, like restricting the use of shooting-target effigies for example, may incidentally regulate certain content commonly associated with the modality, but it arguably does not target the content itself. ^[136] And unlike in Carey v. Brown, where a Chicago law prohibited picketing in neighborhoods unless related to labor protests outside businesses,^[137] there is no specific viewpoint that would allow a bot user to circumvent regulation—only a factual disclosure by the bot or bot user. Opponents may argue that requiring disclosure of an automated account effectively creates a content-based restriction because S.B. 1001 discriminates in favor of bot-human interaction that includes speech about the nature of the bot. But, unlike a nondisclosure rule, a disclosure requirement does not suppress a bot user’s viewpoint.^[138]

Using the comparisons to broadcasters discussed in the previous Section, California may also be able to compare S.B. 1001 to legislation requiring cable networks to include local stations. In Turner Broadcasting System v. FCC, the Supreme Court held that the “must-carry” provisions of the Cable Television Consumer Protection and Competition Act of 1992 requiring cable networks to carry local stations were content-neutral and deserving of intermediate scrutiny.^[139] On its face, a law that has the effect of interfering with the editorial discretion of a content provider can be considered content-neutral if the interference “does not depend upon the content.”^[140] The requirement of S.B. 1001, that a bot not interact with a human with intent to mislead about its artificial nature without disclosure, may interfere with the bot user’s presentation of the bot’s output, but it does not directly discriminate against the viewpoints of the bot user.

The question of whether a regulation is content-neutral is also related to governmental motive. In R.A.V. v. City of Saint Paul, Justice Scalia stated that “[t]he government may not regulate [speech] based on hostility—or favoritism—towards the underlying message expressed.”^[141] This statement gives credence to the idea that a governmental motive analysis is a question of both coverage and protection,^[142] embracing the “negative view” of the First Amendment, “the need to constrain the government’s potentially dangerous exercise of control over expression.”^[143] In a 1996 article, Justice Kagan suggests a helpful definition for impermissible motive: “[T]he government may not restrict expressive activities because it disagrees with or disapproves of the ideas espoused by the speaker; it may not act on the basis of a view of what is a true (or false) belief or a right (or wrong) opinion.”^[144]

California may argue that requiring bot users to disclose the use of bots is still content-neutral because it is motivated by a content-neutral purpose.^[145] In legislative reports concerning S.B. 1001, lawmakers made a point to distinguish between two possible motives for regulating bots: instead of being motivated by the assumption that “the artificial identity of a bot makes the speech inherently less trustworthy,” California lawmakers believe that “certain communications made by persons through bots are intended to mislead and disclosures of such bot-made or bot-facilitated speech are intended to alert and inform users so that the users can more readily filter out those misleading communications from the nonmisleading variety.”^[146] In this way, S.B. 1001 is concerned with the “how” of online communication, not the “what.” Under the law, bots and, by extension, programmers/‌‌users may freely express any viewpoint as long as it is clear that a bot is speaking. Yet, the burden of a compelled disclosure is not imposed on bots that express messages unrelated to commercial transactions or politics.

Even if S.B. 1001 may be characterized as discriminatory against certain content, at least some of that content (deceptive commercial speech) has traditionally been considered unprotected under the First Amendment.^[147] California has a strong argument that a regulation of deceptive bots, though perhaps a “novel restriction on content,” is part of a “tradition of proscription.”^[148] If opponents of the law are persuasive that the disclosure requirement indirectly causes viewpoint discrimination, the state may still have a compelling interest in regulating such content.

2. Political Speech

The next major distinction that could influence whether S.B. 1001 receives strict or lesser scrutiny is the political nature of the restriction.^[149] While the commercial harm to be addressed by S.B. 1001—deception as to whether a human is engaging in a commercial transaction—is arguably produced by speech in pursuit of solely unlawful ends,^[150] the democratic/‌‌political harm—bots posing as humans to influence votes—is produced by speech in pursuit of both lawful and unlawful ends, thus touching at least some protected speech.^[151] However, S.B. 1001’s political prong may have a chance of avoiding strict scrutiny since disclosure requirements, even in the context of political speech, have traditionally been viewed as an acceptable way to mitigate the risk of deception in political campaigns.^[152]

While opponents of S.B. 1001 may have a powerful argument that restricting political expression through the use of bots will have a chilling effect on opportunities for political advocacy, the Supreme Court, in both U.S. v. Harriss^[153] and Citizens United v. FEC,^[154] has upheld regulations requiring disclosure, even if the regulation concerns direct or indirect political advocacy.^[155] Requiring bots to disclose themselves is similar to requiring a political action committee to disclose itself as the creator of a television ad.^[156] Proponents of S.B. 1001 may argue that disclosure of a bot’s autonomous nature is less offensive to First Amendment principles than the disclosure of political donors; S.B. 1001 only requires the puppet to acknowledge its strings, not point at the puppet master. If such a light disclosure is the only thing required of bot users, California may be able to argue that political speech will only be incidentally chilled by requiring disclosure.

Another way of softening the harshness of First Amendment jurisprudence concerning restriction of political speech is to analogize S.B. 1001’s political prong to political action committee (PAC) contributions. In particular, a strong argument can be made that bot programs, with their “black box” qualities,^[157] are more akin to financial contribution speech^[158] or “speech by proxy,”^[159] and less like political expenditures.^[160] In past political spending First Amendment cases, the government has attempted to distinguish between limitations on direct spending (direct “speech-by-proxy”), unconstitutional under Buckley,^[161] and “limitations on contributions to permanent committees,” proxy speech made to an autonomous committee.^[162] The Supreme Court has not provided a clear standard for determining when contribution to an autonomous political group becomes the speech of the group and not the financial donor.^[163] Using the view of the Court expressed in Buckley, that political contribution is protectable speech, a programmer’s code is like a financial contribution and the bot’s end expression is like a PAC’s support of a candidate.^[164] Similar to how cash, a neutral vehicle for the expression of the spender, can filter through an agent to create speech, a programmer’s ones and zeroes are inherently void of expressive content until the bot acts as the agent.^[165] The programming of a bot is more comparable to a political contributor giving funds to a multi-candidate committee, from which the output may be less predictable or controllable by the individual contributor.^[166]

3. The Right to Not Speak

To convince the court to use strict scrutiny, especially when concerning the disclosure requirement of S.B. 1001, opponents will likely point to precedent holding that individuals have a right to not speak. For example, in Riley v. National Federation of the Blind,^[167] a North Carolina law required that solicitors of charitable donations disclose potential donors before an appeal for funds, “essentially mandat[ing] speech by private persons that they would not otherwise make, thereby altering the content of that speech.”^[168] The Supreme Court decided that even though the compelled speech touched commercial speech, it still should receive strict scrutiny.^[169] This case in particular gives opponents an effective argument against S.B. 1001’s disclosure requirement, but it is unclear whether courts will accept comparisons between the Bill’s disclosure requirement and the kind of required disclosure in Riley;^[170] giving the names of potential donors to a charitable cause is both a mixed purpose speech (not purely commercial) and more onerous than disclosing the artificial identity of a bot. Proponents of S.B. 1001 could easily argue that the disclosure of bot-ness is not content-based, but rather identifying.

C. Would S.B. 1001 Survive Strict Scrutiny?

In our hypothetical challenge, if S.B. 1001 is deemed either a content-based speech restriction or, more specifically, a restriction on protected political speech, the court will apply a strict scrutiny analysis.^[171] Under strict scrutiny, only laws narrowly tailored to achieve a compelling governmental interest will be upheld.^[172] In other words, a government facing strict scrutiny must be able to show that the targeted harm is serious and that the chosen solution is the least restrictive way to regulate.^[173]

1. Compelling Interest

The strength of California’s interest in bot regulation depends on the specificity with which a court chooses to frame the interest. If a court approaches the interest analysis in a broad, surface-level way,^[174] California may be able to characterize its interest as generally concerning the integrity of elections and democratic processes.^[175] If a court chooses to more specifically define the governmental interest as an interest in preventing bot influence on the democratic process, it will be more important for California to provide evidence of how bots can be used to manufacture consensus and spread false information within the state itself. To California’s credit, the legislative history of S.B. 1001 shows significant consideration by lawmakers to address the real harm of malicious bots.^[176] But, convincing the judiciary that unregulated automation technology poses a serious concrete harm may prove to be tricky in a court that is already persuaded to find bot disclosure an infringement of protected speech. This Author believes that the evidence weighs in California’s favor. And after all, those who favor a more hands-on approach to emerging technology law need not solve the digital competency problem in our legal institutions to prove that California has a compelling interest in preventing bot harm.

2. Narrowly Tailored

To meet the narrowly tailored prong of the strict scrutiny analysis, California’s argument of least resistance should involve strong comparisons between the speech regulated by S.B. 1001 and other permissible forms of required disclosure. As mentioned above, the Supreme Court has previously recognized, in both Buckley^[177] and Citizens United,^[178] that campaign finance disclosure is a relatively “less restrictive”^[179] option for preventing the harm of deception and corruption in the political process. S.B. 1001’s aim (prevention of deception) and method (disclosure) have already been conceptually approved at the highest levels of the judiciary.

Changes made before the final passage of S.B. 1001 alleviated the risk of an overbreadth challenge by narrowing the law to only regulate bots being used with intent to cause the harm California seeks to prevent.^[180] A law can be deemed overbroad if it regulates more speech than the Constitution allows;^[181] furthermore, a law risks invalidation “if the overbreadth is ‘substantial.’”^[182] California lawmakers effectively limited overbreadth concerns in two distinct ways. First, committee-level changes to mens rea terms help to only target those with “intent to mislead” about the artificial personhood of a bot.^[183] There is no danger of making unintentional confusion over the nature of a bot into an unlawful offense for the innocent bot user. Second, other committee changes helped to target content that specifically falls under otherwise unlawful behavior.^[184]

Overall, lingering questions persist as to whether the specific mechanisms of S.B. 1001 are properly tailored to address the risks associated with unregulated bots. For example, could California act to prevent bot harm that does not include the impinging of speech? Will transparency by bots even prevent human users from being influenced by bot accounts? The fact that these questions are not easily answered may cause problems for California in a strict scrutiny challenge. One thing is for certain; outside of regulating online platforms themselves, compelling bot accounts to disclose their artificial nature is the most direct way to address the problem.

IV. Recommendation

Ultimately, this Author believes that the type of expression regulated by S.B. 1001 will receive a lesser scrutiny and be declared constitutional on its face. Like the categories of defamation and obscenity, automated speech as a category does not contribute such a significant net positive to the marketplace of ideas that its regulation should be considered constitutionally protected.^[185] Though, because the application of bot technology, and thus the speech interests of bot users and programmers, will vary depending on the level of bot autonomy, there is a chance that S.B. 1001 may be found unconstitutional as applied to a particular kind of bot use. This final Section discusses some strategies for improving future bot regulation.

A. Expanding S.B. 1001’s Commercial Prong

As this Comment has shown, while California may effectively argue that S.B. 1001 is merely regulating the method of political communication rather than the content, the plain language of the Bill seems to target political speech. Without the political prong, S.B. 1001 would not be nearly as constitutionally beguiling. If it disappeared, would we miss it? Maybe not. S.B. 1001 could be effective at combating the democratic harms of bots without explicitly targeting expression with political purposes. S.B. 1001’s commercial prong targets the deceptive use of automated profiles to “incentivize a purchase or sale of goods or services in a commercial transaction,” but this language does not expressly include a significant way that bot users may deceptively profit from human users: clicks. As addressed in the shocking Wired article about Macedonian fake-news farms,^[186] one of the biggest challenges to combating fake news and misinformation is the profitability of using incendiary/‌‌false headlines or social posts to drive web traffic and increase ad revenue.^[187]

It may be impossible to measure how much of the disinformation economy is motivated by the desire to make a quick buck and how much is created to intentionally damage democratic processes, but it may be almost as difficult to prove a bot user has violated S.B. 1001’s mens rea requirements. By removing the political prong from the Bill, California could target the deceptive use of bots to generate ad dollars (an undeniably content-neutral aim) and consequently prevent bot users from profiting from divisive political content without the need to prove an intent to deceive for the purposes of effecting a vote. Under the current law, what would stop a potentially unlawful bot user from hiding behind their intention to get clicks in the first place? Expanding the commercial prong to include web traffic under the definition of a transaction could both allow lawmakers to drop the political prong and prevent a loophole.

B. Moving Beyond Transparency

While S.B. 1001’s success will be more accurately judged when the law is first implemented and when other laws are created on the subject of bots, in the long term, this Author is not convinced that transparency will be enough to avoid the potential harm of bots in the future, especially without corresponding education about the power and scope of AI-driven technologies. Social networks are already transparent about how they make their money—they sell our online decisions, preferences, and identities to marketers—yet we continue to make choices that have serious privacy-destroying consequences. Transparency, while elegant and useful for lawmakers seeking to make constitutionally-rubberstamped disclosure arguments, puts responsibility on individuals to control aspects of the digital environment that are out of individual control and can only be changed through concerted network-level action.

While S.B. 1001 may not totally solve the problem of unregulated online bots, it is necessary for governments to at least start and act, if only to build public confidence that the issue is not out of control. If left to the social networks and internet service providers, the regulation of bots will not happen until it is directly profitable or until their ubiquity is an existential threat—a time that may never come for some of the larger tech firms. Because S.B. 1001 only touches the tech companies if they use their own bots to intentionally mislead California citizens, the law (disappointingly) creates no accountability on the part of the networks that facilitate the use of bots. In order to more effectively prevent the types of harm at the core of S.B. 1001, a future bot law must not only go after users, but must also go after social networks for not taking steps to prevent the behavior themselves. This would certainly be difficult to achieve on a state-by-state basis.

S.B. 1001 may serve as a limited first strike in a nationwide political battle over the unchecked power of technology. If the mood in Washington concerning social networks sours during the 2020 campaign,^[188] and if S.B. 1001 can be shown to have some impact, it would not be surprising to see something like 2018’s Senate-generated Bot Disclosure and Accountability Act get passed. On a more ambitious level, the federal government could create an agency to oversee internet communities and their impact on our social wellbeing, giving it the power to promulgate rules for websites that allow for the use of bots. If a federal agency were given a broad mandate by Congress to regulate issues of informational legitimacy and privacy among the major online platforms, that agency could reasonably justify a conclusion that the marketplace of ideas does not benefit from a share-based media system that favors parties with the resources to manipulate the system.^[189] The harms associated with retreating internet privacy and increasing corporate control of our personal data are intertwined with the issue of computers being used to manipulate public opinion; ultimately, a comprehensive federal agency would have the competence and nimbleness to handle smaller policy issues as they pop up across the country.

Matthew Hines