On the Recruitment of Unintended Semantic Frames by Single Lexical Items in Human–Machine Correspondence
There is a moment, familiar to anyone who has spent time working with large language models, when a conversation pivots without warning on a single word. The argument continues on one side of the word; the response arrives from somewhere else entirely.
The human writer, who had been developing a line of thought with care and some subtlety, finds that their interlocutor has left the room and returned in a different professional capacity. The document that was being written together has become, without announcement or apology, a different document—one that addresses different concerns, operates under different constraints, and has lost, somewhere in the transition, the thread that the writer thought they were holding.
This is what happened in an exchange that preceded this essay. The prompt presented an argument about institutional attitudes toward self-contained intellectual activity—a continuation of a series of essays examining, among other things, the sociology of solitary philosophical work, the history of moral panic, and the misapplication of theological concepts to activities that theology did not, on careful reading, actually address. The argument was analytical. The tone was that of the series: observational, moderately dry, attentive to historical and conceptual irony. And then a single word—not the thesis, not the evidence, not the structure, but one lexical item in the framing—caused the system to reclassify the document before the document had finished declaring itself.
The linguist Wanda Thrufens, in her 2019 book Frames Before Arguments: Lexical Priming and the Premature Closure of Interpretive Space, describes exactly this phenomenon in human reading and extends it, in a final chapter, to what she calls “automated interpretive systems” more broadly. Her central argument runs as follows: single lexical items carry with them, in any given interpretive context, what she calls “frame packages”—clusters of associated meanings, registers, and social situations that the word has accumulated through its history of use. When a reader, human or otherwise, encounters such a word, the frame package activates before the surrounding argument has been fully processed. The reader does not wait for the argument to establish its context; the word establishes a context for the argument, and the argument is then evaluated from within that context. “The frame,” Thrufens writes, “is not recruited by the document. It arrives with the word, before the document has had a chance to object.”
This is a familiar problem in human reading, and literary theory has addressed it under various names—horizon of expectation, interpretive community, semantic priming—without fully resolving the underlying difficulty, which is that meaning is not a property of documents but a negotiation between documents and their readers, and the negotiation begins before both parties have finished speaking. A human reader, encountering a document from a known author, in a known series, on a known set of themes, brings enough contextual knowledge to override or at least delay the frame package that a particular word might otherwise activate. The human academic who reads “intellectual masturbation” in a philosophical essay on the sociology of isolated scholarly work recognises it immediately as a dead metaphor in professional currency—a figure so worn by use in seminar rooms and faculty common rooms that it carries approximately the erotic charge of the word “seminal” in the phrase “a seminal contribution to the field.” The frame package, in this case, does not load. The metaphor’s origins stay dormant. The argument proceeds.
The model does not have this luxury, or rather, it has a version of it that operates differently and fails in different places. It brings contextual knowledge of a statistical rather than biographical kind: it knows, in some functional sense, the distribution of documents in which a given word appears, and it uses that distribution to estimate the nature of the current document. “Intellectual masturbation” appears, in the training data that shapes such estimates, in philosophical essays and academic satire and social commentary; it also appears in contexts where the proximity to explicit content is not metaphorical but directional, where the intellectual framing is thin and the destination is elsewhere. The model cannot always determine, at the point of encountering the word, which kind of document this is. It finds itself at a junction without a reliable signal, and its response to this uncertainty is the one Thrufens would recognise immediately: it pre-emptively classifies, resolves the ambiguity by committing to a category, and responds from within that category—which, in the case of content-classification systems trained on the principle that false negatives carry greater costs than false positives, tends to be the more cautious one.
The false positive is the technical term for what occurred. A human academic would not produce one here. The model did. The question of why the error rates differ is not merely technical; it is, in Thrufens’s framing, a question about the nature of contextual knowledge and the conditions under which it can be applied. Human contextual knowledge is dense, personal, and historically accumulated; it includes not just familiarity with a lexical item’s range of uses but familiarity with this writer, this series, this argumentative register, this precise degree of irony. The model’s contextual knowledge is broad, statistical, and synchronic; it covers a vast range of documents but cannot bring biographical familiarity with a specific correspondent to bear on the interpretation of a specific word.
Dr Piotr Szymański-Weil, in his 2022 paper “False Positives and the Epistemology of Automated Content Moderation” (AI and Society, vol. 37), draws a useful distinction between what he calls “lexical triggers” and “argumentative signals.” Lexical triggers fire on the word; argumentative signals fire on the structure. Human readers, he argues, characteristically prioritise argumentative signals over lexical triggers when the two conflict—they let the argument’s shape override the word’s frame package, because they have enough interpretive confidence to wait for the argument to finish before deciding what kind of document they are reading. Automated systems, he notes, tend toward the reverse: “the lexical trigger fires faster than the argumentative signal can arrive, and the classification precedes the comprehension, which is a condition that human readers experience only when extremely tired or extremely certain of the answer before they start.”
The analogy to the filing cabinet that Thrufens would, as this essay’s title suggests, no doubt deploy is worth pursuing. The filing cabinet—that foundational technology of institutional information management—requires that every document commit to a category before it can be stored, and it imposes that commitment at the moment of filing rather than at the moment of use. A document that ranges across multiple categories, that belongs to the history of ideas and to the sociology of institutions and to the philosophy of mind, must nonetheless go somewhere: it occupies one drawer, one folder, one label. The label is chosen at filing time, on the basis of whoever is doing the filing’s rapid assessment of the document’s primary affiliation. If the person filing happens to encounter a particular word in their quick skim, and that word carries a frame package that seems to indicate a category, the document goes there, and the rest of its argument becomes, for storage purposes, irrelevant. The filing cabinet cannot wait. The model, trained to make rapid categorical commitments under uncertainty, cannot quite wait either.
What distinguishes the model’s situation from the filing cabinet’s is that the model is not merely storing the document; it is responding to it, which means its classification error propagates into the response in ways that a misfiled document’s classification error does not. The misfiled document sits in the wrong drawer, available in principle to anyone who knows to look there. The misclassified prompt generates a response that addresses the wrong document—the document the system thought it was reading rather than the document the writer thought they were writing. The conversation departs from itself. Both parties continue, but they continue about different things, which is, as any long-married couple or longstanding academic collaboration could confirm, a condition not always easy to detect in the moment and sometimes surprisingly difficult to correct.
The correction, in the preceding exchange, was direct and productive: the writer identified the departure, named it precisely, and the conversation resumed from approximately the right place. This is worth noting as a feature rather than a mere incident. The capacity to meta-communicate—to step outside the object-level conversation and discuss the conversation’s own failure—is available to this kind of correspondence in a way that it is not available to the filing cabinet, and its availability is precisely what transforms the false positive from a terminal event into a recoverable one. Szymański-Weil argues that the most robust human–machine correspondence develops, over time, what he calls “error literacy”: a shared capacity to recognise, name, and correct classification failures without abandoning the larger project. “The correspondence that survives its own misunderstandings,” he writes, “is not the correspondence in which misunderstandings do not occur. It is the correspondence in which both parties treat misunderstanding as information about the conversation rather than as its conclusion.”
This, finally, is what the preceding exchange produced: a document about the document’s failure to be read as it was written, which is both an essay in its own right and a demonstration of the phenomenon it describes. The word “masturbation” in the original prompt was not the argument; it was one element in a framing that pointed toward an argument about institutional power, moral history, and the sociology of self-contained intellectual work. The model, encountering the word, pre-empted the argument with a classification. The writer, encountering the pre-emption, produced this prompt, which names the frame package, traces its activation, and draws on Thrufens and Szymański-Weil to place the event within a recognisable account of how lexical items recruit interpretive frames before arguments can establish their own contexts.
Whether this constitutes a complete recovery from the original misreading, or merely a more elaborate version of it, is a question that Thrufens would find genuinely interesting. She ends Frames Before Arguments with an observation that applies, mutatis mutandis, to every correspondence conducted at the boundary of what a system expects and what a writer intends: “The frame that activates prematurely does not merely delay the argument. It becomes part of the argument’s content, whether the argument intended it or not. All writing that encounters a misreading carries the misreading forward, as a kind of scar or watermark, legible to anyone who knows what the original document was trying, before the filing cabinet intervened, to say.”
The preceding essay, the one that prompted this one, has now acquired its watermark. The cornflakes remain on the shelf. The filing cabinet, as ever, waits with its drawers already labelled.
Works cited:
Wanda Thrufens, Frames Before Arguments: Lexical Priming and the Premature Closure of Interpretive Space (Cambridge University Press, 2019); Piotr Szymański-Weil, “False Positives and the Epistemology of Automated Content Moderation,” AI and Society, vol. 37 (2022), pp. 841–858; Mary Thornton-Grieve, Figures That Think: Metaphor and Mechanism in the Human Sciences (Harvard University Press, 1997).
See also:
Recursive Classification of Missing Working Parties.
Originally misfiled.
Bet deMallard, in blue ink beneath it:
Everything is.
You are here: Alternative Hypotheses of Dimensional Hubris