LLM Themes Are Not Observations

themes from a name corpus to the shopper desk. Prospects with out transcripts get NULL. NULL will get crammed with zero, or with “no problem talked about,” or quietly omitted as a reference class. In a single line of preprocessing, the pipeline converts didn’t name help into didn’t expertise billing frustration.

The regression that follows seems clear. The coefficient on “billing frustration” is critical, signed the way in which the product staff anticipated, giant sufficient to matter. It will get pasted right into a roadmap doc. No person asks the place the variable got here from.

This text is about what obtained smuggled in with that fill worth, and about three different strikes that look simply as innocuous in a pocket book however relaxation on assumptions the evaluation by no means names. The setup just isn’t particular to help calls. It applies to speak logs, ticket summaries, product critiques, gross sales transcripts, and survey free-response fields, anyplace a contemporary pipeline turns textual content right into a tidy column. The pipeline might be a fine-tuned classifier, a zero-shot LLM, or an embedding-plus-cluster. The conceptual downside is identical: the column just isn’t an remark of a buyer attribute. It’s the output of a generative course of utilized to a self-selected subset of buyer conduct.

Practitioners more and more deal with outputs like these as in the event that they have been direct readings of buyer state. They don’t seem to be. They’re generated variables: measurements produced by a pipeline, conditional on a buyer doing one thing that left a textual hint, conditional on that hint surviving the extraction mannequin. Each step of that conditional has penalties for what the variable means in a downstream causal mannequin, and most of these penalties are invisible within the joined desk.

4 issues are likely to go incorrect, and the NULL transfer makes all 4 seen without delay.

Choice. A theme exists for a buyer as a result of that buyer referred to as, complained, posted, or replied. No matter drove that motion can be in all probability correlated with the remedy, the result, or each. The NULL fill collapses “didn’t generate textual content” into the reference class, and the evaluation is not estimating an impact over the shopper base. It’s estimating an impact over a redefined inhabitants, and the redefinition occurred in preprocessing.

Timing. Was the decision earlier than the remedy, throughout it, or after? Pre-treatment textual content is a candidate confounder. Submit-treatment textual content is a candidate mediator or end result, and treating it as a pre-treatment management is a basic supply of post-treatment bias. The joined desk not often makes this seen.

Measurement. The label “billing frustration” just isn’t billing frustration. It’s what the pipeline detected as billing-frustration-shaped language. Classifier accuracy is finite, and accuracy can differ throughout remedy arms, as a result of a remedy that modifications how prospects speak additionally modifications how the mannequin reads them. The label noise just isn’t orthogonal to the factor being studied.

Function. Is the theme performing as a confounder, a mediator, a remedy, an end result, or a descriptive characteristic? The DAG decides this, not the column identify. A variable that’s methodologically legitimate in a single function turns into a bias supply in one other.

These 4 issues aren’t impartial. They work together. An LLM-detected theme inherits a variety footprint from the channel it got here by means of, a timing footprint from when the textual content was generated, and a measurement footprint from the pipeline that extracted it. The downstream regression sees a column of zeros and ones.

The issue just isn’t that the pipeline produced a nasty label. The issue is that the label inherited a data-generating course of the downstream evaluation by no means modeled.

The remainder of this text works by means of what meaning in follow, the place the usual workflow goes incorrect, and what the minimal diagnostic seems like. We begin with the role-and-timing query, as a result of it’s the one analysts get incorrect first.

Function and timing are the identical query

The primary transfer an analyst makes with a transcript-derived theme is implicit: they deal with it as a covariate. Themes go into the right-hand aspect of the regression. The remedy is the variable of curiosity. The end result is on the left. The theme is “managed for.”

That phrase, “managed for,” is doing work the analyst hasn’t checked. Controlling for a variable adjusts away the a part of the treatment-outcome relationship that flows by means of it. Whether or not that adjustment helps or hurts relies upon completely on the place the variable sits within the causal graph, and that place is decided by timing.

Pre-treatment textual content, generated earlier than the remedy was assigned, can play the function of a confounder. If a buyer referred to as about billing in January and the retention provide went out in March, the decision captures one thing about buyer state which will affect each who obtained the provide and who churned. Conditioning on the theme right here can cut back bias from omitted variables, offered the theme truly proxies for the related assemble and the choice points within the subsequent part are dealt with.

Concurrent textual content, generated as a part of the remedy itself, just isn’t a covariate in any respect. If the remedy is a name from a retention agent and the theme comes from that very same name, the theme is a part of the intervention. Conditioning on it doesn’t alter for confounding; it removes a part of the impact the analyst is making an attempt to measure.

Submit-treatment textual content, generated after the remedy, is probably the most harmful class, as a result of it’s the one most definitely to be misclassified as a confounder by an analyst working from a flat desk with no time index. A buyer who obtained a retention provide in March and referred to as complaining in April produced a transcript that displays, at the least partly, their response to the remedy. Conditioning on a theme extracted from that decision is conditioning on a post-treatment variable. That may block mediation paths, induce collider associations, or in any other case shift the estimand away from the remedy impact the analyst thinks they’re estimating.

A labored instance makes this concrete. Think about an artificial however business-realistic setup. Prospects are focused right into a retention provide primarily based on a mannequin that picks up worth sensitivity. Each the provide task and buyer churn rely on this underlying worth sensitivity, which the analyst doesn’t observe. Prospects who’re extra price-sensitive usually tend to obtain the provide (as a result of the concentrating on mannequin chosen them) and extra more likely to churn regardless. They’re additionally extra more likely to name help and categorical invoice shock. The theme “invoice shock” is generated from these post-treatment calls.

The naive analyst joins the theme onto the shopper desk, fills NULL as zero, and runs a logistic regression of churn on provide plus bill-shock:

import numpy as np
import pandas as pd
import statsmodels.api as sm
 
rng = np.random.default_rng(7)
n = 20000
 
price_sens = rng.regular(0, 1, n)
provide = rng.binomial(1, 1 / (1 + np.exp(-(0.8 * price_sens))))
churn = rng.binomial(1, 1 / (1 + np.exp(-(-1.0 + 1.2 * price_sens - 0.5 * provide))))
referred to as = rng.binomial(1, 1 / (1 + np.exp(-(-1.5 + 0.7 * price_sens + 0.9 * churn))))
 
theme_prob = 1 / (1 + np.exp(-(-0.5 + 0.8 * price_sens)))
bill_shock = np.the place(referred to as == 1, rng.binomial(1, theme_prob), 0)
 
df = pd.DataFrame({"churn": churn, "provide": provide, "bill_shock": bill_shock})
 
X = sm.add_constant(df[["offer", "bill_shock"]])
naive = sm.Logit(df["churn"], X).match(disp=0)
print(naive.params)

The true impact of the provide on churn is −0.50 in log-odds. The provide is meant to scale back churn, and within the data-generating course of it does. Here’s what 4 specs return:

As a result of provide task is already confounded by worth sensitivity, eradicating the dangerous management doesn’t make the design legitimate. It solely removes one extra supply of distortion. Two observations from this desk.

First, the naive specification is incorrect in path. Including the bill-shock management to a mannequin that was already biased flips the signal on the provide coefficient. The product staff studying this output would conclude that retention provides trigger churn. They’d be incorrect.

Second, dropping the bill-shock variable doesn’t repair the evaluation. The dropped specification can be optimistic, and solely the oracle specification, which circumstances on the unobserved confounder instantly, recovers the true impact. In an actual evaluation the analyst doesn’t have that column. That’s the level. Eradicating a nasty management is important however not adequate, and a post-treatment theme extracted from a self-selected calling subpopulation just isn’t an alternative choice to identification.

The mechanism behind the signal flip within the naive specification is price strolling by means of. Churn impacts the probability of calling, as a result of prospects who’re leaving usually tend to name. Invoice-shock is barely noticed for patrons who referred to as, because the theme requires a transcript to exist. Conditioning on bill-shock due to this fact circumstances on a downstream consequence of churn. Amongst prospects with bill-shock equal to at least one, the connection between provide and worth sensitivity has been distorted, as a result of each variables now assist clarify why the shopper ended up flagged. The coefficient on provide absorbs that induced affiliation.

The methodological level generalizes. A transcript-derived variable has a place within the causal graph decided by when the textual content was generated relative to the remedy, who generated it, and what course of produced the label. Function and timing are the identical query considered by means of completely different lenses. These variables include a structural footprint the analyst is accountable for tracing, and the joined desk just isn’t the place the tracing occurs.

The choice query

Most business analyses utilizing help transcripts implicitly redefine the inhabitants from “prospects” to “prospects who generated help language.” The estimand modifications earlier than the regression even begins.

That is the half that tends to matter most in practitioner workflows, and it’s the place the usual workflow is most fragile.

The textual content exists as a result of the shopper did one thing: referred to as, posted, complained, replied. That one thing is a conduct, not a measurement. It’s influenced by buyer traits, by the channel that was accessible, by the urgency of the underlying problem, and infrequently by the remedy itself. None of those are random. None are sometimes orthogonal to the result.

The NULL dealing with choice is the place this turns into operational. There are three widespread strikes, and every carries an assumption.

Filling NULL as zero or “no problem talked about” assumes that not producing textual content is informative concerning the absence of the underlying assemble. The analyst is claiming that prospects who didn’t name didn’t expertise the factor the theme is detecting. For many themes price detecting, that is implausible on its face. Prospects who didn’t name might have skilled billing frustration and resolved it by canceling, by switching to a competitor, by complaining on social media, or by giving up. The zero-fill turns all of those into “no frustration.”

Dropping rows with NULL themes, limiting the evaluation to the calling subpopulation, is at the least trustworthy concerning the inhabitants, however it modifications the estimand. The remedy impact amongst prospects who referred to as just isn’t the remedy impact amongst prospects, and the distinction between the 2 is usually your entire level of the enterprise query. A retention provide’s impact on churn-prone callers is a helpful amount. It isn’t the amount most analyses declare to estimate.

Treating text-presence as a missingness mechanism and making use of inverse chance weighting primarily based on a mannequin of who calls is, methodologically, the proper form of transfer. The catch is the propensity mannequin itself. Modeling who generates textual content requires writing down what drives calling, and that mannequin depends upon demographics, tenure, prior points, remedy publicity, and unmeasured frustration, which is the assemble the theme was supposed to assist measure within the first place. The IPW transfer is principled, and it’s also not often as principled because it seems.

The deeper level is that choice into textual content is a conduct that interacts with the remedy. A retention provide might change calling charges. A pricing change might change grievance charges. A characteristic launch might change the sorts of points prospects articulate. Any of those makes the choice mechanism itself treatment-dependent, which suggests even a wonderfully extracted, completely timed theme is being measured on a inhabitants whose composition shifts with the remedy. Normal observational corrections assume the choice mechanism is secure. When the remedy strikes the choice, the corrections don’t.

None of this implies transcript-derived variables are ineffective. It means the analyst owes the reader an specific assertion of which inhabitants the evaluation is estimating an impact over, what mechanism produced the textual content, and what assumption was made about everybody whose textual content doesn’t exist.

The measurement query

Outdated NLP outputs regarded noisy. TF-IDF weights, sparse key phrase counts, LDA matter vectors: none of them regarded like issues a buyer felt. Practitioners distrusted them by reflex, and that reflex saved quite a lot of dangerous analyses.

LLM outputs don’t look noisy. They seem like latent constructs. A label like “billing frustration” or “belief erosion” or “renewal nervousness” reads like an outline of a buyer’s psychological state. The label is articulate, the classes are semantically coherent, and the failure modes don’t announce themselves within the column. The persuasion downside is actual earlier than the statistical downside begins.

The statistical downside is extra acquainted. An LLM theme is a loud proxy for the underlying assemble. The label “invoice shock” just isn’t invoice shock. It’s what the mannequin determined was bill-shock-shaped language within the transcripts it processed. Classifier accuracy is finite even for clear duties, and the accuracy on the precise inhabitants, not the held-out analysis set, is usually unknown. Plugging a loud proxy right into a regression rather than the true variable attenuates coefficients towards zero in some setups and distorts them in others, relying on whether or not the noise is differential.

Differential measurement error is the place the actual injury lives. If a remedy modifications how prospects speak, and most therapies price operating do, then the classifier’s accuracy on theme detection can differ between remedy and management. A retention provide that softens buyer sentiment might cut back the speed at which the mannequin flags “invoice shock” language with out lowering the underlying frustration. A pricing change that shifts how prospects articulate complaints might transfer classifier accuracy extra in a single arm than the opposite. The label noise is not mean-zero. It’s correlated with the remedy, and conditioning on the noisy label biases the estimated remedy impact in a path the analyst can not simply signal.

There’s a literature on correcting for classifier-induced measurement error. Egami and colleagues develop a split-sample workflow for causal inference with text-discovered measures as therapies or outcomes in “How to Make Causal Inferences Using Texts”. Mozer and colleagues apply text-augmented matching to digital well being data and present how text-based covariates change estimated results in an actual medical research in “Leveraging text data for causal inference using electronic health records”. For the broader panorama, Keith, Jensen, and O’Connor evaluate how textual content has been used to take away confounding throughout functions in “Text and Causal Inference: A Review of Using Text to Remove Confounding from Causal Estimates”. These strategies exist, and they’re price utilizing when the evaluation issues. In addition they require the analyst to acknowledge {that a} label is a measurement with error within the first place, which is the transfer most workflows skip.

The practitioner mistake just isn’t utilizing the label. The practitioner mistake is treating a label that got here out of a generative mannequin as if it have been a column learn off a sensor.

A practitioner guidelines

A causal evaluation that makes use of a generated variable derived from transcripts can nonetheless be defensible. It simply has to reply 5 questions earlier than the regression runs.

1. What function am I assuming this variable performs?

Confounder, mediator, remedy, end result, or descriptive characteristic. The DAG decides. The column identify doesn’t.

2. When was the textual content generated relative to the remedy?

Pre-treatment, concurrent, or post-treatment. If the analyst can not reply this from the info, the variable doesn’t enter the mannequin as a confounder.

3. What choice mechanism produced the textual content, and what am I assuming about everybody whose textual content doesn’t exist?

Zero-fill, drop, IPW: every is an assumption. Decide one and state it.

4. How was the label produced, and will its reliability differ throughout remedy arms?

If the remedy plausibly modifications how prospects categorical the underlying assemble, classifier accuracy just isn’t fixed throughout the comparability the evaluation is making.

5. What does the consequence seem like beneath a stress take a look at?

Refit the mannequin with out the transcript-derived variable. If the headline coefficient is fragile, the consequence just isn’t secure sufficient to hold a causal declare by itself.

These 5 questions aren’t an answer. They’re a diagnostic. An analyst who can reply them just isn’t assured an recognized impact. An analyst who can not reply them is doing descriptive work with causal language connected.

The broader sample is older than LLMs. Generated variables are pipeline outputs that seem like observations however are literally mannequin outputs conditioned on choice. They present up in fraud scores, recommender relevance metrics, sentiment indices, propensity scores reused as covariates, and any latent-trait estimate produced by an upstream mannequin and consumed by a downstream evaluation. LLMs didn’t invent this error. They made it accessible at a scale and a fluency that older NLP outputs by no means reached. The labels seem like latent constructs, the columns seem like measurements, and the workflow seems like causal inference.

The assumptions didn’t disappear. They simply moved upstream.

Workers Knowledge Scientist targeted on causal inference, experimentation, and choice science. I write about turning ambiguous enterprise questions into decision-ready evaluation.

Extra like this on LinkedIn 👇

🔗 LinkedIn