generate buyer journeys that seem easy and interesting, however evaluating whether or not these journeys are structurally sound stays difficult for present strategies.
This text introduces Continuity, Deepening, and Development (CDP) — three deterministic, content-structure-based metrics for evaluating multi-step journeys utilizing a predefined taxonomy somewhat than stylistic judgment.
Historically, optimizing customer-engagement programs has concerned fine-tuning supply mechanics comparable to timing, channel, and frequency to attain engagement and enterprise outcomes.
In apply, this meant you skilled the mannequin to grasp guidelines and preferences, comparable to “Don’t contact prospects too typically”, “Consumer Alfa responds higher to cellphone calls”, and “Consumer Beta opens emails largely within the night.”
To handle this, you constructed a cool-off matrix to stability timing, channel constraints, and enterprise guidelines to manipulate buyer communication.
Up to now, so good. The mechanics of supply are optimized.
At this level, the core problem arises when the LLM generates the journey itself. The difficulty is not only about channel or timing, however whether or not the sequence of messages kinds a coherent, efficient narrative that meets enterprise goals.
And abruptly you notice:
There isn’t any normal metric to find out if an AI-generated journey is coherent, significant, or advances enterprise targets.
What We Anticipate From a Profitable Buyer Journey
From a enterprise perspective, the sequence of contents per journey step can’t be random: it should be a guided expertise that feels coherent, strikes the client ahead via significant levels, and deepens the connection over time.
Whereas this instinct is frequent, it’s also supported by customer-engagement analysis. Brodie et al. (2011) describe engagement as “a dynamic, iterative course of” that varies in depth and complexity as worth is co-created over time.
In apply, we consider journey high quality alongside three complementary dimensions:
Continuity — whether or not every message suits the context established by prior interactions.
Deepening — whether or not content material turns into extra particular, related, or personalised somewhat than remaining generic.
Development — whether or not the journey advances via levels (e.g., from exploration to motion) with out pointless backtracking.
Why Present LLM Analysis Metrics Fall Quick
If we take a look at normal analysis strategies for LLMs, comparable to accuracy metrics, similarity metrics, human-evaluation standards, and even LLM-as-a-judge, it turns into clear that none present a dependable, unambiguous strategy to consider buyer journeys generated as multi-step sequences.
Let’s look at what normal buyer journey metrics can and may’t present.
Accuracy Metrics (Perplexity, Cross-Entropy Loss)
These metrics measure confidence stage in predicting the subsequent token given the coaching information. They don’t seize whether or not a generated sequence kinds a coherent or significant journey.
Similarity Metrics (BLEU, ROUGE, METEOR, BERTScore, MoveScore)
These metrics examine the generated end result to a reference textual content. Nonetheless, buyer journeys not often have a single right reference, as they adapt to context, personalization, and prior interactions. Structurally legitimate journeys might differ considerably whereas remaining efficient.
Undoubtedly, semantic similarity has its benefits, and we’ll use cosine similarity, however extra on that later.
Human Analysis (Fluency, Relevance, Coherence)
Human judgment typically outperforms automated metrics in assessing language high quality, however it’s poorly suited to steady journey analysis. It’s costly, suffers from cultural bias and ambiguity, and doesn’t perform as a everlasting a part of the workflow however somewhat as a one-time effort to bootstrap a fine-tuning stage.
LLM-as-a-Choose (AI suggestions scoring)
Utilizing LLMs to guage outputs from different LLM programs is a formidable course of.
This strategy tends to focus extra on fashion, readability, and tone somewhat than structural analysis.
LLM-as-a-Choose might be utilized in multi-stage use circumstances, however outcomes are sometimes much less exact as a result of elevated danger of context overload. Moreover, fine-grained analysis scores from this methodology are sometimes unreliable. Like human evaluators, LAAJ additionally carries biases and ambiguities.
A Structural Strategy to Evaluating Buyer Journeys
In the end, the first lacking ingredient in evaluating beneficial content material sequences inside the buyer journey is construction.
Probably the most pure strategy to symbolize content material construction is as a taxonomic tree, a hierarchical mannequin consisting of levels, content material themes, and ranges of element.
As soon as buyer journeys are mapped onto this tree, CDP metrics might be outlined as structural variations:
- Continuity: easy motion throughout branches
- Deepening: shifting into extra particular nodes
- Development: shifting ahead via buyer journey levels
The answer is to symbolize a journey as a path via a hierarchical taxonomy derived from the content material house. As soon as this illustration is established, CDP metrics might be computed deterministically from the trail. The diagram beneath summarizes the whole pipeline.
Establishing the Taxonomy Tree
To guage buyer journeys structurally, we first require a structured illustration of content material. We assemble this illustration as a multi-level taxonomy derived straight from customer-journey textual content utilizing semantic embeddings.
The taxonomy is anchored by a small set of high-level levels (e.g., motivation, buy, supply, possession, and loyalty). Each anchors and journey messages are embedded into the identical semantic vector house, permitting content material to be organized by semantic proximity.
Inside every anchor, messages are grouped into progressively extra particular themes, forming deeper ranges of the taxonomy. Every stage refines the earlier one, capturing growing topical specificity with out counting on guide labeling.
The result’s a hierarchical construction that teams semantically associated journey messages and gives a steady basis for evaluating how journeys circulation, deepen, and progress over time.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is established, particular person buyer journeys are mapped onto it as ordered sequences of messages. Every step is embedded in the identical semantic house and matched to the closest taxonomy node utilizing cosine similarity.
This mapping converts a temporal sequence of messages right into a path via the taxonomy, enabling the structural evaluation of journey evolution somewhat than treating the journey as a flat listing of texts.
Defining the CDP Metrics
The CDP framework consists of three complementary metrics: Continuity, Deepening, and Development. Every captures a definite side of journey high quality. We describe these metrics conceptually earlier than defining them formally primarily based on the taxonomy-mapped journey.
Setup and Computation
Earlier than analyzing actual journeys, we make clear two facets of the setup.
(1) how journey content material is structurally represented, and
(2) how CDP metrics are derived from that construction.
Buyer-journey content material is organized right into a hierarchical taxonomy consisting of anchors (L1 journey levels), thematic heads (L2 matters), and deeper nodes that symbolize growing specificity:
Anchor (L1)
└── Head (L2)
└── Baby (L3)
└── Grandchild (L4+)As soon as a journey is mapped onto this hierarchy, Continuity, Deepening, and Development are computed deterministically from the journey’s path via the tree.
Let a buyer journey be an ordered sequence of steps:
J = (x₁, x₂, …, xₙ)
Every step xᵢ is assigned:
- aᵢ — anchor (L1 journey stage)
- tᵢ — thematic head (L2 subject), the place tᵢ = 0 means “unknown”
- ℓᵢ — taxonomy depth stage (L1 = 0, L2 = 1, L3 = 2, …)
Continuity (C)
Continuity evaluates whether or not consecutive messages stay contextually and thematically coherent.
For every transition (xᵢ →xᵢ₊₁), a step-level continuity rating cᵢ ∈ [0, 1] is assigned primarily based on taxonomy alignment, with increased weights given to transitions that keep inside the similar subject or carefully associated branches.
Transitions are ranked from strongest to weakest (e.g., similar subject, associated subject, ahead stage transfer, backward transfer), and
assigned lowering weights:
1 ≥ α₁ > α₂ > α₃ > α₄ > α₅ > α₆ ≥ 0
The general continuity rating is computed as:
C(J) = (1 / (n − 1)) · Σ cᵢ for i = 1 … n−1
Deepening (D)
Deepening measures whether or not a journey accumulates worth by shifting from basic content material towards extra particular or detailed
interactions. It’s computed utilizing two complementary elements.
Journey-based deepening captures how depth modifications alongside the noticed path:
Δᵢᵈᵉᵖᵗʰ = ℓᵢ₊₁ − ℓᵢ, dᵢ = max(Δᵢᵈᵉᵖᵗʰ, 0)
D_journey(J) = (1 / (n − 1)) · Σ dᵢ
Taxonomy-aware deepening measures how deeply a journey explores the precise taxonomy tree, primarily based on the heads it visits.
It evaluates how most of the potential deeper content material objects (youngsters, sub-children, and so forth.) beneath every visited head are later seen
through the journey.
D_taxonomy(J) = |D_seen(J)| / |D_exist(J)|
The ultimate deepening rating is a weighted mixture:
D(J) = λ₁ · D_taxonomy(J) + λ₂ · D_journey(J), λ₁ + λ₂ = 1.
Deepening lies in [0, 1].
Development (P)
Development measures directional motion via journey levels. For every transition, we compute:
Δᵢ = aᵢ₊₁ − aᵢ.
Solely shifting steps (Δᵢ ≠ 0) are thought-about. Let wᵢ denote the relative significance of the present stage.
If Δᵢ > 0 (ahead motion):
cᵢ = wᵢ / Δᵢ
If Δᵢ < 0 (backward motion):
cᵢ = Δᵢ · wᵢ
The uncooked development rating is:
P_raw(J) = Σ cᵢ for all i the place Δᵢ ≠ 0
To sure the rating to[−1, +1], we apply a tanh normalization:
P(J) = (e^(P_raw) − e^(−P_raw)) / (e^(P_raw) + e^(−P_raw))
Making use of CDP Metrics to an Automotive Buyer Journey
To show how structured analysis works on real looking journeys, we generated an artificial automotive customer-journey dataset masking the primary levels of the client lifecycle.
Enter Knowledge: Anchors and Journey Content material
The CDP framework makes use of two primary inputs: anchors, which outline journey levels, and customer-journey content material, which gives the messages to guage.
Anchors symbolize significant phases within the lifecycle, comparable to motivation, buy, supply, possession, and loyalty. Every anchor is augmented with a small set of consultant key phrases to floor it semantically. Anchors serve each as reference factors for taxonomy development and because the anticipated directional circulation used later within the Development metric.
anchor Phrases:
motivation exploration analysis discovery curiosity take a look at drive wants evaluation expertise
buy financing comparability quotes mortgage negotiation credit score pre-approval deposit
supply paperwork signing deposit logistics handover activation
possession upkeep guarantee restore seller assist service inspections
loyalty suggestions satisfaction survey referral improve retention advocacyBuyer-journey content material consists of brief, action-oriented CRM-style messages (emails, calls, chats, in-person interactions) with various ranges of specificity and spanning a number of levels. Though this dataset is synthetically generated, anchor info shouldn’t be used throughout taxonomy development or CDP scoring.
CJ messages:
Discover fashions that match your way of life and private targets.
Take a digital tour to find key options and trims.
Evaluate physique types to evaluate house, consolation, and utility.
E book a take a look at drive to expertise dealing with and visibility.
Use the wants evaluation to rank must-have options.
Filter fashions by vary, mpg, or towing to slim decisions.Taxonomy Building Outcomes
Right here, we utilized the taxonomy development course of to the automotive customer-engagement dataset. The determine beneath exhibits the ensuing customer-journey taxonomy, constructed from message content material and anchor semantics.
Every top-level department corresponds to a journey anchor (L1), which represents main journey levels comparable to Motivation, Buy, Supply, Possession, and Loyalty.
Deeper ranges (L2, L3+) group messages by thematic similarity and growing specificity.
What the Taxonomy Reveals
Even on this compact dataset, the taxonomy highlights a number of practical patterns:
- Early-stage messages cluster round exploration and comparability, steadily narrowing towards concrete actions comparable to reserving a take a look at drive.
- Buy-related content material separates naturally into monetary planning, doc dealing with, and finalization.
- Possession content material exhibits a transparent development from upkeep scheduling to diagnostics, value estimation, and guarantee analysis.
- Loyalty content material shifts from transactional actions towards suggestions, upgrades, and advocacy.
Whereas these patterns align with how practitioners sometimes motive about journeys, they come up straight from the info somewhat than from predefined guidelines.
Why This Issues for Analysis
This taxonomy now gives a shared structural reference:
- Any buyer journey might be mapped as a path via the tree.
- Motion throughout branches, depth ranges, and anchors turns into measurable.
- Continuity, Deepening, and Development are now not summary ideas; they now correspond to concrete structural modifications.
Within the subsequent part, we use this taxonomy to map actual journey examples and compute CDP scores in steps.
Mapping Buyer Journeys onto the Taxonomy
As soon as the taxonomy is constructed, evaluating a buyer journey turns into a structural downside.
Every journey is represented as an ordered sequence of customer-facing messages.
As an alternative of judging these messages in isolation, we undertaking them onto the taxonomy and analyze the ensuing path.
Formally, a journey J = (x₁, x₂, …, xₙ) is mapped to a sequence of taxonomy nodes: (x₁→v₁),(x₂→v₂),…,(xₙ→vₙ) the place every vᵢ is the closest taxonomy node primarily based on embedding similarity.
A Step-by-Step Walkthrough: From Journey Textual content to CDP Scores
To make the CDP framework concrete, let’s stroll via a single buyer journey instance and present how it’s evaluated step-by-step.
Step 1 — The Buyer Journey Enter
We start with an ordered sequence of customer-facing messages generated by an LLM.
Every message represents a touchpoint in a sensible automotive buyer journey:
journey = ['Take a virtual tour to discover key features and trims.';
'We found a time slot for a test drive that fits your schedule.';
'Upload your income verification and ID to finalize the pre-approval decision.';
'Estimate costs for upcoming maintenance items.';
'Track retention offers as your lease end nears.';
'Add plates and registration info before handover.']Step 2 — Mapping the Journey into the Taxonomy
For structural analysis, every journey step is mapped into the customer-journey taxonomy. Utilizing textual content embeddings, every message is matched to its closest taxonomy node. This produces a journey map (jmap), a structured illustration of how the journey traverses the taxonomy.
Desk 2: Every message is assigned to an anchor (stage), a thematic head, and a depth stage within the taxonomy primarily based on semantic similarity within the shared embedding house. This desk acts as the inspiration for all future evaluations.
Step 3 — Making use of CDP Metrics to This Journey
As soon as the journey is mapped, we compute Continuity, Deepening, and Development deterministically from step-to-step transitions.
Desk 3: Every row represents a transition between consecutive journey steps, annotated with indicators for continuity, deepening, and development.
Remaining CDP scores (this journey):
Taken collectively, the CDP indicators point out a journey that’s largely coherent and forward-moving, with one clear second of
deepening and one seen structural regression. Importantly, these insights are derived solely from construction, not from
stylistic judgments in regards to the textual content.
Conclusion: From Scores to Profitable Journeys
Continuity, Deepening, and Development are decided by construction and might be utilized wherever LLMs generate multi-step
content material:
- to match different journeys generated by completely different prompts or fashions,
- to supply automated suggestions for bettering journey technology over time.
On this approach, CDP scores provide structural suggestions for LLMs. They complement, somewhat than exchange, stylistic or fluency-based analysis by offering indicators that mirror enterprise logic and buyer expertise.
Though this text focuses on automotive commerce, the idea is broadly relevant. Any system that generates ordered, goal-oriented content material requires robust structural foundations.
Massive language fashions are already able to producing fluent, persuasive textual content.
The better problem is making certain that textual content sequences kind coherent narratives that align with enterprise logic and consumer expertise.
CDP gives a strategy to make construction specific, measurable, and actionable.
Thanks for staying with me via this journey. Hopefully, this idea helps you assume in a different way about evaluating AI-generated sequences and evokes you to deal with construction as a main sign in your personal programs. All logic offered on this article is carried out within the accompanying Python code on GitHub. In case you have any questions or feedback, please go away them within the feedback part or attain out by way of LinkedIn.
References
Brodie, R. J., et al. (2011). Buyer engagement: Conceptual area, elementary propositions, and implications for analysis.
