Conversational Recsys

"A good waiter does not bring the wine list first. A good waiter asks what the table is celebrating."

Pixel, Curious Librarian Agent

38.4.1 Elicitation vs Inference

Traditional recsys infers preferences from behavior. A user who watched three thrillers gets ranked-higher thrillers tomorrow. The inference is statistical, opaque, and slow: the system needs weeks of clicks before it has a confident profile, and the profile updates lag the user's mood. The cold-start problem is the extreme case of this lag: a brand-new user has no history at all.

Conversational recsys flips the script. The system asks. "What kind of mood are you in?" "Older or newer?" "Subtitles okay?" Three turns of dialogue produce a richer preference signal than weeks of click logs, because the user can speak in terms (mood, occasion, constraints) that are impossible to infer from clicks alone. The dialogue elicitation pattern is especially powerful for occasional-use catalogs (movies you watch once a month, restaurants you visit twice a year) where the inference signal is thin even after years of use.

The pure-inference and pure-elicitation extremes are both wrong in practice. Production conversational recsys blends them. The system carries a long-term profile (from inference) into every conversation as a seed, then uses elicitation to refresh and override the profile for the current ask. The waiter analogy from the epigraph applies: a good waiter remembers the regulars' usual order (inference) but still asks what the table is celebrating tonight (elicitation).

38.4.2 The Turn-Taking Pattern

Figure 38.4.1 shows the canonical four-state loop for a conversational recommender. The states are Elicit, Retrieve, Recommend, and Refine. The user can re-enter the Elicit state at any point by changing their mind, refining a constraint, or rejecting all current candidates.

Figure 38.4.1a: The canonical conversational recsys loop. Solid arrows mark the happy path. The red dashed arrow marks a "change of mind" event where the user's feedback invalidates the current preference profile and the loop returns to Elicit.

Each state has its own job. Elicit gathers preferences. The first question is broad ("what are you in the mood for?"); subsequent questions narrow when uncertainty in the implied retrieval set is high. Retrieve calls the recsys pipeline from Sections 38.2 and 38.3 with the current preference profile as the query. Recommend writes a small ranked list (typically 3 to 5 items, not 50) plus a one-sentence justification per item, grounded in the conversation. Refine reads the user's feedback ("the second one looks good, but cheaper please") and decides which state to return to.

38.4.3 Clarifying Questions

The hardest part of the loop is knowing when to ask a clarifying question instead of pushing on. Asking too often (a question per turn) frustrates the user. Asking too rarely produces bad recommendations the user then has to reject manually. The signal that drives the decision is uncertainty over the implied retrieval set.

A simple heuristic that works well: run the retrieval with the current profile. If the top-20 candidates span more than three distinct categories or more than a 4x price range, ask a clarifying question that targets the biggest source of dispersion. If they cluster tightly, just present them. The clarifying-question generator is yet another LLM call, prompted with the dispersed candidate set and asked to phrase the question whose answer would most reduce the dispersion.

from openai import OpenAI
import json
client = OpenAI()

CLARIFY_SYSTEM = """You write ONE short clarifying question for a recommender.
You see the user's conversation so far and the categories of the top-20 candidates.
If the candidates span many categories, ask the question that best partitions them.
Examples of partitioning dimensions: occasion, price range, length, language,
indoor/outdoor, beginner/expert, mood.
Return STRICT JSON: {"question": "...", "partition_dim": "..."}."""

def clarify(conversation: list[dict], top_categories: list[str]) -> dict:
    user = (
        "Conversation so far:\n" + json.dumps(conversation, indent=2)
        + "\nCategories of top-20 candidates:\n" + ", ".join(top_categories)
    )
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": CLARIFY_SYSTEM},
            {"role": "user", "content": user},
        ],
        response_format={"type": "json_object"},
        temperature=0.4,
    )
    return json.loads(resp.choices[0].message.content)

38.4.4 Justified Recommendations

Conversational surfaces make justification a first-class feature. A traditional widget-based recsys returns a list; the user must read the list and infer why each item is there. A conversational recsys writes a sentence per item. The sentence is grounded in the conversation: it cites the user's stated preferences and the item's relevant attributes. The result is both more trustworthy (the user can spot bad reasoning) and more clickable (the user is given a reason to try a non-obvious item).

from openai import OpenAI
import json
client = OpenAI()

JUSTIFY_SYSTEM = """For each of N candidate items, write ONE sentence (max 25 words)
that justifies the recommendation. Ground the sentence in the user's stated
preferences AND in the item's enriched description. Do not invent attributes.
If a stated preference contradicts an item, return the empty string for that item
(the caller will drop it).

Return STRICT JSON: {"justifications": ["...", "...", ...]} in the same order as input."""

def justify(user_preferences: dict, items: list[dict]) -> list[str]:
    user = (
        "User preferences:\n" + json.dumps(user_preferences, indent=2)
        + "\n\nCandidate items:\n"
        + "\n---\n".join(f"{i}. {it['title']}: {it['enriched_text']}" for i, it in enumerate(items))
    )
    resp = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": JUSTIFY_SYSTEM},
            {"role": "user", "content": user},
        ],
        response_format={"type": "json_object"},
        temperature=0.4,
    )
    return json.loads(resp.choices[0].message.content)["justifications"]

38.4.5 A Minimal Conversational Recsys Loop

The four states in Figure 38.4.1 wire together into the loop below. The loop is intentionally written without a framework so the moving parts are visible. The same shape can be expressed with LangGraph nodes (one node per state) or with a state machine library, with no change to the underlying logic.

from dataclasses import dataclass, field
from openai import OpenAI
import json

client = OpenAI()

@dataclass
class Session:
    history: list = field(default_factory=list)         # list of {role, content}
    preferences: dict = field(default_factory=dict)     # accumulated slot fill
    clarify_count: int = 0
    last_candidates: list = field(default_factory=list)

def elicit(session: Session, user_msg: str):
    """Update preferences from the user's latest message via slot filling."""
    # Reuses the slot-filler from Code Fragment 38.2.3 (Section 38.2.3)
    new_slots = fill_slots(user_msg)
    for k, v in new_slots.items():
        if v is not None:
            session.preferences[k] = v
    session.history.append({"role": "user", "content": user_msg})

def retrieve(session: Session, k: int = 20) -> list[dict]:
    """Call the recsys with the current preferences and return top-k candidates."""
    # Reuses the recsys_search helper that wraps retrieve + rerank
    return recsys_search(preferences=session.preferences, k=k)

def should_clarify(candidates: list[dict]) -> bool:
    cats = {c["category"] for c in candidates[:20]}
    return len(cats) > 3  # high dispersion

def step(session: Session, user_msg: str) -> dict:
    elicit(session, user_msg)
    candidates = retrieve(session, k=20)
    session.last_candidates = candidates

    if should_clarify(candidates) and session.clarify_count < 2:
        session.clarify_count += 1
        # Reuses the clarify helper from Code Fragment 38.4.1
        cats = [c["category"] for c in candidates[:20]]
        q = clarify(session.history, cats)["question"]
        session.history.append({"role": "assistant", "content": q})
        return {"type": "question", "text": q}

    top = candidates[:5]
    # Reuses the justify helper from Code Fragment 38.4.2
    justs = justify(session.preferences, top)
    payload = [{"item": it, "why": j} for it, j in zip(top, justs) if j]
    session.history.append({"role": "assistant",
                            "content": json.dumps([p["item"]["title"] for p in payload])})
    return {"type": "recommendations", "items": payload}

# Driver
s = Session()
while True:
    user = input("You: ")
    if not user.strip():
        break
    out = step(s, user)
    if out["type"] == "question":
        print(f"Assistant: {out['text']}")
    else:
        for r in out["items"]:
            print(f"  - {r['item']['title']}: {r['why']}")

38.4.6 The Warm-Conversation UX

A widget-based recsys feels mechanical. Filter chips, dropdowns, sort orders: each is a small adjustment that produces a new list. A conversational recsys feels different because the system remembers the previous turn. "Show me something similar but cheaper" is a natural sentence in a chat surface and almost impossible to express through widgets. The chat surface trades discoverability (widgets show all the dimensions available) for fluency (the user can refine without learning the dimension names).

Three patterns make the warm-conversation UX work. First, the assistant remembers the previous candidate set; "the second one" should resolve to the actual item without forcing the user to retype its name. Second, the assistant acknowledges the user's refinement before producing the new list ("got it, similar genre but under $20 instead"). Third, the assistant volunteers complementary dimensions the user did not name ("these are all paperback; want me to mix in some Kindle editions?"). All three are short LLM-driven additions to the response template; none requires a new pipeline component.

38.4.7 When Conversational Recsys Is Wrong

Conversational recsys is not the right interface for every product. Three situations argue for the classical widget grid instead. First, when the user already knows what they want, a search box plus filters is faster than a conversation. Second, when the catalog is small and the dimensions are few (a dozen items, two attributes), the conversation is overhead. Third, when the user is browsing for serendipity rather than for a specific outcome, an infinite-scroll feed beats a dialogue that keeps asking what they want.

The hybrid pattern most production systems converge on: ship the widget grid as the default, offer a "chat with the assistant" button for users who cannot find what they want, and quietly use the LLM enrichment pipeline (Section 38.3) to lift the quality of the grid even when the user never opens the chat. The chat is a relief valve, not the only door.