Conversations (HITL)

The Conversations tab is your control room for understanding how users interact with your agent. It’s where you see what’s working, what’s not, and where to improve.

Think of it as your human in the loop (HITL) observability dashboard: you can watch conversations, measure performance, and act immediately when you spot issues.

Why Conversations Matter

• Visibility → Every user query and agent response is logged.
• Quality signals → Scores, user sentiment, and “gaps in training” help you measure effectiveness.
• Actionability → From any conversation you can debug, retrain, or hand off to support.

Finding the Right Conversations

Use filters at the top to zero in on what you care about:

• Status → Show answered, not answered, or support handoff cases.
• Actions → Filter by a specific action (e.g., Create Campaign, Invite User).
• User Sentiment → Quickly find all “thumbs down” conversations.
• Gap in Training → Conversations flagged as misses or poor experiences.

💡

Pro tip: Start each week by filtering for Not Answered + Thumbs Down → these are your high-priority fixes.

Inspecting a Conversation

Click a conversation in the list to open its details:

• Timeline → See exactly what the user asked and how the agent responded.
• User details → Identify who the user was (email, product context).
• Function Execution → View which action was triggered, with arguments and outputs.
• Analysis view → Get an evaluation score with reasoning (why it worked, or what was missing).

This lets you diagnose: Did the agent pick the right action? Did the handler run correctly? Was the answer clear enough?

Taking Action

Inside a conversation you can:

• Train from feedback → Click “Wrong answer? Let’s train this Copilot” to immediately improve the model.
• Debug handlers → Use the Function Execution log to verify input/output.
• Close the loop → Add the missed intent to Knowledge Studio or refine an action in Agent Studio.

Continuous Improvement Workflow

A simple routine teams find effective:

1. Review metrics (Questions Answered %, Gaps in Training).
2. Filter for failed or negative-feedback conversations.
3. Inspect 2–3 examples in detail.
4. Retrain or fix actions/knowledge where gaps show up.
5. Re-evaluate after publishing updates.

Repeat weekly → your agent steadily improves.

Best Practices

• Track conversion score trends over time to measure improvement.
• Encourage users to leave thumbs up/down feedback — it’s gold for training.
• Pair agent responses with function logs to catch whether issues are model-related or integration-related.
• Treat the Conversations tab as a feedback loop, not just a log.

✅ With Conversations, you’re not flying blind — you have a live feedback and debugging loop to keep your agent accurate, useful, and trustworthy.