Skip to content

docs: add ABCD synthetic agent conversation dev note Part 1 - #774

Merged
seayang-nv merged 4 commits into
mainfrom
seayang/abcd-dev-notes
Oct 15, 2026
Merged

seayang-nv merged 4 commits into
mainfrom
seayang/abcd-dev-notes

Conversation

@seayang-nv

@seayang-nv seayang-nv commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adding dev notes on ABCD (agent conversation) experiments.
This PR includes the first part of a two post series.

  • First post describes the ABCD dataset, how to transform the data shape to fit NSS requirements, training/generation, utility and privacy analysis. (this PR)
  • Second post includes prompt only (in-context learning), hyper-parameter sweep and privacy/utility tradeoffs, and data augmentation. (will be included in a future PR)

Proposal doc

Human review

Select exactly one. The default is no human review; uncheck it when choosing another level.

  • ⚪ No human review; automated review only
  • 🟡 Partially reviewed or spot-checked by a human
  • 🔵 Complete diff reviewed by a human
  • 🟢 Complete diff reviewed and verified by a human

Verification performed:

Pre-Review Checklist

Ensure that the following pass:

  • mise run format leaves only intended formatting changes.
  • mise run check ::: test passes locally.
  • mise run test:e2e passes locally
  • mise run test:ci-container passes locally (recommended)
  • GPU CI status check passes -- comment /sync on this PR to trigger a run (auto-triggers on ready-for-review)

Pre-Merge Checklist

  • New or updated tests for any fix or new behavior
  • Updated documentation for new features and behaviors, including docstrings for API docs.

Other Notes

  • Closes #

Summary by CodeRabbit

  • Improvements
    • Compact tables display with smaller text, tighter spacing, and centered table content.
  • Documentation
    • Added a guide to preparing customer-service conversations for synthetic data generation and evaluating utility and privacy.

@seayang-nv
seayang-nv requested a review from a team as a code owner October 8, 2026 16:10
@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA-NeMo/Safe-Synthesizer/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: d353c348-40bc-49ba-a4da-ee1fde24208d
📥 Commits

Reviewing files that changed from the base of the PR and between d9e0df3 and ce31cd4.

⛔ Files ignored due to path filters: 4
  • docs/dev-notes/posts/assets/abcd-conversations-to-synthetic-training-data/action-mix.png is excluded by !**/*.png
  • docs/dev-notes/posts/assets/abcd-conversations-to-synthetic-training-data/conversation-length.png is excluded by !**/*.png
  • docs/dev-notes/posts/assets/abcd-conversations-to-synthetic-training-data/downstream-utility.png is excluded by !**/*.png
  • docs/dev-notes/posts/assets/abcd-conversations-to-synthetic-training-data/membership-inference.png is excluded by !**/*.png
📒 Files selected for processing: 1
  • docs/hooks.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout: 2
  • GitHub Check: Greptile Review
  • GitHub Check: Analyze (python)
🧰 Additional context used
📚 Code guidelines: 2
CONTRIBUTING.md — configured
STYLE_GUIDE.md — configured
📓 Path-based instructions: 4
Review documentation as MkDocs Material content.

⚙️ CodeRabbit configuration file

Files:

  • docs/hooks.py
Review as a senior maintainer for NeMo Safe Synthesizer.

⚙️ CodeRabbit configuration file

Files:

  • docs/hooks.py
Source excerpt: All source files (`.py`, `.sh`, `.yaml`, `.yml`, `.md`) require SPDX copyright headers.

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Files:

  • docs/hooks.py
Applicability reference: AGENTS.md: # AGENTS.md See [STYLE_GUIDE.md](STYLE_GUIDE.md) for detailed code style conventions (Python, markdown, Dockerfiles, shell scripts, testing, config files, docstrings).

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

Files:

  • docs/hooks.py
🔇 Additional comments: 1
docs/hooks.py: 1

29-32: LGTM!

Also applies to: 36-36, 38-40

Walkthrough

The documentation adds conditional compact-table styling and an article about converting ABCD conversations into synthetic training data and evaluating the generated conversations.

Changes

Compact-table presentation

Layer / File(s) Summary
Enable compact-table presentation
docs/hooks.py, docs/css/style.css
When page metadata enables compact tables, the hook wraps the page HTML. The stylesheet centers the wrapper and adjusts table text, alignment, cell padding, and header width.

ABCD synthetic-data article

Layer / File(s) Summary
Document conversation conversion and evaluation
docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md
The article describes conversation-to-row conversion, configuration, generation, output statistics, downstream utility, privacy checks, replay filtering, and planned experiments.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~15 minutes

Change: Feature

Suggested labels: feature

Suggested reviewers: binaryaaron

Merge Risk: 🔵 Low · up to ce31c

The article discloses replay, but its takeaway could lead readers to believe no training content was repeated. Clarify the wording to avoid overstating the privacy results; this is a bounded documentation risk.

Pre-merge checks: Passed 5
Passed Passed checks: 5 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely identifies the main change: adding Part 1 of the ABCD synthetic agent conversation development note. The supporting CSS and hook changes do not alter the primary purpos…
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot added the feature New feature or request label Oct 8, 2026
@greptile-apps

greptile-apps Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 3/5

[Low impact] Adds documentation and styling for a dev note post.

The previous findings remain unresolved, including the tutorial’s unsupported release and the check that misses decreasing fractional timestamps.

Findings

  1. P1 Tutorial installs an unsupported release ▶
  2. P1 Fractional times can move backward ▶
  3. P2 Training input is not reproducible ▶
  4. P2 Cleanup breaks the comprehension rule ▶

Summary

Adds Part 1 of the ABCD conversation synthesis notes.

  • The guide turns ABCD conversations into ordered rows for synthetic-data training.
  • The guide shows how synthetic chats compare with real conversations.
  • Marked documentation pages show smaller, tighter tables.

Reviews (6) · Last reviewed commit: "docs: move ABCD dev note Part 2 to a sep..." · Reviewed by Greptile

@codecov

codecov Bot commented Oct 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@seayang-nv
seayang-nv force-pushed the seayang/abcd-dev-notes branch from a708488 to a76987d Compare October 13, 2026 18:51
@coderabbitai coderabbitai Bot added the docs Documentation-only change label Oct 13, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA-NeMo/Safe-Synthesizer/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: c50f44f5-2a57-4dc1-a3bf-db44ecce11b7
📥 Commits

Reviewing files that changed from the base of the PR and between a708488 and a76987d.

⛔ Files ignored due to path filters: 4
  • docs/dev-notes/posts/assets/abcd-conversations-to-synthetic-training-data/action-mix.png is excluded by !**/*.png
  • docs/dev-notes/posts/assets/abcd-conversations-to-synthetic-training-data/conversation-length.png is excluded by !**/*.png
  • docs/dev-notes/posts/assets/abcd-conversations-to-synthetic-training-data/downstream-utility.png is excluded by !**/*.png
  • docs/dev-notes/posts/assets/abcd-conversations-to-synthetic-training-data/membership-inference.png is excluded by !**/*.png
📒 Files selected for processing: 1
  • docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📜 Review details
⏰ Context from checks skipped due to timeout: 8
  • GitHub Check: Unit Tests (3.12)
  • GitHub Check: Unit Tests (3.13)
  • GitHub Check: Unit Tests (3.14)
  • GitHub Check: Unit Tests (3.11)
  • GitHub Check: Greptile Review
  • GitHub Check: Typecheck
  • GitHub Check: Analyze (Python)
  • GitHub Check: Analyze (python)
🧰 Additional context used
📚 Code guidelines: 2
CONTRIBUTING.md — configured
STYLE_GUIDE.md — configured
📓 Path-based instructions: 6
Review documentation as MkDocs Material content.

⚙️ CodeRabbit configuration file

Files:

  • docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md
Review as a senior maintainer for NeMo Safe Synthesizer.

⚙️ CodeRabbit configuration file

Files:

  • docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md
Source excerpt: All source files (`.py`, `.sh`, `.yaml`, `.yml`, `.md`) require SPDX copyright headers.

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Files:

  • docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md
Source excerpt: Create or edit the `.md` file under the appropriate `docs/` subdirectory.

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Files:

  • docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md
Source excerpt: Documentation pages (in `docs/`): classify as tutorial, how-to, explanation, or reference per the [Diataxis framework](https://diataxis.fr/).

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

Files:

  • docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md
Source excerpt: Use `--` (em-dash) for asides, not `-` (hyphen).

📄 CodeRabbit inference engine (STYLE_GUIDE.md)

Files:

  • docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md
🪛 LanguageTool
docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md

[style] ~118-~118: If you’re making a request, consider inserting ‘please’ to sound more courteous and respectful.
Context: ...rance | | hi! i need to return an item, can you help me with that? | return_size | | []...

(CAN_YOU)

🪛 markdownlint-cli2 (0.23.3)
docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md

[warning] 148-148: Code block style
Expected: fenced; Actual: indented

(MD046, code-block-style)


[warning] 176-176: Code block style
Expected: fenced; Actual: indented

(MD046, code-block-style)

🔇 Additional comments: 1
docs/dev-notes/posts/abcd-conversations-to-synthetic-training-data.md: 1

190-190: 🔒 Security & Privacy

The replace_pii: null example disables PII replacement.

replace_pii uses None as its disabled signal. The CLI configuration accepts explicit null, and the processing path skips NemoPII when replace_pii is None. The example is consistent with the SDK’s enable=False configuration.

@seayang-nv seayang-nv changed the title docs: add ABCD synthetic agent conversation dev notes docs: add ABCD synthetic agent conversation dev note Part 1 Oct 13, 2026
@coderabbitai coderabbitai Bot added bug Defects in shipped behavior chore Maintenance not tied to a user-visible change labels Oct 14, 2026
@coderabbitai coderabbitai Bot removed the docs Documentation-only change label Oct 14, 2026
@seayang-nv
seayang-nv force-pushed the seayang/abcd-dev-notes branch from d65e62a to d9e0df3 Compare October 14, 2026 17:26
Add a two-part dev-note series on synthesizing and tuning agent conversations with ABCD, with figures, and compact table styling for pages that opt in via compact_tables front matter.

Signed-off-by: seayang <seayang@nvidia.com>
Signed-off-by: Sean Yang <seayang@nvidia.com>
Signed-off-by: Sean Yang <seayang@nvidia.com>
Signed-off-by: Sean Yang <seayang@nvidia.com>
Signed-off-by: Sean Yang <seayang@nvidia.com>
@seayang-nv
seayang-nv force-pushed the seayang/abcd-dev-notes branch from d9e0df3 to ce31cd4 Compare October 14, 2026 17:31
@seayang-nv seayang-nv added docs Documentation-only change and removed bug Defects in shipped behavior feature New feature or request chore Maintenance not tied to a user-visible change labels Oct 14, 2026
@seayang-nv
seayang-nv added this pull request to the merge queue Oct 14, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 14, 2026
@seayang-nv
seayang-nv added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit 8fc2904 Oct 15, 2026
27 checks passed
@seayang-nv
seayang-nv deleted the seayang/abcd-dev-notes branch October 15, 2026 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants