Repository navigation
Conversation
Adds a read-only skill that pulls Red Hat Customer Portal KB Solution articles for Red Hat Developer Hub and compares a rolling 90-day window against a rolling 365-day window. The taxonomy is data rather than code: the built-in clusters stay the default, --themes PATH applies a taxonomy proposed for the investigation, and --themes none hands the article corpus to the agent to group. The report and JSON carry the signals needed to judge a taxonomy: per-window unmatched counts, and a warning when a single article supplies most of a cluster's caseCount365. - dependency-free Node 18+ script; no build step, no dependencies - catalog entry, /ask-rhdh routing table, README row, install manifest - one generic installable-group test replaces per-skill assertions - 34 script tests covering windows, taxonomy modes, and report shapes Assisted-By: Codex
|
@rm3l @schultzp2020 I just put up a PR for a new skill. I couldn't find an MCP for KB articles (sounds like one may be coming later though), so had Codex write up a script and skill that hits the public site. If this proves to be something we use long term, and we figure out the JEV style classifiers and model access, my assumption is I could then refactor to utilize that for better and cheaper analysis. It doesn't totally fit with the other release skills, but that felt like the best current categorization. Maybe in the future it should move to a planning or operations bucket? It was written with GLM and DeepSeek so I could try out ChatGPT on EnMaaS models. |
The first cut answered "what did support publish recently" while reading as "what are customers hitting". Of 171 RHDH articles, the 16 created in the last 90 days carry 10 linked cases, and one 9-product CVE article holds 83 of the 187 corpus cases. Both distortions are now explicit. - demand view: whole corpus weighted by caseCount_365, not windowed by createdDate, since that count cannot be re-windowed below a year - casesRaw, casesAttributed (divided across tagged products), and casesDedicated (single-product only) per theme and per article - corpus concentration: largest article named, share of raw and attributed weight, warning above 25% - theme movement classified against a baseline that excludes the recent window, with a five-article guard that prints insufficient rather than a ratio - KCS state mix and refresh candidates for stale high-touch articles - component folding with --component-aliases for real synonyms - emergent terms mined from titles and tags, with ubiquitous-term and singleton floors - schemaVersion 2; existing JSON keys and report sections preserved - 53 script tests (was 34) Assisted-By: Codex
…vered An explicit --recent-since/--recent-until pair ends before the default comparison window does, so a calendar quarter's comparison silently included the following month. The caveat now names the uncovered span, counts its days, and points at --comparison-until. Assisted-By: Codex
With 16 recent articles, any word used twice in the year has a 4.6% chance of landing only in the recent window, so a hundred candidate terms yield about four false 'new' hits per run. Raise the floor to three recent articles, drop symptom vocabulary from titles, and state the baseline length on rising rows so the rate claim is visible. Assisted-By: Codex
The corpus published 46 articles in the prior half-year and 25 in the recent half. Rating themes on absolute rate turned that single corpus-wide slowdown into six 'declining' themes, two of which had gained share. Classes now read share ratio (1.25x / 0.8x) and the report states movement.corpusRateRatio so the absolute shift stays visible. Assisted-By: Codex
Bare product nouns were doing the theme work. plugins? alone appeared in 40% of the corpus, taking upgrade-plugin-reliability to 66% of a window and inverting the demand ranking; catalogs? found articles about the software catalog rather than catalog scale, so catalog-database-scale claimed 39 articles and 22 attributed cases. Version 2 requires plugin-lifecycle context and catalog or data-layer context, and drops ui, custom, entity, large, and many. Shares now top out at 41%, unmatched rises from 1 to 5 of 73 comparison articles. The report names the version so runs are never compared across sets. Assisted-By: Codex
|
Moved to internal GL instead: https://gitlab.cee.redhat.com/rhidp/rhdh-internal-skills/-/merge_requests/5 /close |
|
@rm3l: Closed this PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
What this adds
/rhdh-kb-theme-analysisanswers "what are customers hitting with RHDH this quarter?" frompublic Red Hat KB data instead of from memory.
Ask a question and it pulls every RHDH Solution Article from the Customer Portal knowledge
base, then reports two views that are never mixed: what support published in a window
(
createdDate), and what customers hit, weighted by each article's linked support casesacross the whole corpus. It groups articles into support themes, reports each theme's share,
per-day rate, and case weight, and links every article it names back to its KB page so the
answer can be checked.
Typical asks:
It is read-only and public
The skill reads the public KB search API and nothing else. No Jira, no case records, no
private data. Release questions that need Jira stay with
/rhdh-release-status, and writingdocumentation stays with
/rhdh-feature-docs.What the PR does
no
package.json.--themes PATHapplies a taxonomy written for a specific question, and
--themes nonereturns thearticles for the agent to group itself.
separate demand view weights the whole corpus by
caseCount_365and ignores publicationdate, because that API count cannot be re-windowed below a year.
times the demand of a product-specific one: every theme and article carries
casesRaw,casesAttributed, andcasesDedicated.rather than absolute rate, and states
corpusRateRatioso a corpus-wide slowdown is notreported as eight separate theme trends.
optional
--component-aliasesfile, and emergent vocabulary mined from titles and tagswith floors that survive a multiple-comparisons check.
taxonomy needs replacing.
/ask-rhdhrouting table, the README, and theinstall manifest.
tests.
What the output gives you
Two views, deliberately kept apart. The rolling windows measure what support published; the
demand view weights the whole corpus by
caseCount_365and ignores publication date. Caseweight is annual by construction — the API publishes a fixed trailing-year count — so the
skill says so rather than implying a quarterly case number.
Per theme and per article the report carries three case figures:
casesRaw(plain sum),casesAttributed(each article divided across the products it is tagged to), andcasesDedicated(single-product articles only).Against the live corpus that changes the story. Of 172 RHDH articles, 81 carry any linked
cases, summing to 186 raw and 110.2 attributed. A single 9-product CVE article holds 44.6% of
raw case weight (83 cases) but only 8.4% attributed; it is now named under "Corpus
concentration" instead of quietly setting the tone of a theme total.
support-version-clarityreads 102 raw cases and 28.2 attributed — the difference is that same article.
The report also adds KCS state (154 verified, 13 unverified, 5 work-in-progress) and a refresh
list of 11 articles that carry cases and have gone unedited for over a year; theme movement
classified on share with the corpus rate stated alongside, which is what separates a real
trend from a corpus-wide slowdown; and emergent vocabulary mined from titles and tags, floored
so that chance words do not appear as findings.
The built-in taxonomy is versioned as
built-in-v2, and the report names the version it used.Version 1 matched bare product nouns —
plugins?,catalogs?,ui,custom,entity—which took
upgrade-plugin-reliabilityto 66% of a window and madecatalog-database-scalemostly find articles about the software catalog. Version 2 requiresplugin-lifecycle and catalog or data-layer context;
catalog-database-scalefell from 39articles and 22 attributed cases to 9 and 10.
Verification
node --test "skills/**/tests/*.test.mjs": 132 pass, 64 for this skill.uv run pytest: 991 pass. The 6 failures intest_base_images_and_rpms.pyandtest_check_trusted_tasks.pyreproduce on the base commit and are macOS-only.python3 scripts/validate_skill_catalog.py --root .: valid.access.redhat.comover the default 90/365 window, a calendar quarter, 120,150 and 180-day windows, a 730-day comparison, and a 180/360 halves comparison.