feat: read Jira summary and status for every key in batched searches

With RELEASE_SCOPE_JIRA_TOKEN (or JIRA_TOKEN) set, collect reads all Jira keys of
the report with POST /rest/api/2/search, 100 keys per query, and records their
summary, status, status category and type under a top-level jira block. A failed
search is kept in the report and makes collect exit 1. The page shows each key
with summary and status and counts not-done issues per service. The report
schema is now version 2; render rejects version 1 with a clear message.

Author: lesnik512

AGENTS.md

@@ -15,10 +15,10 @@ issue title.
 
 ## Architecture
 
-`_use_case.py` drives one run; `_gitlab.py` is the only module that speaks HTTP; `_rows.py` is pure and turns the range
-plus merge requests into rows; `_render.py` is pure and turns a report into the Markdown page. Tests mock GitLab only
-with respx routes (pytest-httpx2): the `gitlab` fixture in `tests/conftest.py` declares one named static route per call
-of the scenario in `tests/payloads.py`. A test changes a response by re-mocking a named route; add no fakes, callbacks,
+`_use_case.py` drives one run; `_gitlab.py` and `_jira.py` are the only modules that speak HTTP; `_rows.py` is pure and
+turns the range plus merge requests into rows; `_render.py` is pure and turns a report into the Markdown page. Tests
+mock GitLab and Jira only with respx routes (pytest-httpx2): the `gitlab` and `jira` fixtures in `tests/conftest.py`
+declare one named static route per call of the scenario in `tests/payloads.py`. A test changes a response by re-mocking a named route; add no fakes, callbacks,
 or stubs.
 
 The package must stay free of any company's hostnames, group paths, job names, or Jira project keys: every such value

CONTEXT.md

@@ -27,5 +27,9 @@ latest pipeline whose ref is a tag in the row.
 **Failed job**:
 A job or bridge whose status is `failed`, including those with `allow_failure`; the flag is reported, not filtered.
 
+**Not done**:
+A Jira issue whose status category is anything but `done`. Jira's category, not the status name, which each workflow
+names differently.
+
 **Settled fact**:
 Data GitLab will not change for the same key, and so the only data the cache may hold.

README.md

@@ -5,12 +5,12 @@
 [![License](https://img.shields.io/github/license/modern-python/release-scope.svg)](https://github.com/modern-python/release-scope/blob/main/LICENSE)
 
 `release-scope` collects what sits between production and the default branch across GitLab services: tags, MRs,
-Jira keys, failed jobs.
+Jira issues, failed jobs.
 
 For every service it reads the latest successful production deployment, walks the default branch down to that
 commit, and writes one JSON report: a row per merge request or direct commit, newest first, with the tags that
 point into it, the environments running it, the Jira keys its MR mentions, and the failed jobs of its main-branch
-and tag pipelines.
+and tag pipelines. With a Jira token, it also reads the summary and status of every key in one batched search.
 
 ## Quickstart
 
@@ -27,7 +27,8 @@ uvx release-scope collect --group team/backend --output report.json --cache cach
 collect; the report is still written and names the error on that service. A service GitLab denies access to fails
 alone, and its error lists the project settings and member page to check. A project with CI/CD or Environments
 disabled is reported with a warning and no rows, without querying it. Only a rejected token, or a group or project
-passed on the command line that the token cannot see, stops the run.
+passed on the command line that the token cannot see, stops the run. A failed Jira search is recorded in the report
+and also exits `1`; the GitLab part is still written.
 
 ## Configuration
 
@@ -40,6 +41,7 @@ Every setting is an environment variable; nothing about a GitLab or Jira instanc
 | `RELEASE_SCOPE_ENVIRONMENTS` | `["production"]` | Environments shown per service, as a JSON list |
 | `RELEASE_SCOPE_PRODUCTION_ENVIRONMENT` | `production` | Environment whose deployed commit starts the range |
 | `RELEASE_SCOPE_JIRA_ENDPOINT` | unset | When set, Jira keys link to `<endpoint>/browse/<KEY>` |
+| `RELEASE_SCOPE_JIRA_TOKEN` or `JIRA_TOKEN` | unset | Jira Server/Data Center personal access token; when set, issues are fetched |
 | `RELEASE_SCOPE_JIRA_PROJECT_KEYS` | `[]` | Keep only keys of these Jira projects; empty keeps all |
 | `RELEASE_SCOPE_MAX_COMMITS` | `1000` | Stop walking a service's range after this many commits |
 | `RELEASE_SCOPE_REQUEST_TIMEOUT` | `10` | Per-request timeout in seconds |
@@ -48,7 +50,9 @@ Every setting is an environment variable; nothing about a GitLab or Jira instanc
 
 The report is versioned by `schema_version`; the models live in
 [`release_scope/_report.py`](https://github.com/modern-python/release-scope/blob/main/release_scope/_report.py).
-One row, trimmed:
+Top-level `jira` is `null` without a Jira token; otherwise it holds `issues` by key (summary, status, status
+category, issue type), the `missing` keys Jira did not return, and an `error` if the search failed. One row,
+trimmed:
 
 ```json
 {
@@ -73,8 +77,9 @@ uvx release-scope collect --group team/backend --output report.json --cache cach
 
 The page opens with a table of the services that have pending changes or problems, with the ref each environment runs;
 services already up to date collapse into one expandable table. Each service with changes then has a collapsible table
-of its rows: the tag linked to its pipeline, the merge requests or direct commit, Jira keys, where the change is
-deployed, and the failed jobs of its main-branch and tag pipelines.
+of its rows: the tag linked to its pipeline, the merge requests or direct commit, Jira keys with summary and status,
+where the change is deployed, and the failed jobs of its main-branch and tag pipelines. With Jira issues, the
+summary table also counts the issues per service whose status is not done.
 
 Chain the two commands with `;`, not `&&`: `collect` exits `1` when a service failed, which is exactly when the page
 should show it. Alert on the exit code of `collect`, not on whether to render. `render` fails only when it cannot

release_scope/__main__.py

@@ -1,17 +1,17 @@
 import importlib.metadata
+import json
 import pathlib
 import typing
 
 import modern_di_typer
-import pydantic
 import typer
 
 from release_scope import ioc
 from release_scope._cache import Cache
 from release_scope._errors import ConfigError, ReleaseScopeError
 from release_scope._files import write_text_atomic
 from release_scope._render import render_markdown
-from release_scope._report import Report
+from release_scope._report import SCHEMA_VERSION, Report
 from release_scope._settings import Settings, load_settings
 from release_scope._use_case import CollectUseCase
 
@@ -95,7 +95,10 @@ def _collect_command(  # noqa: PLR0913, PLR0917
     typer.echo(f"{len(report.services)} services, {rows} rows, {len(failed)} failed -> {output}", err=True)
     for service in failed:
         typer.echo(f"Error: {service.error}", err=True)
-    if failed:
+    jira_error: typing.Final = report.jira.error if report.jira else None
+    if jira_error:
+        typer.echo(f"Error: {jira_error}", err=True)
+    if failed or jira_error:
         raise typer.Exit(code=1)
 
 
@@ -105,8 +108,17 @@ def _render_command(
     output: typing.Annotated[pathlib.Path, typer.Option("--output", "-o", help="Where to write the Markdown page.")],
 ) -> None:
     try:
-        report = Report.model_validate_json(report_path.read_bytes())
-    except (OSError, pydantic.ValidationError) as exc:
+        raw = json.loads(report_path.read_bytes())
+        version = raw.get("schema_version") if isinstance(raw, dict) else None
+        if isinstance(version, int) and version < SCHEMA_VERSION:
+            typer.echo(
+                f"Error: Cannot read report {report_path}: schema_version {version} is not supported; "
+                "run collect again.",
+                err=True,
+            )
+            raise typer.Exit(code=ConfigError.exit_code)
+        report = Report.model_validate(raw)
+    except (OSError, ValueError) as exc:
         typer.echo(f"Error: Cannot read report {report_path}: {type(exc).__name__}.", err=True)
         raise typer.Exit(code=ConfigError.exit_code) from exc
     try:

release_scope/_errors.py

@@ -21,3 +21,7 @@ def __init__(self, message: str, *, resource: str, status: int | None = None, re
         self.resource = resource
         self.status = status
         self.reason = reason
+
+
+class JiraError(ReleaseScopeError):
+    pass

release_scope/_jira.py

@@ -0,0 +1,99 @@
+import collections.abc
+import dataclasses
+import typing
+
+import httpware
+import pydantic
+
+from release_scope._errors import JiraError
+
+
+_SEARCH: typing.Final = "/rest/api/2/search"
+_BATCH_SIZE: typing.Final = 100
+_FIELDS: typing.Final = ("summary", "status", "issuetype")
+
+
+class StatusCategory(pydantic.BaseModel):
+    key: str | None = None
+
+
+class Status(pydantic.BaseModel):
+    name: str
+    category: StatusCategory | None = pydantic.Field(default=None, alias="statusCategory")
+
+
+class IssueType(pydantic.BaseModel):
+    name: str
+
+
+class IssueFields(pydantic.BaseModel):
+    summary: str
+    status: Status
+    issuetype: IssueType | None = None
+
+
+class Issue(pydantic.BaseModel):
+    key: str
+    fields: IssueFields
+
+
+class _SearchResults(pydantic.BaseModel):
+    total: int
+    issues: list[Issue]
+
+
+def _error_messages(exc: httpware.StatusError) -> str:
+    try:
+        payload: typing.Final = exc.response.json()
+    except ValueError:
+        return ""
+    messages: typing.Final = payload.get("errorMessages") if isinstance(payload, dict) else None
+    if not isinstance(messages, list):
+        return ""
+    return " ".join(str(message) for message in messages)
+
+
+def _translate(exc: httpware.ClientError) -> JiraError:
+    if isinstance(exc, httpware.UnauthorizedError):
+        return JiraError("Jira rejected the token (401). Check that it is valid and not expired.")
+    if isinstance(exc, httpware.StatusError):
+        status: typing.Final = exc.response.status_code
+        details: typing.Final = _error_messages(exc)
+        suffix: typing.Final = f": {details.rstrip('.')}." if details else "."
+        return JiraError(f"Jira returned {status} for the issue search{suffix}")
+    return JiraError(f"Jira issue search failed: {type(exc).__name__}.")
+
+
+def _batches(keys: collections.abc.Sequence[str]) -> collections.abc.Iterator[collections.abc.Sequence[str]]:
+    for start in range(0, len(keys), _BATCH_SIZE):
+        yield keys[start : start + _BATCH_SIZE]
+
+
+@dataclasses.dataclass(frozen=True, slots=True, kw_only=True)
+class JiraApi:
+    http: httpware.Client
+
+    def search_issues(self, keys: collections.abc.Sequence[str]) -> list[Issue]:
+        issues: typing.Final[list[Issue]] = []
+        for batch in _batches(keys):
+            quoted = ", ".join(f'"{key}"' for key in batch)
+            issues.extend(self._search(f"key in ({quoted})"))
+        return issues
+
+    def _search(self, jql: str) -> list[Issue]:
+        issues: typing.Final[list[Issue]] = []
+        while True:
+            body = {
+                "jql": jql,
+                "fields": list(_FIELDS),
+                "startAt": len(issues),
+                "maxResults": _BATCH_SIZE,
+                "validateQuery": False,
+            }
+            try:
+                page = self.http.post(_SEARCH, json=body, response_model=_SearchResults)
+            except httpware.ClientError as exc:
+                raise _translate(exc) from exc
+            issues.extend(page.issues)
+            if not page.issues or len(issues) >= page.total:
+                return issues

release_scope/_render.py

@@ -3,7 +3,18 @@
 import html
 import typing
 
-from release_scope._report import EnvironmentState, FailedJob, PipelineState, Report, Row, Service, TagRef
+from release_scope._report import (
+    EnvironmentState,
+    FailedJob,
+    JiraIssue,
+    JiraKeyRef,
+    JiraState,
+    PipelineState,
+    Report,
+    Row,
+    Service,
+    TagRef,
+)
 
 
 _STATUS_ICONS: typing.Final = {
@@ -107,11 +118,18 @@ def _pending(service: Service) -> str:
     return f"{_plural(len(service.rows), 'change')}{untagged}"
 
 
-def _summary_row(service: Service, environments: list[str]) -> list[str]:
+def _not_done(service: Service, issues: dict[str, JiraIssue]) -> str:
+    keys: typing.Final = {key.key for row in service.rows for key in row.jira_keys}
+    count: typing.Final = sum(1 for key in keys if key in issues and issues[key].status_category != "done")
+    return f"{count} not done" if count else ""
+
+
+def _summary_row(service: Service, environments: list[str], jira: JiraState | None) -> list[str]:
     return [
         _link(_cell(service.project), service.project_url),
         *(_environment_ref(_environment(service, name)) for name in environments),
         _pending(service),
+        *([_not_done(service, jira.issues)] if jira else []),
         _failure_counts(service),
     ]
 
@@ -161,11 +179,17 @@ def _failed_jobs(row: Row) -> str:
     return "<br>".join(parts)
 
 
-def _row(row: Row) -> list[str]:
+def _jira_key(key: JiraKeyRef, issues: dict[str, JiraIssue]) -> str:
+    text: typing.Final = _link(_cell(key.key), key.url)
+    issue: typing.Final = issues.get(key.key)
+    return f"{text} {_cell(issue.summary)} · {_cell(issue.status)}" if issue else text
+
+
+def _row(row: Row, issues: dict[str, JiraIssue]) -> list[str]:
     return [
         "<br>".join(_tag(tag) for tag in row.tags),
         _merge_requests_or_commits(row),
-        ", ".join(_link(_cell(key.key), key.url) for key in row.jira_keys),
+        "<br>".join(_jira_key(key, issues) for key in row.jira_keys),
         ", ".join(_cell(name) for name in row.environments),
         _failed_jobs(row),
     ]
@@ -179,7 +203,7 @@ def _counts(service: Service) -> str:
     )
 
 
-def _rows_section(service: Service, production: str) -> list[str]:
+def _rows_section(service: Service, production: str, issues: dict[str, JiraIssue]) -> list[str]:
     ordered: typing.Final = sorted(service.environments, key=lambda item: item.name != production)
     environments: typing.Final = [f"{_cell(item.name)} {_environment_ref(item)}" for item in ordered]
     production_state: typing.Final = _environment(service, production)
@@ -189,18 +213,18 @@ def _rows_section(service: Service, production: str) -> list[str]:
         "",
         *_collapsed(
             f"{_plural(len(service.rows), 'change')}{since}",
-            _table(_ROW_HEADER, (_row(row) for row in service.rows)),
+            _table(_ROW_HEADER, (_row(row, issues) for row in service.rows)),
         ),
     ]
 
 
-def _service_section(service: Service, production: str) -> list[str]:
+def _service_section(service: Service, production: str, issues: dict[str, JiraIssue]) -> list[str]:
     lines: typing.Final = [f"## {_inline(service.project)}", ""]
     if service.error:
         lines.extend([f"❌ {_inline(service.error)}", ""])
     lines.extend(line for warning in service.warnings for line in (f"⚠️ {_inline(warning)}", ""))
     if service.rows:
-        lines.extend(_rows_section(service, production))
+        lines.extend(_rows_section(service, production, issues))
     return lines
 
 
@@ -218,6 +242,9 @@ def render_markdown(report: Report) -> str:
         _LEGEND,
         "",
     ]
+    jira: typing.Final = report.jira
+    if jira and jira.error:
+        lines.extend([f"❌ {_inline(jira.error)}", ""])
     if not report.services:
         lines.append("No services were collected.")
         return "\n".join(lines) + "\n"
@@ -229,8 +256,14 @@ def render_markdown(report: Report) -> str:
     up_to_date: typing.Final = [service for service in report.services if _state(service) is None]
     environments: typing.Final = _environment_names(report)
     if attention:
-        header: typing.Final = ["Service", *(_cell(name) for name in environments), "Pending", "Failed jobs"]
-        lines.extend([*_table(header, (_summary_row(service, environments) for service in attention)), ""])
+        header: typing.Final = [
+            "Service",
+            *(_cell(name) for name in environments),
+            "Pending",
+            *(["Jira"] if jira else []),
+            "Failed jobs",
+        ]
+        lines.extend([*_table(header, (_summary_row(service, environments, jira) for service in attention)), ""])
     else:
         lines.extend(["All services are up to date.", ""])
     if up_to_date:
@@ -246,6 +279,7 @@ def render_markdown(report: Report) -> str:
                 ),
             )
         )
+    issues: typing.Final = jira.issues if jira else {}
     for service in attention:
-        lines.extend(_service_section(service, production))
+        lines.extend(_service_section(service, production, issues))
     return "\n".join(lines).rstrip("\n") + "\n"