Proposal and Design for Root-Level Nested Aggregation Fields

[marcdaniels-toast]

Problem

sub_aggregations let a client aggregate nested collections (e.g. Team.seasons_nested), but only as a child of a parent-level aggregation (team_aggregations { nodes { sub_aggregations { seasons_nested { ... } } } }).

No real pagination. Sub-aggregation connections expose first and has_next_page but no after/cursors. They're built on a plain terms aggregation with a size cap, not a composite aggregation. Composite's cursor-based pagination only works today for top-level aggregations, where there's a single grouping level with nothing above it. Once a nested aggregation sits under a parent grouping, a cursor per parent bucket isn't well-defined, so cursor pagination isn't offered at all. This is the core problem this design addresses.

Proposed feature

Offer nested types as root-level aggregation query fields:

query {
  teamSeasonPlayerAggregations {
    pageInfo { hasNextPage endCursor }
    nodes {
      groupedBy { ... }
      countDetail { ... }
      aggregatedValues { ... }
    }
  }
}

The query schema would guarantee no grouping happens above the target nested level for this field because it aggregates directly over the nested collection with no parent-level bucketing in the way. That's what makes it valid to use a real composite aggregation (nested inside the nested wrapper), giving after-based cursor pagination consistent with top-level aggregation.

This is additive in that existing sub_aggregations are untouched and keep today's first-only behavior. Root-level fields are a new, separate entry point into the same nested collections.

As a secondary benefit, these new root-level fields also give clients a shorter path to a deeply-nested collection. To aggregate Player rows nested three levels down (Team → TeamSeason → Player), a client using today's sub_aggregations must traverse an extra nodes { sub_aggregations { ... } } } layer per intermediate level, even when they have no interest in grouping at Team or Season at all (an intermediate level can be left ungrouped, but its nodes/sub_aggregations wrapper still has to be written out); a root field skips straight to the target level. This is a convenience; the query already works today, just more verbosely.

Ancestor filtering

For a root field aggregating Player three levels deep — Team --(seasons_nested)--> TeamSeason --(some_middle_nested)--> Middle --(players_nested)--> Player, (Middle doesn't exist in the teams schema today but illustrates a deep nesting case) clients may want to filter on ancestor fields (Team, TeamSeason, Middle), not just Player fields.

Proposal: the field's own type (Player) filters are specified inline in filter, exactly as it would for a sub_aggregations field's filter today. Every ancestor, including the root, gets its own key, one per step in the path, nested to mirror the actual path structure:

filter: {
  team: {
    league: { equal_to_any_of: ["NFL"] }
    seasons_nested: {
      year: { gt: 2020 }
      some_middle_nested: {
        someMiddleField: { ... }
      }
    }
  }
  name: { equal_to_any_of: ["Smith"] }   # Player, inline (target type)
}

team is the outermost/topmost ancestor key (the root document type); seasons_nested is the field Team uses to reach TeamSeason, and some_middle_nested is the field TeamSeason uses to reach Middle — each nests inside the previous ancestor's block since each ancestor is only reachable through the one before it.

The path from root to target nests one level at a time, each key being the field-path segment used to reach that level; only the target type's own fields are inline. This avoids two problems with a flatter alternative: field name collisions between the root type and the target type (both being flattened into the same object could produce duplicate GraphQL field names), and inconsistent treatment of the root ancestor vs. every other ancestor.

Keying ancestor blocks by field-path segment (rather than type name) keeps them unambiguous even when two ancestor levels share a type, or the same type is reachable via two differently-named paths. The teams schema's nested_fields/nested_fields2 is exactly this case — each duplicate path already has a distinct field name to key off of. These key names should be configurable via the schema DSL, consistent with how other schema element names are already customizable.

Naming / schema surface

Every distinct nested path to a given type needs its own uniquely-named root field. A naming scheme derived automatically from type names alone has potential problems: the teams schema's nested_fields/nested_fields2 (two distinct paths to the same TeamNestedFields type) would collide if the root field name were built purely from the target type.

Proposal: root-level aggregation fields are specified in the schema. A schema author explicitly opts-in by declaring which nested paths get promoted to a root field, and supplies the root field name in doing so, rather than any name being auto-derived from the path or type name. Since names are author-supplied, schema-build-time validation should reject duplicate root field names across different declarations, so a naming mistake surfaces immediately as a build error.

Proposed DSL shape — a declaration on the innermost nested field, naming the root field it should be promoted to:

t.field "players_nested", "[Player!]!" do |f|
  f.mapping type: "nested"
  f.root_aggregation_field "team_season_player_aggregations"
end

Known gap: no client-specified sorting (acknowledged, not addressed here)

Neither top-level aggregations nor sub-aggregations support client-specified sorting today — ordering is implicit, driven by grouping keys (ascending, via composite sources for top-level;
doc_count descending, then re-sorted client-side for stability, for sub-aggregations). This is a documented design choice (config/site/query-api/aggregations.md), not an oversight.

Why it matters here: pagination is often only useful paired with sorting — e.g. "top 5 seasons by win count" needs sorting by an aggregated_values metric, not by grouping key. Adding cursor
pagination to root-level fields doesn't make that possible on its own; it only lets you page through buckets in grouping-key order.

Sorting by a metric would need an ES bucket_sort pipeline aggregation, which interacts awkwardly with composite's after_key traversal (which assumes ascending key order) — combining the two isn't a drop-in change and would need its own design.

Decision: sorting is out of scope for this design. Root-level aggregation fields ship without client-specified sorting, matching current behavior everywhere else.

Non-goals / deferred

• Ancestor filtering may ship as a follow-on to a first cut that only filters within the target nested path and its descendants. The filter-key design above should anticipate ancestor filtering from the start so the later extension isn't a breaking change.
• Cross-sibling-nested filtering / reverse_nested-based root counts (e.g. "count distinct Teams that have a Player matching some condition") — out of scope here, potential future extension.
• Client-specified sorting — see above; out of scope here.

Open questions

1. Whether composite grouping on non-nested array fields within the target nested type works correctly (unverified).

Appendix: other aspects considered

reverse_nested isn't needed (for the base feature)

It seemed like ancestor filtering (e.g. filtering a Player root field by a TeamSeason or Team field) would need Elasticsearch's reverse_nested aggregation, since the query "starts" at the nested level. It doesn't — reverse_nested isn't used anywhere in the codebase today, and it isn't required here either. The root field is a resolver-level convenience; the actual ES request still starts at the real index root and nests downward as usual:

filter(team) → nested(season) → filter(season) → nested(player) → composite(player)

This is the same filter-composition pattern Aggregation::Query#filter_detail already builds for today's parent-filtered sub-aggregations. The only new part is that the resolver skips exposing the intermediate Team/Season grouping levels and returns the innermost composite result directly.

reverse_nested would only be needed for a different class of query we're not targeting — e.g. "count distinct Teams that have a Player matching some condition," or filtering by a sibling nested collection at the same depth. This is not in scope.

Recursive/self-referential nested types: not a concern

A type like Category with a self-referential nested field (subcategories: [Category!]!, allowing arbitrary depth) would need special handling for this feature, but ElasticGraph already disallows that pattern at schema-build time regardless — it rejects any cycle among plain/nested/embedded fields (schema_definition/results.rb, check_for_circular_dependencies!). The only legal self-reference is through relates_to_one/relates_to_many relationship fields, which are a different mechanism and are already excluded from sub-aggregation path generation.

Net effect: the nested-type hierarchy this feature walks is guaranteed acyclic and finite by an existing, unrelated check. No extra depth limit or cycle guard needed.

Composite-agg-inside-nested-agg

Aggregating a nested field always requires wrapping it in an ES nested aggregation. Putting a composite aggregation as its child is standard, well-supported Elasticsearch:

{
  "aggs": {
    "seasons": {
      "nested": { "path": "seasons_nested" },
      "aggs": {
        "by_year": {
          "composite": {
            "size": 10,
            "sources": [
              { "year": { "terms": { "field": "seasons_nested.year" } } }
            ]
          }
        }
      }
    }
  }
}

This is why the "no grouping above the target level" constraint matters: after_key pagination only makes sense when composite is the single, top-most grouping construct in that branch. A terms/composite agg above it (e.g. bucketing by Team first) would produce one after_key per parent bucket, making "page 2 of this nested list" ill-defined — which is the exact situation today's sub_aggregations are stuck in.

Unverified: how composite grouping behaves on plain (non-nested) array fields within the target nested type — composite sources generally expect one value per document, and nested docs satisfy that, but an array-valued field inside the nested type might not.

Key code references

• elasticgraph-graphql/lib/elastic_graph/graphql/aggregation/nested_sub_aggregation.rb — current
  nested sub-aggregation query builder (terms + size cap, no composite).
• elasticgraph-graphql/lib/elastic_graph/graphql/aggregation/composite_grouping_adapter.rb —
  existing composite/after_key pagination (top-level only).
• elasticgraph-graphql/lib/elastic_graph/graphql/aggregation/query.rb (filter_detail) — filter
  composition across parent/nested levels; the pattern ancestor filtering would reuse.
• elasticgraph-schema_definition/lib/elastic_graph/schema_definition/mixins/supports_filtering_and_aggregation.rb
  — where sub-aggregation fields are defined without pagination support.
• elasticgraph-graphql/spec/acceptance/sub_aggregations_spec.rb — current sub-aggregation
  behavior and test coverage.
• config/schema/teams.rb — schema used throughout this exploration (nested vs. _object field
  pairs, nested_fields/nested_fields2 duplicate-path case).

[myronmarston]

I'm excited to see this proposal--I think it's going to be a big unlock. I also see it as merely being part 1 in a larger story of a direction I'd like ElasticGraph to go: I'd like ElasticGraph to treat nested documents as first-class documents--besides supporting direct aggregation of them (this proposal), I'd also like to see us add support for querying raw nested documents. For example, see the warning in our list filtering docs--in a case like that music schema, it would be useful to be able to directly filter and paginate albums (or maybe artistAlbums) rather than having to filter and paginate artists (and then apply client-side filtering to get the albums you really care about).

I bring this up because it's good to bear in mind as we design this feature: the design choices we make for this feature should work as-is in the future with direct nested document queries. (For example: the GraphQL input shape for how you filter on parent document fields should be the same for the two cases).

This proposal has several points where there are interesting decisions to make. I'm going to split off separate responses to discuss each so that we can have threaded discussions about each part of the design.

[myronmarston]

Ancestor filtering

To me, this is one of the main open questions and I'd like to enumerate the options carefully and consider them all. To start with, I'm going to use the music schema used on EG's public website and its bootstrapping template. (I find it easier to work with than the example you gave: the _nested fields don't have very friendly names and the Middle type makes the examples harder to follow.) The music schema looks like this:

• Artist
  • albums
    • tracks
  • tours
    • shows

For a root field aggregating tracks (artistAlbumTrackAggregations), three observations shape the whole design space:

1. Cross-level OR is inexpressible in the datastore. Our filter composition is a filter agg per level inside each nested agg, and an ES/OpenSearch nested context can't reference parent-document fields. So "albumName = X OR artist yearFormed > 2000" can't be executed efficiently, period. Any design that puts ancestor keys inside the target's filter input makes anyOf: [{artist: ...}, {name: ...}] syntactically writable but unexecutable--we'd need runtime validation errors, or to drop anyOf from the generated type. Designs that separate the levels structurally make the invalid thing unwritable, which I strongly prefer.
2. Two distinct semantics exist, and both are useful. "The containing album is named X" (ancestor-chain filter--unique per track) vs. "the artist has some album named X" (anySatisfy--that album may not contain the track being aggregated). A design that reuses ArtistFilterInput for the root level gives us both for free, as long as they're structurally distinguishable.
3. Every input shape requires an input type definition. GraphQL doesn't let us mix fields from different levels on one input without defining a new input type just for that combination. So any shape that puts ancestor keys alongside (or inside) target-type fields forces us to generate new filter input types per exposed path--potentially bloating the schema--whereas shapes that keep each level's filter as a standalone value can reuse the existing *FilterInput types as-is.

Also relevant: filtering on root document fields is essential for efficiency (index targeting with rollover, shard routing), so the root level's filter needs to be cleanly extractable and prominent, not an afterthought.

With that, the shapes I see:

A. Outside-in, path-nested, target fields inline (as proposed):

# SDL: a new chain of input types per exposed path. Every level's fields are
# duplicated onto a path-specific type because each level mixes in a key that
# doesn't exist on the standard FilterInput for that type.
input ArtistAlbumTrackAggregationsFilterInput {
  # ...all TrackFilterInput fields, duplicated inline (anyOf omitted or runtime-validated)...
  lengthInSeconds: IntFilterInput
  artist: ArtistAlbumTrackAggregationsArtistFilterInput
}

input ArtistAlbumTrackAggregationsArtistFilterInput {
  # ...all ArtistFilterInput fields, duplicated, except `albums` changes meaning/type:
  bio: ArtistBioFilterInput
  albums: ArtistAlbumTrackAggregationsAlbumFilterInput  # containing-album semantics, no anySatisfy
}

input ArtistAlbumTrackAggregationsAlbumFilterInput {
  # ...all AlbumFilterInput fields, duplicated...
  name: StringFilterInput
}

artistAlbumTrackAggregations(filter: ArtistAlbumTrackAggregationsFilterInput, ...): ...

artistAlbumTrackAggregations(filter: {
  artist: {
    bio: {yearFormed: {gt: 2000}}
    albums: {name: {equalToAnyOf: ["Greatest Hits"]}}   # the *containing* album
  }
  lengthInSeconds: {gt: 500}                            # Track fields inline
})

B. Inside-out parent chain, target fields inline:

# SDL: a new type per level except the root, which can reuse ArtistFilterInput
# as-is (it's the end of the chain--no extra key needed on it).
input ArtistAlbumTrackAggregationsFilterInput {
  # ...all TrackFilterInput fields, duplicated inline...
  lengthInSeconds: IntFilterInput
  parentAlbum: AlbumAsParentOfTrackFilterInput
}

input AlbumAsParentOfTrackFilterInput {
  # ...all AlbumFilterInput fields, duplicated...
  name: StringFilterInput
  parentArtist: ArtistFilterInput  # reused!
}

artistAlbumTrackAggregations(filter: ArtistAlbumTrackAggregationsFilterInput, ...): ...

filter: {
  lengthInSeconds: {gt: 500}
  parentAlbum: {
    name: {equalToAnyOf: ["Greatest Hits"]}
    parentArtist: {bio: {yearFormed: {gt: 2000}}}
  }
}

C. Separate argument per level:

# SDL: no new input types at all--every argument reuses an existing FilterInput.
artistAlbumTrackAggregations(
  filter: TrackFilterInput
  albumFilter: AlbumFilterInput
  artistFilter: ArtistFilterInput
  first: Int
  after: Cursor
): ...

artistAlbumTrackAggregations(
  filter: {lengthInSeconds: {gt: 500}}                       # TrackFilterInput, unchanged meaning
  albumFilter: {name: {equalToAnyOf: ["Greatest Hits"]}}     # AlbumFilterInput: the containing album
  artistFilter: {bio: {yearFormed: {gt: 2000}}}              # ArtistFilterInput: the root document
)

D. Single filter: argument, flat per-level keys (target keyed too):

# SDL: one thin wrapper type per exposed path; every value reuses an existing
# FilterInput. Deliberately no anyOf/not on the wrapper (see observation 1).
# Named after the nested path, not the aggregation field, so the same type can
# be shared with the future raw-nested-document query field for this path.
input NestedArtistAlbumTrackFilterInput {
  track: TrackFilterInput
  album: AlbumFilterInput
  artist: ArtistFilterInput
}

artistAlbumTrackAggregations(filter: NestedArtistAlbumTrackFilterInput, ...): ...

filter: {
  track: {lengthInSeconds: {gt: 500}}
  album: {name: {equalToAnyOf: ["Greatest Hits"]}}
  artist: {bio: {yearFormed: {gt: 2000}}}
}

E. Hybrid of C and D: target filter: + single parentFilters: wrapper argument:

# SDL: one thin wrapper per exposed path covering only the ancestor levels; the
# target keeps the standard filter: convention. No anyOf/not on the wrapper.
# Named after the path so the future raw-nested-document query field can share it.
input NestedArtistAlbumTrackParentFiltersInput {
  album: AlbumFilterInput
  artist: ArtistFilterInput
}

artistAlbumTrackAggregations(
  filter: TrackFilterInput
  parentFilters: NestedArtistAlbumTrackParentFiltersInput
  ...
): ...

artistAlbumTrackAggregations(
  filter: {lengthInSeconds: {gt: 500}}
  parentFilters: {
    album: {name: {equalToAnyOf: ["Greatest Hits"]}}
    artist: {bio: {yearFormed: {gt: 2000}}}
  }
)

F. Global parent traversal key on every nested type's FilterInput (B generalized):

# SDL: every nested type's *existing* FilterInput gains a reserved `parent` key,
# with one sub-key per possible containing type. New types are per-*type*
# (not per-path), and today's sub_aggregations filters would gain ancestor
# filtering for free--no new root-field-specific filter machinery at all.
input TrackFilterInput {
  # ...existing fields, plus:
  parent: NestedParentOfTrackFilterInput
}

input NestedParentOfTrackFilterInput {
  album: AlbumFilterInput  # which itself now has parent: {artist: ...}
}

artistAlbumTrackAggregations(filter: TrackFilterInput, ...): ...

filter: {
  lengthInSeconds: {gt: 500}
  parent: {album: {
    name: {equalToAnyOf: ["Greatest Hits"]}
    parent: {artist: {bio: {yearFormed: {gt: 2000}}}}
  }}
}

How they compare:

	A (proposed)	B	C	D	E	F
New generated types	1 + one per level, per path	one per level (root reuses ArtistFilterInput), per path	0	1 wrapper per path	1 wrapper per path	one per nested type (shared across paths)
Collisions with the target type's own fields (Track.artist, Track.album)	yes	yes	none	none	none	one reserved name (parent), customizable
Cross-level OR structurally prevented	no	no	yes	yes (no anyOf on the wrapper)	yes	no (needs runtime validation)
filter: keeps its universal meaning (the target type's own FilterInput, as on sub_aggregations and artistAggregations today)	no	no	yes	no	yes	yes--but every nested type's FilterInput changes globally
Root filter extractable/prominent	yes	buried at the deepest nesting	yes	yes	yes	buried at the deepest nesting
"Containing X" vs anySatisfy both expressible & distinguishable	blurred (its albums: key looks like ArtistFilterInput.albums but means something different--no anySatisfy wrapper)	blurred	yes, clearly	yes, clearly	yes, clearly	yes, clearly
Existing sub_aggregations gain ancestor filtering for free	no	no	no	no	no	yes

A couple of other notes:

• Inside-out vs. outside-in becomes moot under C, D, and E--the levels are flat siblings, so there's no ordering to argue about, and no level is awkwardly privileged over another. (F is inherently inside-out.)
• Intermediate levels reuse existing filter types cleanly: albumFilter: {tracks: {anySatisfy: ...}} ("the containing album has some track matching...") is expressible in the datastore (nested queries compose downward), so full AlbumFilterInput reuse is sound at every level.
• All of these shapes translate to the same efficient datastore request (filter(artist) → nested(albums) → filter(album) → nested(tracks) → filter(track) → composite), and all can carry their filter shape over to the future raw-nested-document query fields as-is.

I'd rule out A and B, but I see genuine strengths in each of the others:

• C: zero new generated types, and it preserves what I think is our strongest existing convention--filter: always takes the type's own FilterInput, exactly as sub_aggregations and root aggregation fields do today. Cost: N arguments instead of one (and N GraphQL variables for clients that parameterize all levels).
• D: a single argument and a single extension point, one variable for clients, and the wrapper type could be shared as-is with the future raw-nested-document query field for the same path (hence naming it NestedArtistAlbumTrackFilterInput after the path rather than the aggregation field). Cost: one generated wrapper type per exposed path, and filter: means something new on these fields (a mitigation: name the argument something other than filter--e.g. filters--to keep the existing convention unbroken).
• E: arguably the best of C and D--filter: keeps its universal meaning and ancestor filters get a single extension point/variable. Cost: still one generated wrapper per exposed path, and the two-argument shape is a little unusual.
• F: the most uniform--one mechanism added to every nested type's FilterInput, new types are per-type rather than per-path, and existing sub_aggregations filters gain ancestor filtering for free, with no root-field-specific filter machinery at all. But it shares A/B's core flaw (anyOf: [{parent: ...}, ...] is writable but unexecutable, requiring runtime validation), and a new reserved key is a breaking change for schemas that already have a parent field (mitigable via schema_element_name_overrides, same as anySatisfy et al.).

On the proposed shape (A) specifically, my concerns are: it needs a new chain of generated types anyway, it has the target-field collision problem (nothing stops Track from having its own artist or album field), and it blurs the containing-vs-anySatisfy distinction noted above.

[myronmarston]

Stepping back from the filtering details in my other comment, I want to float an alternative to the entire proposal: instead of adding new root-level aggregation fields, what if we made the existing aggregations API able to express "ungrouped"?

The query this proposal enables can almost be written today:

artistAggregations(filter: {bio: {yearFormed: {gt: 2000}}}) {  # root filter: index/shard targeting works as usual
  nodes {                                                      # no groupedBy selected--one implicit bucket
    subAggregations {
      albums(filter: {name: {equalToAnyOf: ["Greatest Hits"]}}) {  # ancestor filter, "containing album" semantics
        nodes {                                                # again, one implicit bucket
          subAggregations {
            tracks(filter: {lengthInSeconds: {gt: 500}}, first: 10) {
              nodes { groupedBy { ... } aggregatedValues { ... } }
            }
          }
        }
      }
    }
  }
}

Ancestor filtering already exists in this shape--each level's existing filter: argument is the ancestor filter, with exactly the right "containing document" semantics, no new filter input types, and no way to express a cross-level OR (which the datastore couldn't execute anyway). The only thing actually missing is cursor pagination on tracks, and the blocker is that the schema can't know the ancestor levels are ungrouped--after: would have to be a runtime error whenever a sibling selection groups above.

Idea: an ungrouped field (name TBD--unbucketed?) alongside nodes. Because it's a distinct field returning a distinct type, the type system itself carries the "nothing groups above this level" guarantee down the query--exactly the precondition composite pagination needs--so validity is schema-enforced, and it's self-documenting why cursors are available where they are:

type ArtistAggregationConnection {
  # ...existing nodes / edges / pageInfo...
  ungrouped: ArtistUngroupedAggregation   # NEW: the single implicit bucket, directly
}

type ArtistUngroupedAggregation {
  count: JsonSafeLong
  aggregatedValues: ArtistAggregatedValues
  subAggregations: ArtistUngroupedSubAggregations
  # Note that this has on `groupedBy` since it's ungrouped!
}

type ArtistUngroupedSubAggregations {
  # cursor pagination available here, because the types guarantee no grouping above:
  albums(filter: AlbumFilterInput, first: Int, after: Cursor): ArtistAlbumUngroupedSubAggregationConnection
}

type ArtistAlbumUngroupedSubAggregationConnection {
  pageInfo: PageInfo                              # real cursors (composite agg)
  nodes: [ArtistAlbumSubAggregation!]!            # group at this level...
  ungrouped: ArtistAlbumUngroupedAggregation      # ...or keep drilling down
}

artistAggregations(filter: {bio: {yearFormed: {gt: 2000}}}) {
  ungrouped {
    subAggregations {
      albums(filter: {name: {equalToAnyOf: ["Greatest Hits"]}}) {
        ungrouped {
          subAggregations {
            tracks(filter: {lengthInSeconds: {gt: 500}}, first: 10, after: "...") {
              pageInfo { hasNextPage endCursor }
              nodes { groupedBy { ... } aggregatedValues { ... } }
            }
          }
        }
      }
    }
  }
}

What it buys:

• No new root fields, no opt-in DSL, no naming decisions--no author-supplied field names, no filter argument names, no per-path declarations. Every nested path gets this automatically.
• No new filter design at all--filter: keeps its universal meaning at every level, existing filter input types are reused untouched, and the root filter sits where it always has (so rollover index targeting and shard routing work unchanged).
• Schema-enforced validity--the composite-pagination precondition lives in the type system, not in runtime validation.
• Trivial migration for existing sub_aggregations consumers: swap nodes for ungrouped at ancestor levels and gain cursors.
• A potentially useful convenience on its own: when using aggregations but not grouping, it's always felt a little bit odd to deal with nodes since there's only ever one node. This makes that shape first class, with no need to mess with a list (which the type system implies could be empty).

What it costs:

• The verbose multi-level traversal remains--the proposal's secondary "shorter path to deep collections" benefit is lost (though ungrouped { subAggregations { ... } } is lighter than writing out an artificial single-bucket nodes list).
• A parallel set of ungrouped-variant output types per sub-aggregation level--the schema bloat moves to output types.
• It does nothing for a future raw-nested-document query feature--raw documents aren't aggregations, so that would still need its own entry point and filter shape.
• Discoverability--the capability is tucked inside connection types rather than advertised as a prominent root field.

These aren't mutually exclusive: we could ship ungrouped as the foundational mechanism now, and later add root-level fields as pure sugar over it if the ergonomics warrant.

[myronmarston]

This could also mean some performance impact

I’m not following—-what aspect of this design do you expect to have perf impact?

[marcdaniels-toast]

Sorry, finally getting to read this. This sounds like a clever solution and I really like the sound of it. I need to fully digest some of this idea, mostly because I'm still a bit unfamiliar with the aggregation API generally. For example, the thing which I think I see above is that we can still group at the deepest nesting level, right? In the example above tracks can still be grouped and paged through? It's just the ancestor levels need to be ungrouped in order to support paging at the innermost level?

Terminology-wise it struck me as odd to see subAggregations combined with ungrouped in the level below that:

    subAggregations {
      albums(filter: {name: {equalToAnyOf: ["Greatest Hits"]}}) {
        ungrouped {
        }
    }

since until this discussion I would have thought aggregation implied grouping, but I realize now that is not necessarily the case. It's like in some SQL variants you can say:

SELECT
 SUM(amt) as total
FROM TABLE
GROUP

(I think some dialects even let you omit the GROUP keyword.) This yields exactly one row/bucket for all data. I think that's what we're talking about here: this ensures exactly one result "bucket" for all the ancestor levels. And yet I can still have pages of results if I use groupedBy { } or not at the innermost (leaf?) nesting level?

I'll also play with this concept as much as I can with the existing behavior to get a feel for this aggregation API. (Can I even omit groupedBy currently I wonder?)

[myronmarston]

the thing which I think I see above is that we can still group at the deepest nesting level, right?

Correct.

In the example above tracks can still be grouped and paged through? It's just the ancestor levels need to be ungrouped in order to support paging at the innermost level?

Yes and yes.

since until this discussion I would have thought aggregation implied grouping, but I realize now that is not necessarily the case.

Aggregation isn't meant to imply grouping. It's meant to imply that we are working with collections of documents in aggregate (rather than individual docs). When operating on a collection of documents in aggregate, we can apply operations like grouping, counting, or computing metrics from individual field values.

I think that's what we're talking about here: this ensures exactly one result "bucket" for all the ancestor levels.

Yep.

And yet I can still have pages of results if I use groupedBy { } or not at the innermost (leaf?) nesting level?

Yep.

[jwils]

I like the concept of ungrouped. A few things to figure out. In this example:

query ArtistAndAlbumSummary(
  $countryCursor: Cursor
  $albumCursor: Cursor
) {
  artistAggregations(
    filter: { bio: { yearFormed: { gt: 2000 } } }
    first: 5
    after: $countryCursor
  ) {
    nodes {
      groupedBy { bio { homeCountry } }
      count
    }
    pageInfo {
      hasNextPage
      endCursor
    }

    ungrouped {
      subAggregations {
        albums(first: 10, after: $albumCursor) {
          nodes {
            groupedBy { name }
            countDetail { exactValue }
          }
          pageInfo {
            hasNextPage
            endCursor
          }
        }
      }
    }
  }
}

I think these would become sibling queries but it's a little odd and unclear to me how we'd want pagination?

filter: { bio: { yearFormed: { gt: 2000 } } } I think would apply to ungrouped but first: 5 after: $countryCursor wouldn't?

[myronmarston]

I think these would become sibling queries but it's a little odd and unclear to me how we'd want pagination?

I think it would work how you say:

filter: { bio: { yearFormed: { gt: 2000 } } } I think would apply to ungrouped but first: 5 after: $countryCursor wouldn't?

It might feel odd to have arguments which don't effect subfields but that's always been the case. For example, pagination args are completely ignored here:

query {
  artistAggregations(first: 5, after: "eyJ1dWlkIjoiZGNhMDJkMjAtYmFlZS00ZWU5LWEwMjctZmVlY2UwYTZkZTNhIn0") {
    __typename
  }
}

They simply don't apply to __typename. (filter: would also be ignored).

The way I think about the pagination args is that they specifically apply to the nodes/edges/pageInfo subfields. Other fields such as ungrouped simply ignore the pagination args.