Skip to content

refactor(engine): simplify for doing our own OLAP partitioning - #5376

Merged
mrkaye97 merged 4 commits into
mainfrom
mk/rm-hypertables
Oct 31, 2026
Merged

mrkaye97 merged 4 commits into
mainfrom
mk/rm-hypertables

Conversation

@mrkaye97

Copy link
Copy Markdown
Contributor

Description

Simplifying some OSS code to go along with doing our own OLAP table partitioning

Type of change

  • Refactor (non-breaking changes to code which doesn't change any behaviour)

Checklist

Changes have been:

  • Documented (where applicable)
  • Added to CHANGELOG (where applicable) -- see Keep a Changelog

🤖 AI Disclosure

No AI

@vercel

vercel Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hatchet-docs Ready Ready Preview Oct 29, 2026 8:40pm UTC

Request Review

@github-actions github-actions Bot added the engine Related to the core Hatchet engine label Oct 29, 2026
@greptile-apps

greptile-apps Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

Simplifies OLAP partition maintenance and separates historical metric fallback from partition ownership.

  • The OLAP job creates and cleans up partitions for all listed tables.

No actionable issues were found. Runtime tests were not run.

Reviews (1) · Last reviewed commit: "fix: comment" · Reviewed by Greptile

Comment on lines +187 to +209
SELECT
SPLIT_PART(entry, ':', 1) AS parent_table,
SPLIT_PART(entry, ':', 2) AS partition_frequency
FROM UNNEST(ARRAY[
'v1_tasks_olap:daily',
'v1_dags_olap:daily',
'v1_runs_olap:daily',
'v1_events_olap:daily',
'v1_event_to_run_olap:daily',
'v1_event_lookup_table_olap:weekly',
'v1_incoming_webhook_validation_failures_olap:daily',
'v1_cel_evaluation_failures_olap:daily',
'v1_payloads_olap:daily',
'v1_otel_trace_olap:daily',
'v1_otel_trace_lookup_olap:daily',
'v1_task_events_olap:daily',
'v1_dag_to_task_olap:daily',
'v1_lookup_table_olap:monthly',
'v1_statuses_olap:monthly',
'v1_agg_task_statuses_minute_olap:daily',
'v1_agg_task_events_minute_olap:daily',
'v1_agg_events_minute_olap:daily'
]::TEXT[]) AS entry

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reworked this a bit - IMO this is a lot easier to read / understand than the old way

@mrkaye97
mrkaye97 merged commit de4db90 into main Oct 31, 2026
60 checks passed
@mrkaye97
mrkaye97 deleted the mk/rm-hypertables branch October 31, 2026 13:42

This branch was successfully deployed

1 active deployment
Preview — c9cb075f Deployed Oct 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine Related to the core Hatchet engine

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants