[RdeBaaij]

Adding the first verson of E2E monitoring (ketenmonitoring) Amsterdam Schema.

[github-actions]

Validation Results:

Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/loaders.py", line 151, in get_table
return self._table_cache[key]
~~~~~~~~~~~~~~~~~^^^^^
KeyError: ('ketenmonitoring', 'dataproducten/v1')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/loaders.py", line 471, in read_json_path
with dataset_file.open() as stream:
~~~~~~~~~~~~~~~~~^^
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/pathlib/init.py", line 771, in open
return io.open(self, mode, buffering, encoding, errors, newline)
~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/runner/work/amsterdam-schema/amsterdam-schema/datasets/ketenmonitoring/dataproducten/v1.json'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/loaders.py", line 275, in _get_table
table_json = self._read_table(dataset.id, table_ref)
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/loaders.py", line 460, in _read_table
return read_json_path(self.root / dataset_path / f"{table_ref}.json")
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/loaders.py", line 478, in read_json_path
raise SchemaObjectNotFound(str(dataset_file)) from e
schematools.exceptions.SchemaObjectNotFound: /home/runner/work/amsterdam-schema/amsterdam-schema/datasets/ketenmonitoring/dataproducten/v1.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.14.9/x64/bin/schema", line 6, in
sys.exit(main())
~~~~^^
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/cli.py", line 196, in main
schema()
~~~~~~^^
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/click/core.py", line 1631, in call
return self.main(*args, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/click/core.py", line 1552, in main
rv = self.invoke(ctx)
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/click/core.py", line 2032, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/click/core.py", line 1415, in invoke
return ctx.invoke(self.callback, **ctx.params)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/click/core.py", line 910, in invoke
return callback(*args, **kwargs)
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/cli.py", line 817, in batch_validate
instance = dataset.json_data(inline_tables=True, inline_publishers=False)
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/types.py", line 414, in json_data
self.json(
~~~~~~~~~^
inline_tables=inline_tables,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...<2 lines>...
scopes=scopes,
^^^^^^^^^^^^^^
)
^
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/types.py", line 389, in json
tables = version.get_tables()
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/types.py", line 979, in get_tables
return list(
self._get_tables(include_nested=include_nested, include_through=include_through)
)
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/types.py", line 1002, in _get_tables
yield from self.tables
^^^^^^^^^^^
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/functools.py", line 1126, in get
val = self.func(instance)
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/types.py", line 964, in tables
table = self.schema.loader.get_table(self.schema, table_json["$ref"])
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/loaders.py", line 153, in get_table
table = self._get_table(dataset, table_ref)
File "/opt/hostedtoolcache/Python/3.14.9/x64/lib/python3.14/site-packages/schematools/loaders.py", line 283, in _get_table
raise DatasetTableNotFound(
f"Dataset '{dataset.id}' has no table ref: '{table_ref}'!"
) from e
schematools.exceptions.DatasetTableNotFound: Dataset 'ketenmonitoring' has no table ref: 'dataproducten/v1'!

[azure-pipelines]

PermissionDenied: This API is disabled for users without the databricks-sql-access or workspace-consume entitlements. Contact your administrator for more information. [ReqId: abbcdada-dfe2-4ba1-8fc4-6fa9a4c03f64]. Config: host=https://adb-1035858348023409.9.azuredatabricks.net, account_id=1a569a49-ad12-41a7-801a-578b41e58bae, workspace_id=1035858348023409, discovery_url=https://adb-1035858348023409.9.azuredatabricks.net/oidc/.well-known/oauth-authorization-server, token_audience=https://adb-1035858348023409.9.azuredatabricks.net/oidc/v1/token, client_id=c5b7e837-7c48-4569-9430-cf635862d0df, azure_tenant_id=72fca1b1-2c2e-4376-a445-294d80196804, auth_type=azure-cli, warehouse_id=36349c2f1ee60c53. Env: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_WAREHOUSE_ID

If you have made the relevant changes, comment /update-uc to sync.

[pstokkink]

Hier moet ook nog een "required": ["alle", "verplichte", "velden"] in. Zie https://schemas.data.amsterdam.nl/docs/ams-schema-spec.html#tabel-schema

[RdeBaaij]

Oke, ik ga deze dataset weghalen want ik wil alles in Databricks docuementeren.