Parquet: Fix incorrect pruning for nested fields in optional structs

[szehon-ho]

Broken case

An optional struct may be null even when one of its nested fields is required. In that case, dereferencing the nested field produces null.

For example, Spark writes both of these rows to one Parquet file:

CREATE TABLE t (
  id INT NOT NULL,
  stats STRUCT<measurement: DOUBLE NOT NULL>
) USING iceberg;

INSERT INTO t VALUES
  (0, NULL),
  (1, STRUCT(CAST(34.0 AS DOUBLE)));

This query must return id = 0 because stats.measurement is null when stats is null:

SELECT id FROM t WHERE stats.measurement IS NULL;

Before this fix, Spark returned no rows. The double writer reported value_count = 1 and null_value_count = 0 for stats.measurement, instead of 2 and 1. Iceberg therefore concluded that the file could not contain a null measurement and incorrectly pruned it.

Cause and fix

Float and double maintain writer-side metrics to exclude NaN values from bounds. Geometry and geography also maintain writer-side metrics for average value sizes. These metrics replace the Parquet footer metrics, but null optional parents were not propagated to metrics produced by descendant writers.

This change propagates an optional parent's null count to every descendant custom metric while preserving bounds, NaN counts, original types, and average value sizes. The existing nested string test continues to cover the unaffected footer-metrics path.

Testing

• :iceberg-parquet:test --tests "org.apache.iceberg.parquet.TestParquetValueWriters" passes, covering float, double, geometry, and geography beneath a null optional parent, plus the existing direct geospatial case.
• The nested double Spark SQL test passes across all three Spark 4.1 catalog configurations. Before the fix, all 3 cases returned [] instead of [[0]].
• :iceberg-parquet:spotlessJavaCheck and :iceberg-spark:iceberg-spark-4.1_2.13:spotlessJavaCheck pass.

---

AI Disclosure

• Model: GPT-5
• Platform/Tool: Codex
• Human Oversight: partially reviewed
• Prompt Summary: Fix nested Parquet field metrics for float, double, geometry, and geography beneath optional parents, with unit and Spark SQL regression coverage.

[huan233usc]

LGTM thanks