Apache Iceberg version
main @ 9d7b2c5
Query engine
None — the bug is in the MapReduce/Hive input format (iceberg-mr).
Please describe the bug
IcebergInputFormat.IcebergRecordReader.openTask() (mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.java line 310) builds the reader without idToConstant(...), so identity partition values recorded only in the manifest are never handed to the reader. Reading such a file returns null in the partition columns.
This is a regression from a2802c4 (#15333), which merged the Avro, ORC and Parquet branches into one ReadBuilder. Before that change all three passed constantsMap(task, IdentityPartitionConverters::convertConstant). The sibling reader touched by the same PR, data/src/main/java/org/apache/iceberg/data/GenericReader.openFile(), kept .idToConstant(partition).
Steps to reproduce
- Create an identity-partitioned table.
- Add a data file that does not physically contain the partition columns, with the partition values set on the manifest entry.
TableMigrationUtil.listPartition(...) produces exactly this — it takes partition values from Hive metadata, not from the file — so any table imported with add_files or migrate is in this state.
- Read the table through
IcebergInputFormat.
Expected: the partition columns hold the manifest values. Actual: they are null.
Additional context
N/A
Apache Iceberg version
main @ 9d7b2c5
Query engine
None — the bug is in the MapReduce/Hive input format (
iceberg-mr).Please describe the bug
IcebergInputFormat.IcebergRecordReader.openTask()(mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.javaline 310) builds the reader withoutidToConstant(...), so identity partition values recorded only in the manifest are never handed to the reader. Reading such a file returnsnullin the partition columns.This is a regression from a2802c4 (#15333), which merged the Avro, ORC and Parquet branches into one
ReadBuilder. Before that change all three passedconstantsMap(task, IdentityPartitionConverters::convertConstant). The sibling reader touched by the same PR,data/src/main/java/org/apache/iceberg/data/GenericReader.openFile(), kept.idToConstant(partition).Steps to reproduce
TableMigrationUtil.listPartition(...)produces exactly this — it takes partition values from Hive metadata, not from the file — so any table imported withadd_filesormigrateis in this state.IcebergInputFormat.Expected: the partition columns hold the manifest values. Actual: they are
null.Additional context
N/A