Skip to content

test: add mixed-format schema evolution reads#366

Closed
liujiwen-up wants to merge 1 commit into
apache:mainfrom
liujiwen-up:fix/258-mixed-format-reorder-move-column
Closed

test: add mixed-format schema evolution reads#366
liujiwen-up wants to merge 1 commit into
apache:mainfrom
liujiwen-up:fix/258-mixed-format-reorder-move-column

Conversation

@liujiwen-up
Copy link
Copy Markdown
Contributor

@liujiwen-up liujiwen-up commented Jun 5, 2026

Purpose

Linked issue: close #258

Add Spark-provisioned consistency coverage for mixed-format schema evolution reads. The new cases cover projection and filter interactions across Java/Spark-generated Parquet, ORC, and Avro data files.

Brief change log

  • Add mixed-format fixtures for add-column, type-promotion, and reorder/move-column schema evolution.
  • Cover projection on only an added column and reordered projection such as age, id.
  • Cover filters on an added column with age IS NULL and age = 50 while preserving projection order.
  • Cover filter columns that are not part of projection for mixed-format type-promotion reads.
  • Verify mixed-format reorder/move-column reads map values by field id rather than physical position.

Tests

  • cargo fmt: passed
  • git diff --check: passed
  • cargo test -p paimon-integration-tests --test read_tables --no-run: passed
  • Runtime integration tests were not run locally because Docker daemon was unavailable, so the Spark warehouse fixtures could not be provisioned.

API and Format

No API or storage format changes. This PR only adds Spark provisioning and Rust integration test coverage.

Documentation

No documentation update required.

@liujiwen-up liujiwen-up changed the title test: cover mixed-format schema evolution reads test: add mixed-format schema evolution reads Jun 5, 2026
@liujiwen-up liujiwen-up closed this Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add more consistency file tests from Java data files

1 participant