feat(transaction): add OverwriteAction with CoW delete support

[glitchy]

Which issue does this PR close?

Part of #2186

This is the first in a series of PRs working toward full Copy-on-Write (CoW) and Merge-on-Read (MoR) support. CoW comes first because it provides the foundation that MoR eventually depends on. This PR delivers complete CoW overwrite semantics. Subsequent PRs will add RowDeltaAction for writing position/equality delete files (MoR write path), scan-side delete file reconciliation (MoR read path), and compaction.

Adds OverwriteAction, a new TransactionAction that produces snapshots with Operation::Overwrite semantics. It adds new data files and optionally removes existing data files by rewriting affected manifests with entries marked as ManifestStatus::Deleted.

Supporting changes:

• ManifestWriter::add_deleted_entry()--the existing add_entry() unconditionally sets status to Added; there was no way to write deleted entries
• SnapshotProducer::snapshot_id() getter--needed to stamp the snapshot ID on deleted manifest entries
• FastAppendAction::existing_manifest() now preserves delete-only manifests so that deleted entries produced by OverwriteAction survive subsequent appends

Are these changes tested?

5 new unit tests in transaction::overwrite::tests:

• test_empty_data_overwrite_action--error on empty file list
• test_overwrite_snapshot_properties--custom properties flow to snapshot summary
• test_overwrite_incompatible_partition_value--rejects mismatched partition types
• test_overwrite_basic--verifies updates, requirements, operation type, manifest structure, sequence numbers
• test_overwrite_with_deleted_files--end-to-end: append via catalog, overwrite with deletes via catalog, verify original file is Deleted and replacement is Added

[xanderbailey]

Hey! This is exactly the PR I was about to implement but I was going to add the wiring for DataFusion also so you can do INSERT OVERWRITE I think it's not a lot of extra code to modify IcebergCommitExec to take InsertOp. I'm happy to add this is a follow-up PR if you were wanting to focus on the core iceberg part.

[xanderbailey]

Gave it a go here glitchy#1

[glitchy]

Hey! This is exactly the PR I was about to implement but I was going to add the wiring for DataFusion also so you can do INSERT OVERWRITE I think it's not a lot of extra code to modify IcebergCommitExec to take InsertOp. I'm happy to add this is a follow-up PR if you were wanting to focus on the core iceberg part.

Thanks @xanderbailey. Yep I wanted to keep this PR focused on the core Iceberg transaction layer. A follow-up PR wiring it into DataFusion with INSERT OVERWRITE via IcebergCommitExec would be great--happy to review it once this one lands.

[xanderbailey]

Hey! This is exactly the PR I was about to implement but I was going to add the wiring for DataFusion also so you can do INSERT OVERWRITE I think it's not a lot of extra code to modify IcebergCommitExec to take InsertOp. I'm happy to add this is a follow-up PR if you were wanting to focus on the core iceberg part.

Thanks @xanderbailey. Yep I wanted to keep this PR focused on the core Iceberg transaction layer. A follow-up PR wiring it into DataFusion with INSERT OVERWRITE via IcebergCommitExec would be great--happy to review it once this one lands.

Sounds good to me!

[timsaucer]

This is a nice PR!

I did run into a problem when I tried using it with my code. Specifically I was getting a panic when I would delete more files than I added during an overwrite. I wrote up a PR targeting your branch with a proposed solution: glitchy#2