Skip to content

[GH-539] Add dataset for floating point columns#100

Open
prtkgaur wants to merge 5 commits intoapache:masterfrom
prtkgaur:alpFloatingPointDataset
Open

[GH-539] Add dataset for floating point columns#100
prtkgaur wants to merge 5 commits intoapache:masterfrom
prtkgaur:alpFloatingPointDataset

Conversation

@prtkgaur
Copy link

@prtkgaur prtkgaur commented Dec 8, 2025

Add dataset needed for testing performance of floating point values.

@prtkgaur prtkgaur mentioned this pull request Dec 8, 2025
Open
@prtkgaur
Copy link
Author

prtkgaur commented Jan 13, 2026

Add dataset needed for testing performance of floating point values.
Files are in csv format, I did this for easier testing. Let me know if the you (the reviewer thinks otherwise).

@prtkgaur prtkgaur marked this pull request as ready for review January 13, 2026 18:02
@emkornfield
Copy link

emkornfield commented Jan 13, 2026

Thanks for the PR:

  1. Lets finalize on the spec changes first (I think there is still some easy room for reducing size).
  2. Lets at least have two simple examples, where the content is well defined that covers the following cases:
    a. Null values
    b. 0 Exceptions for blocks
    c. 1 exception for a block
    d. >1 exception for a block.
    e. A block with all exceptions.
    f. differning exponent/value per block.

(are there other data variations that are important to cover?)

@platypii platypii mentioned this pull request Jan 25, 2026
Open
@sfc-gh-pgaur
Add ALP-encoded parquet test files and expected CSVs
15315d8 
Generated from floatingpoint_spotify1.csv (9 double columns, 15000 rows)
and floatingpoint_arade.csv (4 double columns, 15000 rows) using the C++
generate_alp_parquet utility with ALP encoding, no dictionary, uncompressed.

These files are used for cross-language ALP correctness testing: read the
parquet file and verify all values match the expected CSV bit-exactly.
@sfc-gh-pgaur
Add Java-generated ALP-encoded parquet test files
16b8d35
@sfc-gh-pgaur
Add float32 ALP-encoded test data files
982d98a 
Add C++ and Java-generated float32 ALP parquet files and expected CSVs
for cross-language interop testing of ALP encoding with FLOAT type.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@prtkgaur @emkornfield @sfc-gh-pgaur