sql: added `shard_pipe` for running run per-shard Polars queries by jpc · Pull Request #19 · HumeAI/wsds

jpc · 2026-01-28T16:58:11Z

📚 PR Stack

#	Branch	Base	PR
1	`jpc/preloading`	`main`	#16
2	`jpc/validate-shards-in-sql`	`jpc/preloading`	#18
3	`jpc/shard-pipe`	`jpc/validate-shards-in-sql`	#19 ◀️
4	`jpc/sql-without-index`	`jpc/shard-pipe`	#20
5	`jpc/special-shard-columns`	`jpc/sql-without-index`	#21
6	`jpc/remove-in-progress`	`jpc/special-shard-columns`	#22
7	`jpc/fix-jupyter-repr`	`jpc/remove-in-progress`	#23
8	`jpc/sql-select-dotted`	`jpc/fix-jupyter-repr`	#24
9	`jpc/optimize-durations`	`jpc/sql-select-dotted`	#25
10	`jpc/source-links`	`jpc/optimize-durations`	#26
11	`jpc/audio-qol`	`jpc/source-links`	#27
12	`jpc/warn-subsampling-in-dataloader`	`jpc/audio-qol`	#28
13	`jpc/wsds-inspect-head`	`jpc/warn-subsampling-in-dataloader`	#29
14	`jpc/feather-index`	`jpc/wsds-inspect-head`	#30
15	`jpc/indexer`	`jpc/feather-index`	#31
16	`jpc/keys-diff`	`jpc/indexer`	#32
17	`jpc/fields-cleanup`	`jpc/keys-diff`	#33
18	`jpc/s3-shards`	`jpc/fields-cleanup`	#34
19	`jpc/indexing-fixes`	`jpc/s3-shards`	#35
20	`jpc/validate-keys-on-the-fly`	`jpc/indexing-fixes`	#36
21	`jpc/repr-missing-last`	`jpc/validate-keys-on-the-fly`	#37
22	`jpc/drop-slots`	`jpc/repr-missing-last`	#38

sql: added shard_pipe for running run per-shard Polars queries

3c94c72