feat(python/sedonadb): add Python GDAL configuration API#689
Draft
Kontinuation wants to merge 21 commits intoapache:mainfrom
Draft
feat(python/sedonadb): add Python GDAL configuration API#689Kontinuation wants to merge 21 commits intoapache:mainfrom
Kontinuation wants to merge 21 commits intoapache:mainfrom
Conversation
Add the sedona-gdal crate providing runtime-loaded GDAL FFI bindings via libloading. This includes the SedonaGdalApi function-pointer struct, dynamic symbol loading, the GdalApi handle with call_gdal_api! macro, error types, and global API registration.
- Introduce GdalApiBuilder with lazy initialization pattern (similar to sedona-proj's ProjCrsEngineBuilder): configure_global_gdal_api now stores a builder in a Mutex, and get_global_gdal_api builds the API on first access. This allows callers to override configuration before the first GDAL operation. - Add compile_error! for unsupported platforms (non-unix, non-windows) in current_process_library. - Add 9 unit tests covering loading errors, symbol loading, builder configuration, and global API access.
Kontinuation
changed the title
Contributor
There was a problem hiding this comment.
Pull request overview
Adds first-class GDAL dynamic-loading configuration to SedonaDB’s Python package, backed by a new sedona-gdal Rust crate and a pyo3-exposed configuration hook.
Changes:
- Introduces
sedona-gdalcrate to dynamically load GDAL symbols and manage a global GDAL API. - Adds
configure_gdal_shared(Rust/pyo3) andconfigure_gdal()(Python) with multiple discovery presets. - Auto-attempts GDAL configuration on
sedonadbimport and updates CI to install GDAL dev libs.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| rust/sedona/Cargo.toml | Adds sedona-gdal dependency and gdal feature wiring. |
| python/sedonadb/Cargo.toml | Adds sedona-gdal workspace dependency for Python extension. |
| python/sedonadb/src/lib.rs | Exposes configure_gdal_shared to Python via pyo3. |
| python/sedonadb/python/sedonadb/context.py | Implements configure_gdal() and GDAL library discovery presets. |
| python/sedonadb/python/sedonadb/init.py | Calls configure_gdal(preset="auto") on import. |
| c/sedona-gdal/Cargo.toml | Defines new sedona-gdal crate and feature flags for gdal-sys. |
| c/sedona-gdal/src/lib.rs | Crate module structure and public re-exports. |
| c/sedona-gdal/src/register.rs | Global GDAL API builder/config and lazy initialization with driver registration. |
| c/sedona-gdal/src/dyn_load.rs | Dynamic symbol loading implementation (libloading). |
| c/sedona-gdal/src/gdal_api.rs | GdalApi wrapper around loaded function pointers and CPL error helper. |
| c/sedona-gdal/src/gdal_dyn_bindgen.rs | Generated-ish bindings struct/types for dynamically loaded GDAL API surface. |
| c/sedona-gdal/src/errors.rs | Error types for init/loading and runtime GDAL errors. |
| Cargo.toml | Adds c/sedona-gdal to workspace and workspace dependency list. |
| Cargo.lock | Adds sedona-gdal and gdal-sys entries. |
| .github/workflows/rust.yml | Installs libgdal-dev in Rust CI. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
|
Just a note that I have a local windows machine that I found quite helpful when writing the PROJ loader and I'm happy to debug something here if you're running into an issue. |
Kontinuation
force-pushed
the
feat/sedona-gdal-python
branch
2 times, most recently
from
d0df55b to
26ea16a
Compare
Member
Author
|
I asked a coding agent to work on multi-platform CIs, and it pinpointed the problem loading C++ symbol on Windows, but didn't come up with a correct fix. I have worked on a proper fix manually and now it should be fine. The mangled name of MEMDataset::Create works for GDAL>=3.5, I have downloaded every pyogrio versions from 0.5.0 to 0.12.1 to verify that. Apparently a better solution is to add MEMCreate to the upstream GDAL C API. |
Kontinuation
force-pushed
the
feat/sedona-gdal-python
branch
2 times, most recently
from
f322ce1 to
45f89ed
Compare
… verification - Add GDALVersionInfo to FFI dynamic bindings (gdal_dyn_bindgen.rs, dyn_load.rs) - Add version_info() method on GdalApi (gdal_api.rs) - Add gdal_version() pyo3 function in lib.rs (returns Option<String>) - Add gdal_version() Python wrapper in context.py and export from __init__.py - Add test_gdal.py pytest that verifies GDAL loads and reports version >= 3.x
Kontinuation
force-pushed
the
feat/sedona-gdal-python
branch
from
45f89ed to
73f6b85
Compare
…d, prioritize rasterio than pyogrio
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
configure_gdal()Python API to thesedonadbpackage that automatically locates and loads GDAL's shared library at import timeauto,pyogrio,rasterio,conda,homebrew,system— covering common GDAL installation methodsgdal_version()to query the loaded GDAL library version from Python.so/ macOS.dylib/ Windows.dll+ condaLibrary/bin/layout)test_gdal.pypytest that verifies GDAL loads successfully and reports a version >= 3.xDependencies