S3 stage add query

[abhishek-pattern]

Replace stage overwrite  with include_query_id

[Copilot]

Pull request overview

This PR bumps ds-platform-utils to 0.4.1 and updates the Snowflake→S3 unload COPY INTO query generation, alongside a small lint-annotation change in the S3-stage loader helper.

Changes:

• Bump package version from 0.4.0 to 0.4.1 (project + lockfile).
• Update Snowflake→S3 COPY INTO options to include INCLUDE_QUERY_ID = TRUE (replacing OVERWRITE = TRUE).
• Remove the # noqa: PLR0913 suppression from _copy_s3_to_snowflake.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated no comments.

File	Description
uv.lock	Locks version bump to 0.4.1.
src/ds_platform_utils/metaflow/s3_stage.py	Adjusts COPY INTO unload options and modifies lint suppression for a multi-arg function.
pyproject.toml	Updates project version to 0.4.1.

Comments suppressed due to low confidence (2)

src/ds_platform_utils/metaflow/s3_stage.py:80

• _generate_snowflake_to_s3_copy_query replaced OVERWRITE = TRUE with INCLUDE_QUERY_ID = TRUE, which changes COPY INTO behavior when exporting to a caller-provided s3_path that already contains files (previously would overwrite; now default is non-overwrite and may error/leave stale files). Consider keeping OVERWRITE = TRUE (and adding INCLUDE_QUERY_ID = TRUE in addition), or making overwrite configurable via a function parameter so existing callers don’t get a breaking behavior change.

    INCLUDE_QUERY_ID = TRUE
    FILE_FORMAT = (TYPE = 'parquet')
    MAX_FILE_SIZE = {max_file_size}
    HEADER = TRUE
    DETAILED_OUTPUT = TRUE;

src/ds_platform_utils/metaflow/s3_stage.py:203

• Removing # noqa: PLR0913 from _copy_s3_to_snowflake will make Ruff flag this function for too many arguments (it currently has 8 parameters, and the repo enables Ruff PL rules). Either re-add the noqa (with a short rationale like other functions in the repo), or refactor the signature (e.g., group related options into a config object) to satisfy PLR0913.

def _copy_s3_to_snowflake(  # noqa: PLR0913
    s3_path: str,
    table_name: str,
    table_definition: Optional[List[Tuple[str, str]]] = None,
    warehouse: Optional[Union[Literal["XS", "MED", "XL"], str]] = None,
    use_utc: bool = True,
    auto_create_table: bool = False,
    overwrite: bool = False,
    use_logical_type: bool = True,

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[vinay79n]

looks good to me