Skip to content

Streaming query validation: detect full-table-scan requirements #93

@vmvarela

Description

@vmvarela

Part of #69
Depends on #92 (chunked CSV reading)

Description

In streaming mode, some SQL operations require seeing the full dataset before producing output (e.g. unbounded ORDER BY, certain JOINs). Detect these cases and emit a clear error or warning instead of silently producing wrong or truncated results.

Acceptance Criteria

  • Queries with unbounded ORDER BY in --stream mode emit a clear error message
  • Queries with multi-table JOINs that aren't supported in streaming mode emit a clear error
  • Error message explains the limitation and suggests removing --stream or rewriting the query
  • Safe queries (WHERE, LIMIT, bounded GROUP BY) pass validation without warnings
  • Validation runs before any data is processed (fail fast)

Notes

  • Parse the query AST or use SQLite's EXPLAIN to detect full-scan operations
  • Keep the validator conservative: false positives (blocking a safe query) are better than false negatives (silently wrong output)

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:mediumShould be done soonsize:sSmall — 1 to 4 hoursstatus:readyRefined and ready for sprint selectiontype:featureNew functionality

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions