Skip to content

fix: Native engine crashes on literal DateTrunc and TimestampTrunc#3668

Open
0lai0 wants to merge 3 commits intoapache:mainfrom
0lai0:native_engine_literal_DateTrunc
Open

fix: Native engine crashes on literal DateTrunc and TimestampTrunc#3668
0lai0 wants to merge 3 commits intoapache:mainfrom
0lai0:native_engine_literal_DateTrunc

Conversation

@0lai0
Copy link
Contributor

@0lai0 0lai0 commented Mar 11, 2026

Which issue does this PR close?

Closes #3342

Rationale for this change

When ConstantFolding is disabled, literal trunc() and date_trunc() calls reach the native engine with both arguments as Scalar. The existing match arms only handled (Array, Scalar) and (Array, Array) , so the (Scalar, Scalar) case fell through to the error branch and crashed.

What changes are included in this PR?

  • Use into_array(num_rows) to convert the date argument to an Array before matching on format, so all input combinations are handled uniformly.

  • Convert the timestamp argument to an Array via into_array(num_rows) before matching on format.

How are these changes tested?

Added test cases in CometTemporalExpressionSuite that disable ConstantFolding and verify literal trunc() and date_trunc() queries execute correctly on Comet and produce results matching Spark.

@mbutrovich
Copy link
Contributor

Broadcasting scalars to arrays to fit existing API limitations seems like a hack. It also means in the (scalar, scalar) case we return an array, when we could also return a scalar.

@0lai0
Copy link
Contributor Author

0lai0 commented Mar 12, 2026

Thanks @mbutrovich for the reminder—I’ve updated the PR.

Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are already SQL file tests for this exact scenario in trunc_date.sql and trunc_timestamp.sql. The literal queries on lines 36-37 and 39-40 respectively are currently marked with ignore(https://github.com/apache/datafusion-comet/issues/3342), which is the issue this PR closes. It would be great to remove those ignore annotations so the existing SQL tests cover this fix. That would also mean the new Scala tests in CometTemporalExpressionSuite might not be needed since the SQL file tests already cover multiple format strings (year, month, quarter, day) rather than just year.

@0lai0
Copy link
Contributor Author

0lai0 commented Mar 14, 2026

Thanks @andygrove comment. PR updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Native engine crashes on literal DateTrunc and TimestampTrunc

3 participants