Repository navigation
[VL] Large timestamp outside of range #7109
Description
Backend
VL (Velox)
Bug description
Timestamps that great then 32767 years without but do not overflow will be outside of range.
Seq(
(1723240800675l)
).toDF("time")
spark.sql("""
select from_unixtime(event_timestamp, 'HH') from test
""")
// vanilla: 56577-04-30 00:11:15
// gluten: Timestamp is outside of supported range of [-32767-01-01, 32767-12-31]It may be because std::chrono::time_point is used in time conversion, which has range limitations.
Spark version
None
Spark configurations
No response
System information
No response
Relevant logs
No response
Activity
zml1206 commentedon Sep 4, 2024
Seems to be a known issue facebookincubator/velox#10641
ccat3z commentedon Sep 4, 2024
It maybe not same issue, facebookincubator/velox#10641 still use std::chrono. But the expected result is 56577, which std::chrono not supported.
ccat3z commentedon Sep 4, 2024
I am trying to remove the dependency on std::chrono to get rid of the limitations.
wForget commentedon May 23, 2025
@NEUpanning Does #7375 resolve this issue?
NEUpanning commentedon May 23, 2025
@wForget No. See task 'Fix date range validation in SimpleDateFormat standard' part in facebookincubator/velox#10354