Repository navigation
Evaluate moving from chrono to Jiff #4754
Description
Activity
I ran a small spike to scope this concretely, converting one timezone-free expression (
next_day) from chrono to jiff end to end: #4755. Sharing what we learned and a suggested approach for the broader work.Key learnings
-
We do not need to wait on arrow-rs. The Arrow boundary for date/time columns is raw integers (
Date32Array::valuereturnsi32epoch days; timestamps arei64). Our expression code can build jiff types directly from those integers and skip Arrow's chrono-returning helpers (Date32Type::to_naive_date_opt,date32_to_datetime). So Comet's own code can move to jiff today. -
chrono will remain in the dependency tree until arrow-rs migrates, because
arrowdepends on chrono and pullschrono-tzvia its feature. Moving Comet to jiff therefore buys API ergonomics and a single date/time library in our code, but does not shrink the dependency tree on its own. Full chrono removal is a separate downstream goal gated on upstream arrow-rs. -
jiff keeps epoch-day conversion private on purpose (
from_unix_epoch_day/to_unix_epoch_dayare crate-private). The public idiom is an anchor date plus a daySpan:- epoch day to date:
Date::constant(1970, 1, 1).checked_add(days.days()) - date to epoch day:
epoch.until(date).get_days()
This is slightly more verbose than chrono, so a tiny shared helper module is worth adding up front.
- epoch day to date:
-
jiff civil dates span years -9999..=9999, narrower than chrono and Date32. Out-of-range values should map to NULL (matching today's nullable behavior). Not a concern for realistic Spark data, but worth a conscious decision per expression.
-
The real cost is the timezone path, not the calendar math. Date-only expressions are a near-mechanical translation (chrono
Weekday::days_sincemaps 1:1 to jiffWeekday::since, etc.). Timestamp and timezone expressions are where the work is: they need jiff's tzdb features and a rework oftimezone.rs(currently chrono-tz based), plus careful comparison of jiff's tz behavior against Spark and chrono-tz.
Suggested approach
- Phase 1 (low risk): migrate timezone-free date expressions (
next_day,dayname/monthname,make_date, ...) behind a sharedepoch_day <-> jiff::civil::Datehelper. Usejiff = { default-features = false, features = ["std"] }for these since the tzdb bundles are not needed. - Phase 2: migrate timestamp/timezone expressions, enabling jiff's tzdb features and reworking
timezone.rs. Validate tz behavior against Spark. - Phase 3: track arrow-rs for chrono removal; only then can chrono leave
Cargo.lock.
Each phase is independently shippable, and Phase 1 can be split per expression. The spike in #4755 is a working example of the Phase 1 pattern.
-
I'm the author of Jiff and I'd be happy to answer any questions if you have them! Especially around time zones, I should be able to clarify any points of confusion around the differences between Jiff and
chrono-tz. They are quite different at the API level (with Jiff not having aTimeZonetrait), but Jiff's core behavior should be a strict superset ofchrono. (The most notable thing that Jiff handles in a strictly more flexible way thanchronoare gaps in civil datetimes when a time zone transition moves from a smaller UTC offset to a bigger UTC offset. i.e., Typically when a region enters daylight saving time.)jiff keeps epoch-day conversion private on purpose (from_unix_epoch_day / to_unix_epoch_day are crate-private). The public idiom is an anchor date plus a day Span:
It's likely that this will change with a new
jiff::civil::UnixEpochDaytype. I imagine this is quite useful if you want to represent dates as 32-bit integers. Without it, I think you have to roll your own math to convert a Gregorian date to a single day number. Jiff really should own that math because it's quite performance critical and easy to get wrong.Reacted by Andy GroveI'm the author of Jiff and I'd be happy to answer any questions if you have them! Especially around time zones, I should be able to clarify any points of confusion around the differences between Jiff and
chrono-tz. They are quite different at the API level (with Jiff not having aTimeZonetrait), but Jiff's core behavior should be a strict superset ofchrono. (The most notable thing that Jiff handles in a strictly more flexible way thanchronoare gaps in civil datetimes when a time zone transition moves from a smaller UTC offset to a bigger UTC offset. i.e., Typically when a region enters daylight saving time.)jiff keeps epoch-day conversion private on purpose (from_unix_epoch_day / to_unix_epoch_day are crate-private). The public idiom is an anchor date plus a day Span:
It's likely that this will change with a new
jiff::civil::UnixEpochDaytype. I imagine this is quite useful if you want to represent dates as 32-bit integers. Without it, I think you have to roll your own math to convert a Gregorian date to a single day number. Jiff really should own that math because it's quite performance critical and easy to get wrong.Thanks @BurntSushi! I appreciate the offer. We'll try and figure things out ourselves, but will definitely ask for help if we hit anything the LLMs can't manage.
Reacted by Andrew Gallant
What is the problem the feature request solves?
Chrono is being deprecated - chronotope/chrono-tz#231
Recommendation is to use Jiff instead.
Let's explore this.
Describe the potential solution
No response
Additional context
No response