[nicoloangileri]

Problem

predict_df groups rows by item in first-appearance order (normalize_df) and then slices each series using lengths from df[id_column].value_counts(sort=False). Those lengths only match the row blocks for some id dtypes:

• for a categorical id column, value_counts follows the category order (sorted by default in pd.Categorical) and also counts categories that have no rows;
• on pandas 2.2 it sorts string[python] ids, which is [BUG] Cannot infer frequency when id_column has string[python] dtype #440.

When the lengths do not match the rows, every series gets the wrong slice. With the frequency inferred, this shows up as a misleading error on regular data. With an explicit freq there is no error, and the forecasts are attached to the wrong series and dates:

import numpy as np
import pandas as pd
from chronos import BaseChronosPipeline

pipeline = BaseChronosPipeline.from_pretrained("amazon/chronos-2", device_map="cpu")
rng = np.random.default_rng(0)
df = pd.concat(
    [
        pd.DataFrame({"item_id": "ETH", "timestamp": pd.date_range(end="2026-08-24", periods=300, freq="D"),
                      "target": 2500 + 20 * rng.standard_normal(300).cumsum()}),
        pd.DataFrame({"item_id": "BTC", "timestamp": pd.date_range(end="2026-09-23", periods=500, freq="D"),
                      "target": 80000 + 500 * rng.standard_normal(500).cumsum()}),
    ],
    ignore_index=True,
)
df_cat = df.assign(item_id=pd.Categorical(df["item_id"]))  # categories are sorted: BTC, ETH

pipeline.predict_df(df_cat, prediction_length=2)
# ValueError: Could not infer frequency for series ETH

pipeline.predict_df(df_cat, prediction_length=2, freq="D")[["item_id", "timestamp", "predictions"]]
#   item_id  timestamp   predictions
# 0     ETH 2025-11-28  78949.562500   <- forecast from BTC values, dated inside BTC's history
# 1     ETH 2025-11-29  78946.390625
# 2     BTC 2026-09-24  74845.507812   <- context cut short
# 3     BTC 2026-09-25  74842.585938

With object ids the same data gives ETH 2278.51 from 2026-08-25 and BTC 74867.05 from 2026-09-24.

Change

A new get_series_lengths(df, id_column) in df_utils.py counts rows per id with pd.factorize + np.bincount: the same first-appearance order normalize_df groups by, only ids that have rows, and the lengths always sum to len(df), whatever the id dtype. It replaces value_counts(sort=False) at the seven places that computed series lengths:

• df_utils.py: infer_freq_from_df, make_future_df, the future_df row-count check in validate_df, and the deprecated convert_df_input_to_list_of_dicts_input (both branches);
• base.py: BaseChronosPipeline.predict_df, used by Chronos and Chronos-Bolt;
• chronos2/preprocess.py: from_data_frame, where a comment stated the ordering assumption.

Before and after, with categorical ids (Chronos-2 unless noted):

path	on main	with this PR
inferred freq	ValueError: Could not infer frequency for series ...	same forecasts as object ids
explicit freq="D"	wrong forecasts, no error	same forecasts as object ids
with future_df, a category without rows	ValueError: future_df must contain prediction_length=3 rows per item. Found 1 items ...	same forecasts as object ids
validate_inputs=False	ValueError: Could not infer frequency ...	same forecasts as object ids
Chronos-Bolt predict_df	ValueError: Could not infer frequency ...	same forecasts as object ids

Relation to #440 and #470

This also fixes #440: on pandas 2.2.3, infer_freq_from_df on the issue's kind of data (many weekly series, hashed string[python] ids, unequal lengths) raises on main and returns W-MON with this change. It does not reproduce on pandas 3.

Credit to @MohammedAlkindi, who noted on #470 that the pattern survived the #507 refactor and listed the same call sites; #470 itself targets functions that no longer exist. Two things differ from that analysis. For categorical ids the future_df row-count check in validate_df is affected too, since a category without rows is counted with zero rows. And groupby(sort=False).size(), the approach of #470, still counts unused categories on pandas 2.2 ([2, 3, 0] for two series and one empty category), while factorize + bincount gives [2, 3].

Fixes #440

Tests

• test_get_series_lengths_follows_row_order: categorical, categorical with an unused category, and string[python] ids.
• test_infer_freq_and_make_future_df_handle_categorical_ids.
• test_predict_df_with_categorical_item_ids_matches_object_ids for Chronos-2 and Chronos-Bolt, with inferred and explicit freq, with and without an unused category. The eight cases fail on main and pass with this change.

pytest: 568 passed, 20 xfailed, on both pandas 3.0.6 and pandas 2.2.3. mypy src test: no issues.

[nicoloangileri]

Hi @shchur @abdulfatir, a gentle ping on this one when you have a moment. It's a small fix, and I'm happy to adjust anything or rebase if needed. Thanks!

[shchur]

Thank you @nicoloangileri for the fix!