[fuziontech]

Problem

The duckling events backfill DAG started failing with a ClickHouse S3 error during the Parquet export step:

Code: 499. DB::Exception: The specified bucket does not exist.

The regression traces to #63834 (merged the day before this fix). That PR replaced DuckLakeCatalog with DuckgresServer in the backfill connection path and, as part of it, stopped reading the org's bucket name from the stored DuckLakeCatalog.bucket field and instead recomputed it deterministically as posthog-duckling-{org}-{suffix} via derive_duckling_bucket.

DuckgresServer has no bucket field, so the new code had nothing to read from and reconstructed the name from the org id. When an org's actually-provisioned bucket name doesn't match that exact pattern (or the env suffix resolves differently than expected in the Dagster runtime), ClickHouse is handed a bucket that doesn't exist and the export blows up — surfacing as a raw ClickHouse 499 deep inside the op rather than a readable error.

Changes

_resolve_duckling_target now reads bucket / bucket_region from the org's stored DuckLakeCatalog row — the source of truth written at provisioning time — and only falls back to the deterministic derive_duckling_bucket derivation when there is no catalog row (e.g. dev mode, where the lookup returns None).

This restores the pre-#63834 behavior for the bucket while keeping the new duckgres-backed connection path intact. The catalog lookup reuses the existing get_ducklake_catalog_for_organization helper.

How did you test this code?

I'm an agent (Claude Opus 4.8). Automated checks I actually ran:

• isolated ruff check and ruff format --check (line-length 120) on the modified file — both clean
• python3 -c "import ast; ast.parse(...)" syntax check

I did not run the full Python suite locally (the local venv is stale and hogli/flox were unavailable). Relying on CI for the full lint + test run. No automated test was added — the change is a narrow source-of-truth swap in an existing resolver; a unit test asserting "stored catalog bucket wins over derived" would be a reasonable follow-up.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Debugged from a Dagster stack trace the user pasted. Walked the call chain (duckling_events_backfill → export_events_to_duckling_s3 → INSERT INTO FUNCTION s3(...)) to the ClickHouse 499, then used git log/git show to bisect the bucket-resolution change to #63834. Confirmed DuckgresServer carries no bucket field while DuckLakeCatalog.bucket is still populated, so the fix reads from the catalog and falls back to the derived name only when no row exists.

No skills were applicable (Dagster DAG / ClickHouse export, not a DRF/migration/frontend surface).

🤖 Generated with Claude Code

[github-actions]

Hey @fuziontech! 👋

It looks like your git author email on this PR isn't your @posthog.com address (fuziontech@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[Copilot]

Pull request overview

Fixes the duckling events backfill Dagster job’s S3 Parquet export failures by restoring the bucket/region source-of-truth to the organization’s persisted DuckLakeCatalog row (written at provisioning time), while keeping the newer duckgres-backed connection path.

Changes:

• Update _resolve_duckling_target to prefer DuckLakeCatalog.bucket / bucket_region for the org.
• Fall back to derive_duckling_bucket(org_id) only when no catalog row exists (e.g. dev mode).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[greptile-apps]

Comments Outside Diff (1)

1. posthog/dags/events_backfill_to_duckling.py, line 177-183 (link)

   The DucklingTarget docstring now contradicts the fix: _resolve_duckling_target does make a catalog-DB lookup (via get_ducklake_catalog_for_organization) and the bucket is preferentially read from that stored row, not deterministically derived. Any developer reading the class docstring will have the wrong mental model.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: posthog/dags/events_backfill_to_duckling.py
   Line: 177-183
   
   Comment:
   The `DucklingTarget` docstring now contradicts the fix: `_resolve_duckling_target` does make a catalog-DB lookup (via `get_ducklake_catalog_for_organization`) and the bucket is preferentially read from that stored row, not deterministically derived. Any developer reading the class docstring will have the wrong mental model.
   
   
   
   How can I resolve this? If you propose a fix, please make it concise.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
posthog/dags/events_backfill_to_duckling.py:177-183
The `DucklingTarget` docstring now contradicts the fix: `_resolve_duckling_target` does make a catalog-DB lookup (via `get_ducklake_catalog_for_organization`) and the bucket is preferentially read from that stored row, not deterministically derived. Any developer reading the class docstring will have the wrong mental model.

```suggestion
    """Resolved per-org duckling backfill target: connection identity + S3 storage.

    Built once per run from the team's organization id. The duckgres connection is driven
    by make_duckgres_conninfo; the S3 bucket is read from the org's stored DuckLakeCatalog
    row (provisioning source of truth) and only falls back to deterministic derivation when
    no catalog row exists (e.g. dev mode).
    """
```

_{Reviews (1): Last reviewed commit: "chore(dags): warn when duckling bucket f..." | Re-trigger Greptile}

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-06-19 17:43 UTC	Run
prod-us	✅ Deployed	2026-06-19 17:54 UTC	Run
prod-eu	✅ Deployed	2026-06-19 17:57 UTC	Run