Skip to content

feat: add project secret api keys - #39865

Closed
sakce wants to merge 1 commit into
masterfrom
feat/project-scoped-api-keys
Closed

sakce wants to merge 1 commit into
masterfrom
feat/project-scoped-api-keys

Conversation

@sakce

@sakce sakce commented Oct 17, 2025 •

Copy link
Copy Markdown
Contributor

Problem

Customers have explicitly asked for project-level API keys, so when their employees leave the company their API keys are not breaking their apps.

But we also already have a Feature Flags use case that has a token that is not tied to a user. (long term, we would move that over to project secret API keys)

Closes #843

Changes

Introducing a ProjectSecretAPIKey model, along with a new Permission that only allows specific scopes (we don't want this key type to allow all endpoints, read the discussion in the RFC)

Also implements this auth/permission on the Endpoints model, so that we see it in action. Will happily break this out into a separate PR after validating approach here.

Pretty much clones the frontend approach we have for PersonalAPIKeys (we can refactor PersonalAPIKeys to reuse this later on).

How did you test this code?

  • lots of unit tests

👉 Stay up-to-date with PostHog coding conventions for a smoother review.

Changelog: (features only) Is this feature complete?

Comment thread posthog/api/routing.py Resolved
Comment thread posthog/clickhouse/query_tagging.py Resolved
Comment thread frontend/src/lib/scopes.tsx Resolved
Comment thread posthog/api/routing.py Resolved
Comment thread products/endpoints/backend/api.py Resolved
@sakce

sakce commented Oct 20, 2025

Copy link
Copy Markdown
Contributor Author

Okay, useful input, thanks gents @haacked & @benjackwhite 🎩

What if we go about it this way - only allow this key type (SDK naming TBC) on specific actions that make our customers' apps run, i.e. endpoint execution, FF local evaluation, and such.

Configuration (CRUD) is done by humans, execution by machines.

This way, we exclusively specify the permission_ and authentication_ classes on the route that needs it, we avoid creating a new ViewSet for these endpoints, we don't have to worry about the created_by_X issue, and we don't add the condition inside the TeamAndOrgViewSetMixin?

@Piccirello

Copy link
Copy Markdown
Member

The model LGTM and covers the feedback I shared in https://github.com/PostHog/product-internal/pull/843. Will leave the more important permissioning points to the folks with more context.

@haacked

haacked commented Oct 20, 2025

Copy link
Copy Markdown
Contributor

What if we go about it this way - only allow this key type (SDK naming TBC) on specific actions that make our customers' apps run, i.e. endpoint execution, FF local evaluation, and such.

What's "endpoint execution" in this context?

@github-actions

github-actions Bot commented Oct 21, 2025 •

Copy link
Copy Markdown
Contributor

Size Change: +697 B (+0.02%)

Total Size: 3.7 MB

ℹ️ View Unchanged
Filename Size Change
frontend/dist/toolbar.js 3.7 MB +697 B (+0.02%)

compressed-size-action

@sakce

sakce commented Oct 21, 2025 •

Copy link
Copy Markdown
Contributor Author

What's "endpoint execution" in this context?

@haacked, I'm referring to the new Endpoints product that we're building. Essentially a pre-defined query meant to power B2B2C dashboards.

Endpoint execution is the /api/projects/2/endpoints/event-count/run route. See the changed files in products/endpoints/backend of this PR for more.

@sakce
sakce force-pushed the feat/project-scoped-api-keys branch from 58175cc to 4194f11 Compare October 21, 2025 09:07

@sakce sakce left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored without touching the TeamAndOrgViewSet 👀

Comment thread posthog/permissions.py Resolved
Comment on lines +236 to +237
authentication_classes=[ProjectSecretAPIKeyAuthentication],
permission_classes=[ProjectSecretAPIKeyPermission],

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the key difference - we only set this on the action we want.

@posthog-bot

This comment was marked as off-topic.

@sakce
sakce marked this pull request as ready for review October 21, 2025 16:14
@sakce
sakce requested a review from a team as a code owner October 21, 2025 16:14
@posthog-project-board-bot posthog-project-board-bot Bot moved this from Todo to In Review in Feature Flags Oct 21, 2025
@posthog-bot
posthog-bot requested a review from a team October 21, 2025 16:16
@sakce
sakce force-pushed the feat/project-scoped-api-keys branch from b50c23b to c8e1b06 Compare October 22, 2025 08:49

sakce commented Oct 22, 2025 •

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@joshsny

joshsny commented Oct 22, 2025 •

Copy link
Copy Markdown
Contributor

How strongly do customers want this? I can totally see the benefit, but I think there is also a major drawback to introducing new keys for users to understand, which I've already seen happening with the feature flags key we introduced.

We could make the UX of creating personal api keys for a team much better, or even create them by default for a team. We could also handle transferring the team scoped keys in the UX for removing a team member from a team to avoid the user leaves and everything breaks problem.

Having tokens team scoped feels like the right thing to do, but not sure if it's worth the added complexity for users and for us if there are other ways around it to keep request.user working nicely.

@benjackwhite benjackwhite left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally this is a fantastic PR. Some changes I think we need to make and then we can roll with it

Comment thread frontend/src/lib/api.ts Resolved
Comment thread posthog/api/project_secret_api_key.py Resolved
capture_exception(e)
raise ValidationError("Failed to create project secret API key.")

def validate_scopes(self, scopes):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kind of thing is why I prefer the DRF serializers as it makes it more encapsulated. Not blocking but the code in this viewset feels much more verbose than the personal_api_key one for example which is very similar...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im with you on verbosity - I'll 📎 this for cleanup in a subsequent PR.

@sakce sakce Oct 22, 2025 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And maybe we get Julian to give input on use of PydanticModelMixin by then

Comment thread posthog/auth.py Resolved
Comment thread posthog/auth.py Resolved
Comment thread posthog/permissions.py Resolved
@sakce
sakce force-pushed the feat/project-scoped-api-keys branch 3 times, most recently from ea0a90b to fa2245b Compare October 22, 2025 13:59
@haacked

haacked commented Nov 18, 2025

Copy link
Copy Markdown
Contributor

do you know if these changes around ProjectSecretAPIKeyUser would shake up Django and impact FF's when we deploy?

I don't think it would. I can try and test this branch out soon, but we have a lot going on right now so probably won't get to it immediately.

@posthog-bot

Copy link
Copy Markdown
Contributor

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week. If you want to permanentely keep it open, use the waiting label.

@sakce sakce added waiting Prevents stale-bot from marking the PR as stale. and removed stale labels Nov 27, 2025
@posthog-bot

Copy link
Copy Markdown
Contributor

Visual regression: Storybook UI snapshots updated

Changes: 2 snapshots (2 modified, 0 added, 0 deleted)

What this means:

  • Snapshots have been automatically updated to match current rendering
  • Next CI run will switch to CHECK mode to verify stability
  • If snapshots change again, CHECK mode will fail (indicates flapping)

Next steps:

  • Review the changes to ensure they're intentional
  • Approve if changes match your expectations
  • If unexpected, investigate component rendering

Review snapshot changes →

@sakce
sakce force-pushed the feat/project-scoped-api-keys branch from 62b50bc to f2fb85f Compare December 22, 2025 17:28
Comment thread products/endpoints/backend/api.py Resolved
Comment thread products/endpoints/backend/api.py Resolved
@cursor

cursor Bot commented Dec 22, 2025

Copy link
Copy Markdown

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
Learn more about Cursor Agents

@sakce
sakce force-pushed the feat/project-scoped-api-keys branch from 25dd9d5 to b780509 Compare December 22, 2025 20:18

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9 issues found across 37 files

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="frontend/src/queries/schema.json">

<violation number="1" location="frontend/src/queries/schema.json:21400">
P2: The `scopes` array items should reference `ProjectSecretAPIKeyAllowedScope` enum to ensure proper schema validation. Currently any string is accepted.</violation>
</file>

<file name="products/endpoints/backend/api.py">

<violation number="1" location="products/endpoints/backend/api.py:205">
P1: Logic bug: changing from `is not None` to truthiness check breaks `is_active=False` handling. When a user explicitly sets `is_active=False`, this condition evaluates the falsy `False` value and incorrectly defaults to `True`. The original check `data.is_active is not None` was correct.</violation>
</file>

<file name="posthog/permissions.py">

<violation number="1" location="posthog/permissions.py:496">
P1: Missing null check for `required_scopes` before passing to `_check_required_scope_satisfied`. When `_get_required_scopes` returns `None`, this will raise a `TypeError` when iterating. The existing `APIScopePermission` class handles this case properly.</violation>
</file>

<file name="posthog/api/project_secret_api_key.py">

<violation number="1" location="posthog/api/project_secret_api_key.py:156">
P2: Update method allows setting an empty label when whitespace-only strings are passed. The create method properly validates `if not data.label or not data.label.strip()`, but update only checks `if data.label:` before stripping. A whitespace-only label like `&quot;   &quot;` would pass the truthy check but strip to an empty string.</violation>

<violation number="2" location="posthog/api/project_secret_api_key.py:203">
P2: Inconsistent use of `get_object_or_404` instead of `self.get_object()`. Other methods (retrieve, update, roll) use the standard `self.get_object()` which respects DRF&#39;s built-in permission checks and queryset filtering. Using `get_object_or_404` directly bypasses this standard flow.</violation>
</file>

<file name="posthog/api/test/test_project_secret_api_keys.py">

<violation number="1" location="posthog/api/test/test_project_secret_api_keys.py:286">
P2: Test is incomplete: missing status code assertion and verification that the key actually changed. Consider capturing the original `mask_value` before rolling and asserting it differs afterward, plus adding a status code check.</violation>
</file>

<file name="frontend/src/scenes/settings/project/ProjectAPIKeys.tsx">

<violation number="1" location="frontend/src/scenes/settings/project/ProjectAPIKeys.tsx:116">
P3: Grammatical error: &#39;allow programmatic access a very limited set&#39; is missing &#39;to&#39;. Should be &#39;allow programmatic access to a very limited set&#39;.</violation>

<violation number="2" location="frontend/src/scenes/settings/project/ProjectAPIKeys.tsx:120">
P3: Grammatical error: &#39;scopes that allow to the project&#39;s data&#39; should be &#39;scopes that allow access to the project&#39;s data&#39;.</violation>
</file>

<file name="frontend/src/scenes/settings/shared/APIKeyTable.tsx">

<violation number="1" location="frontend/src/scenes/settings/shared/APIKeyTable.tsx:132">
P1: Logic is inverted: `actionsDisabled` being `false` (default) hides the actions column, while `true` shows it. The condition should be `!actionsDisabled` to match the prop name semantics.</violation>
</file>

Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR

Comment thread frontend/src/queries/schema.json Resolved
Comment thread products/endpoints/backend/api.py Resolved
Comment thread posthog/permissions.py Resolved
Comment thread posthog/api/project_secret_api_key.py Resolved
Comment thread posthog/api/project_secret_api_key.py Resolved
@sakce
sakce force-pushed the feat/project-scoped-api-keys branch 2 times, most recently from f1a8dd3 to d88a134 Compare December 23, 2025 12:49
@tests-posthog

tests-posthog Bot commented Dec 23, 2025

Copy link
Copy Markdown
Contributor

Visual regression: Storybook UI snapshots updated

Changes: 2 snapshots (2 modified, 0 added, 0 deleted)

What this means:

  • Snapshots have been automatically updated to match current rendering
  • Next CI run will switch to CHECK mode to verify stability
  • If snapshots change again, CHECK mode will fail (indicates flapping)

Next steps:

  • Review the changes to ensure they're intentional
  • Approve if changes match your expectations
  • If unexpected, investigate component rendering

Review snapshot changes →

@sakce
sakce force-pushed the feat/project-scoped-api-keys branch from 968d169 to 49343ac Compare December 23, 2025 14:40

@haacked haacked left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me take some more time to look at this. But found a few issues worth looking at.

Comment thread products/endpoints/backend/api.py Resolved
Comment thread posthog/api/project_secret_api_key.py Resolved
Comment thread posthog/api/project_secret_api_key.py Resolved
Comment thread posthog/api/project_secret_api_key.py Resolved
@sakce
sakce force-pushed the feat/project-scoped-api-keys branch from 49343ac to 0762510 Compare January 2, 2026 13:12

sakce commented Jan 2, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favour of the stack splitting this in smaller chunks

#44200 feat: add project secret api key support for Endpoints execution
#44199 feat: add project secret api keys to frontend
#44198 feat: add project secret api key API & authenticator
#44197 feat: add project api key model

@sakce sakce closed this Jan 2, 2026
@github-project-automation github-project-automation Bot moved this from In Review to Done in Feature Flags Jan 2, 2026
sakce added a commit that referenced this pull request Feb 3, 2026
## Problem

<!-- Who are we building for, what are their needs, why is this important? -->

<!-- Does this fix an issue? Uncomment the line below with the issue ID to automatically close it when merged -->
<!-- Closes #ISSUE_ID -->

Splits #39865 - second part. API and authenticator changes.

## Changes

<!-- If there are frontend changes, please include screenshots. -->
<!-- If a reference design was involved, include a link to the relevant Figma frame! -->
- adds an authenticator class for ProjectSecretAPIKey
- adjusts the permissions
- adds the API for project secret API key (pretty much the same thing as personal api key)

## How did you test this code?
- locally
<!-- Briefly describe the steps you took. -->
<!-- Include automated tests if possible, otherwise describe the manual testing routine. -->

<!-- Docs reminder: If this change requires updated docs, please do that! Engineers are the primary people responsible for their documentation. 🙌 -->

👉 _Stay up-to-date with [PostHog coding conventions](https://posthog.com/docs/contribute/coding-conventions) for a smoother review._

## Publish to changelog?

<!-- For features only -->

<!-- If publishing, you must provide changelog details in the #changelog Slack channel. You will receive a follow-up PR comment or notification. -->

<!-- If not, write "no" or "do not publish to changelog" to explicitly opt-out of posting to #changelog. Removing this entire section will not prevent posting. -->
no
@haacked haacked removed this from Feature Flags Jun 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting Prevents stale-bot from marking the PR as stale.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants