Skip to content

docs: generate the SDK reference as Markdown with go/doc - #410

Merged
gewenyu99 merged 16 commits into
mainfrom
markdown-reference-adapter
Nov 11, 2026
Merged

gewenyu99 merged 16 commits into
mainfrom
markdown-reference-adapter

Conversation

@gewenyu99

@gewenyu99 gewenyu99 commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

We have a wave of breaking changes with new major versions for most SDKs. References are now read mostly by bots, not humans, and the HogRef format is heavy. Strapi then syncs references/ daily, and posthog.com builds the pages (PostHog/squeak-strapi#260, PostHog/posthog.com#21971).

  • internal/cmd/reference uses only the standard library: go/doc reads each exported symbol and go/doc/comment renders its doc comment as Markdown. Our code groups symbols by capability, from docs/capabilities.json in sdk-specs order and writes front matter, with no regex and no third-party tool.
  • docs/capabilities.json maps each symbol to its sdk-specs capability, so the doc comments stay as they are. Generation fails on a stale entry, an unknown capability, or an unmapped client method.
  • Adds docs/getting-started.md and a release step that checks out sdk-specs and regenerates references/.
  • The release's reference step is best-effort, so a failure never blocks a release. .github/workflows/generate-references.yml reruns it by hand from main and commits references/ when anything changed. It follows the prior art in posthog-js's generate-references.yml, and it's the same in every SDK repo apart from the generation step.

💚 How did you test it?

  • go run ./internal/cmd/reference --sdk-specs <checkout> reproduces the pages in the next PR. The release step hasn't run in CI.

📝 Checklist

  • All commits are signed
  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Claude Code: Sonnet 5.5 built it and Opus 5.5 opened it, directed by Vincent. Not yet reviewed line by line.

Merge order

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@gewenyu99 gewenyu99 self-assigned this Nov 5, 2026

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 4b84cfd: 1 Must fix.

This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Comment thread .github/workflows/release.yml Resolved
@github-actions

github-actions Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

posthog-go Compliance Report

Date: 2026-11-11 17:42:18 UTC
Duration: 169186ms

✅ All Tests Passed!

111/111 tests passed


Capture_V1 Tests

✅ 94/94 tests passed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint ✅ 8ms
Endpoint And Method.Does Not Use Legacy Endpoints ✅ 7ms
Required Headers.Has Authorization Bearer Header ✅ 6ms
Required Headers.Has Content Type Json ✅ 6ms
Required Headers.Has Posthog Sdk Info Format ✅ 7ms
Required Headers.Has Posthog Attempt Header ✅ 6ms
Required Headers.Has Posthog Request Id ✅ 6ms
Required Headers.Has Posthog Request Timestamp ✅ 7ms
Required Headers.Has User Agent ✅ 6ms
Body Format.Body Has Created At And Batch ✅ 6ms
Body Format.No Api Key In Body ✅ 7ms
Body Format.No Sent At In Body ✅ 5ms
Event Format.Event Has Required Root Fields ✅ 6ms
Event Format.Event Uuid Is Valid ✅ 7ms
Event Format.Event Timestamp Is Rfc3339 ✅ 6ms
Event Format.Distinct Id Is String ✅ 6ms
Event Format.Distinct Id At Root Not Properties ✅ 7ms
Event Format.Custom Properties Preserved ✅ 16ms
Event Format.Set Properties Preserved ✅ 6ms
Event Format.Set Once Properties Preserved ✅ 7ms
Event Format.Groups Properties Preserved ✅ 6ms
Event Format.Sdk Generates Uuid If Not Provided ✅ 6ms
Event Format.Event Has Required Root Fields Batch ✅ 9ms
Event Format.Event Uuid Is Valid Batch ✅ 9ms
Event Format.Event Timestamp Is Rfc3339 Batch ✅ 9ms
Event Format.Distinct Id Is String Batch ✅ 8ms
Event Format.Distinct Id At Root Not Properties Batch ✅ 9ms
Event Format.Custom Properties Preserved Batch ✅ 9ms
Event Format.Set Properties Preserved Batch ✅ 8ms
Event Format.Set Once Properties Preserved Batch ✅ 10ms
Event Format.Groups Properties Preserved Batch ✅ 8ms
Event Format.Sdk Generates Uuid If Not Provided Batch ✅ 9ms
Batch Behavior.Multiple Events In Single Batch ✅ 31ms
Batch Behavior.Batch Envelope Smoke ✅ 11ms
Batch Behavior.Flush With No Events Sends Nothing ✅ 4ms
Batch Behavior.Flush At Triggers Batch ✅ 1007ms
Batch Behavior.Created At Reflects Batch Creation Time ✅ 8ms
Deduplication.Generates Unique Uuids ✅ 14ms
Deduplication.Different Events Same Content Different Uuids ✅ 9ms
Deduplication.Preserves Uuid On Retry ✅ 5132ms
Deduplication.Preserves Timestamp On Retry ✅ 5137ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 5136ms
Deduplication.No Duplicate Events In Batch ✅ 33ms
Header Behavior On Retry.Attempt Header Starts At One ✅ 6ms
Header Behavior On Retry.Attempt Header Increments On Retry ✅ 10139ms
Header Behavior On Retry.Request Id Preserved On Retry ✅ 5136ms
Header Behavior On Retry.Different Requests Have Different Request Ids ✅ 2014ms
Header Behavior On Retry.Request Timestamp Changes On Retry ✅ 5135ms
Response Format Validation.Success Response Has Uuid Keyed Results ✅ 8ms
Response Format Validation.Success Response Has Ok For Each Event ✅ 29ms
Response Format Validation.Success No Retry After When All Ok ✅ 26ms
Response Format Validation.Success Retry After Present When Retry Events ✅ 130ms
Response Format Validation.Success No Retry After When Drop Only ✅ 28ms
Response Format Validation.Response Echoes Request Id ✅ 7ms
Retry Behavior.Retries On 408 ✅ 5134ms
Retry Behavior.Retries On 500 ✅ 5136ms
Retry Behavior.Retries On 503 ✅ 5135ms
Retry Behavior.Retries On 504 ✅ 5134ms
Retry Behavior.Retryable Errors Have Retry After ✅ 2133ms
Retry Behavior.Respects Retry After On Retryable Error ✅ 8131ms
Retry Behavior.Does Not Retry On 400 ✅ 2132ms
Retry Behavior.Does Not Retry On 401 ✅ 2133ms
Retry Behavior.Does Not Retry On 402 ✅ 2132ms
Retry Behavior.Does Not Retry On 413 ✅ 2132ms
Retry Behavior.Does Not Retry On 415 ✅ 2134ms
Retry Behavior.Non Retryable Errors Have No Retry After ✅ 2132ms
Retry Behavior.Implements Backoff ✅ 15133ms
Retry Behavior.Max Retries Respected ✅ 15145ms
Partial Batch Handling.Handles 200 Full Success ✅ 2009ms
Partial Batch Handling.Handles 200 With All Ok ✅ 3031ms
Partial Batch Handling.Does Not Retry Dropped Events ✅ 3032ms
Partial Batch Handling.Does Not Retry Limited Events ✅ 3031ms
Partial Batch Handling.Prunes Ok Events On Partial Retry ✅ 5136ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry ✅ 5133ms
Partial Batch Handling.Retries Only Retry Events From Partial ✅ 5136ms
Partial Batch Handling.Partial Retry Preserves Uuids ✅ 5135ms
Partial Batch Handling.Partial Retry Attempt Header Increments ✅ 5137ms
Partial Batch Handling.Partial Retry Request Id Preserved ✅ 5135ms
Partial Batch Handling.Respects Retry After On Partial ✅ 5133ms
Partial Batch Handling.Unknown Result Treated As Terminal ✅ 3031ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry ✅ 3033ms
Compression.Sends Gzip Content Encoding ✅ 17ms
Compression.No Content Encoding When Disabled ✅ 6ms
Compression.Compressed Body Is Decompressible ✅ 7ms
Error Handling.Does Not Retry On Unknown 4Xx ✅ 2131ms
Event Options.Cookieless Mode Override ✅ 7ms
Event Options.Disable Skew Correction Override ✅ 6ms
Event Options.Process Person Profile Override ✅ 6ms
Event Options.Product Tour Id Override ✅ 6ms
Event Options.Unset Options Omitted ✅ 6ms
Event Options.Options Override In Batch ✅ 8ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties ✅ 7ms
Geoip And Historical Migration.Historical Migration Set In Body ✅ 5ms
Geoip And Historical Migration.Historical Migration Absent By Default ✅ 7ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 15ms
Request Payload.Flags Request Uses V2 Query Param ✅ 16ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 16ms
Request Payload.Flags Request Omits Authorization Header ✅ 15ms
Request Payload.Token In Flags Body Matches Init ✅ 16ms
Request Payload.Groups Round Trip ✅ 16ms
Request Payload.Groups Default To Empty Object ✅ 17ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 15ms
Request Payload.Disable Geoip Omitted Defaults To False ✅ 16ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 16ms
Request Lifecycle.No Flags Request On Init Alone ✅ 3ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 5ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 29ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 15ms
Retry Behavior.Retries Flags On 502 ✅ 18ms
Retry Behavior.Retries Flags On 504 ✅ 17ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 17ms

gewenyu99 and others added 2 commits November 5, 2026 12:07
go/doc reads each exported symbol and go/doc/comment renders its doc comment
as Markdown, so the generator needs no third-party tool and no regex. It only
groups symbols by their posthog-capability tag in sdk-specs order and writes
front matter. The release step checks out sdk-specs for the headings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The snapshot includes doc comments, and the tags add a line to some of them.
No signatures change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gewenyu99 and others added 3 commits November 5, 2026 14:25
ghcommit-action passes an untracked folder to ghcommit as one path, which fails
on the first release that writes references/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The posthog-capability lines showed up in IDE hover help for every SDK
user. The generator now reads a label-to-slug map next to it, and fails
on stale labels, unknown slugs, and unmapped SDK utils members.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gewenyu99 gewenyu99 changed the title docs: generate the SDK reference as Markdown with gomarkdoc docs: generate the SDK reference as Markdown with go/doc Nov 5, 2026
gewenyu99 and others added 4 commits November 5, 2026 15:41
The release's reference step is best-effort, so a failure never blocks a
release. generate-references.yml reruns it from main on demand and commits
references/ when anything changed. Same shape as posthog-js's
generate-references.yml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The reference step is best-effort, so a failure leaves the release green. A
::warning:: annotation now says so and points to generate-references.yml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gewenyu99
gewenyu99 marked this pull request as ready for review November 10, 2026 19:36
@gewenyu99
gewenyu99 requested a review from a team as a code owner November 10, 2026 19:36
@veria-ai

veria-ai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

PR overview

No security issues were found in this pull request.

Reviews

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed b01313e: 1 Must fix, 2 Should fix.

Comment thread internal/cmd/reference/main.go Outdated
p := pkg.Printer()
p.HeadingLevel = 5
p.DocLinkURL = func(l *comment.DocLink) string { return "#" + anchor(l.Recv+l.Name) }
return strings.TrimSpace(string(p.Markdown(d)))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 · Render doc-comment examples as fenced code blocks

p.Markdown renders doc-comment examples as indented code blocks, which MDX does not support. The generated NewSlogCaptureHandler example contains &slog.HandlerOptions{Level: slog.LevelInfo}, so MDX interprets the braces as an expression and fails compilation. This prevents the generated reference from building on posthog.com; render examples with fenced code blocks.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

}

func write(file, front string, body []string) {
check(os.WriteFile(filepath.Join("references", file), []byte(strings.Join(append([]string{front}, body...), "\n\n")+"\n"), 0o644))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Create the output directory before writing references

On a fresh checkout, references/ does not exist and neither workflow creates it. The first write call therefore exits with open references/posthog-go-1.md: no such file or directory, producing no reference pages and failing the manual generation workflow. Create the output directory before calling os.WriteFile so generation works from a clean checkout.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment thread internal/cmd/reference/main.go Outdated
Comment on lines +214 to +216
func typeEntry(t *doc.Type) entry {
children := append(funcs(t.Funcs, "posthog."), funcs(t.Methods, t.Name+".")...)
return entry{t.Name, block(t.Decl), markdown(t.Doc), children}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Include constants and variables associated with types

go/doc attaches typed constants to doc.Type.Consts, but typeEntry only renders the declaration, functions, and methods. Consequently, the reference omits declarations for CompressionGzip, CaptureModeAnalyticsV1, and every UnresolvedFlagReason constant, leaving their available values undocumented; render each type's associated constants and variables too.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment thread internal/cmd/reference/main.go Outdated
p := pkg.Printer()
p.HeadingLevel = 5
p.DocLinkURL = func(l *comment.DocLink) string { return "#" + anchor(l.Recv+l.Name) }
return strings.TrimSpace(string(p.Markdown(d)))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] The PostHog Review P1 here holds up: escape() only touches Paragraph and List blocks, and p.Markdown prints comment.Code as a tab-indented block. MDX doesn't support indented code, so those lines get parsed as prose, and braces like &slog.HandlerOptions{Level: slog.LevelInfo} become JSX expressions. references/posthog-go-1.md on #411 has 26 such lines (e.g. the NewSlogCaptureHandler example), so the page would likely fail to compile on posthog.com. Could we print Code blocks as fenced ```go blocks ourselves (they don't need escaping inside a fence), then regenerate #411?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@turnipdabeets
turnipdabeets requested a review from a team November 10, 2026 21:08
@gewenyu99
gewenyu99 added this pull request to stack #442 November 11, 2026 16:55
gewenyu99 and others added 3 commits November 11, 2026 12:03
…erences/

- Doc-comment examples printed as indented code, which MDX doesn't
  support, so braces in them parsed as JSX. Print them as fenced Go blocks.
- go/doc groups typed constants and variables under their type, and the
  type entry left them out, so values like CompressionGzip and every
  UnresolvedFlagReason weren't documented. Render them under the type.
- Create references/ before writing, so generation works from a clean
  checkout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Print each declaration with only the comments of the fields and
  methods go/doc kept, so unexported fields' comments don't show, and
  drop the blank lines a filtered member leaves.
- Put package constants and variables under Constants and Variables
  headings in SDK utils.
- Keep the EnqueueClient type entry, which public signatures take.
- Align columns with spaces, the way gofmt does.
- Tag a code example go only when it looks like Go, and text otherwise.
- Point [Name] doc links at the symbol's heading.
- Drop empty parts, the brace escapes the site no longer needs, and the
  printer's escaped backticks.
- Read the module path from go.mod, so the next major's page installs and
  imports .../posthog-go/v2.
- Fix the Properties example, which put the dots at the start of lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed bed9314: 1 Must fix.

This is review round 3 on this pull request. When findings keep coming back in the same code, the approach is usually the problem. Before the next fix, check whether a different design would remove the whole class of findings.

return "#" + anchor("posthog."+l.Name)
}

var mdx = strings.NewReplacer("<", "&lt;")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 · Escape braces in generated prose for MDX

Generating either reference page includes the SlogCaptureHandler prose client.Enqueue(Exception{...}). escape leaves these braces intact, so MDX treats {...} as JavaScript and compilation fails with Could not parse expression with acorn: Unexpected token, blocking the website build even with examples fenced. Escape prose braces or render these snippets as inline code.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't render these pages as MDX. posthog.com blocks them from MDX here: https://github.com/PostHog/posthog.com/blob/b07e215fdada73f80aec4a42ecc8040a3c5d7be2/gatsby-config.js#L143-L145 and renders them as plain Markdown here: https://github.com/PostHog/posthog.com/blob/b07e215fdada73f80aec4a42ecc8040a3c5d7be2/src/templates/sdk/SdkMarkdownReference.tsx#L74. I tested this on a local build, and the live Python reference renders the same raw braces. Thanks!

…s inside parentheses

A ... placeholder is read as an argument, so examples such as
NewWithConfig(...) still parse as Go, and prose with a parenthesis no
longer passes as code. A const block no longer opens with a blank line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread docs/capabilities.json Resolved
gewenyu99 and others added 2 commits November 11, 2026 12:38
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main added Client.Emails, and the generator stops on a member with no
capability, so the release step would skip the pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@eli-r-ph

Copy link
Copy Markdown
Contributor

Small cleanups before merge, none blocking: the review-bot threads on fenced code, typed constants and creating references/ look fixed and can be resolved. The P1 on escaping braces needs a reply, since posthog.com now renders these pages as plain Markdown. The merge-order list still says "with gomarkdoc". TestConcurrentClientOperations on Go 1.21 looks flaky and needs a rerun.

@gewenyu99
gewenyu99 merged commit cf76ebf into main Nov 11, 2026
54 of 55 checks passed
@gewenyu99
gewenyu99 deleted the markdown-reference-adapter branch November 11, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants