Skip to content

SEO/AEO: one structured-data graph, llms.txt, AI crawlers named - #44

Merged
Abuubkar merged 1 commit into
devfrom
seo/aeo
Sep 13, 2026
Merged

Abuubkar merged 1 commit into
devfrom
seo/aeo

Conversation

@Abuubkar

Copy link
Copy Markdown
Owner

The search half of SEO was done in #38. This is the answer-engine half — what
ChatGPT, Perplexity, Gemini and AI Overviews read, which is not the same set of
things Googlebot reads.

Structured data is now one graph

It was two islands: Organization from the root on every page, FAQPage from
the home route, neither carrying an @id, so no node referred to any other.

Now the root emits Organization + WebSite with stable @ids, and each route
adds its own nodes pointing at them:

Route Adds
/ WebPage, Service (with the Services section as a nested OfferCatalog), FAQPage
/privacy, /terms, /hipaa WebPage with dateModified, BreadcrumbList

Consumers merge a graph by @id across separate <script> tags, so the
publisher is described once and everything else refers to it.

New fields, all derived from messages.json — nothing invented:

  • Organization.knowsAbout — the specialty list. This is the field an engine
    asked "who bills for dermatology practices" has to match on.
  • Organization.contactPoint, email
  • Service.hasOfferCatalog — the four Services cards, nested by stage of the
    revenue cycle, each line item as an Offer
  • Service.audience
  • WebPage.dateModified on the legal pages, parsed from the one date already in
    legal.json rather than stored a second time

Two keys added to messages.json: meta.serviceType and meta.audience.

public/llms.txt

The landing page restated as plain text an answer engine can quote: the
positioning bullets, services with every line item, the four steps, specialties,
supported EHRs, the full Why Us comparison table, what gets reported monthly,
the fit/not-fit lists, all eight FAQ answers, and the four pages.

Generated by scripts/llms-txt.mjs, run from pnpm build. Generated rather
than written because a second hand-kept copy of these facts is exactly the drift
that put a wrong logo in the WhatsApp preview. The output is committed, so a
copy change shows up in the diff.

robots.txt names the AI crawlers

This is the finding worth reading. https://execil.net/robots.txt does not
serve the file in public/ — Cloudflare prepends a managed block that
disallows GPTBot, ClaudeBot, Google-Extended, Applebot-Extended,
meta-externalagent, Amazonbot, CCBot and Bytespider
, and sets ai-train=no.
Default-on, not a decision anyone made here.

Everything else in this PR is decoration while that stands. Filed as #43 with
the dashboard steps — it needs the Cloudflare account, so no agent can do it.

public/robots.txt now names each of those crawlers in an explicit Allow
group. Repeating a user-agent merges the groups and an equal-specificity Allow
wins the tie-break, so for crawlers that follow it the block is already
reversed. A fallback, not the fix.

FAQ anchors

Each answer gets id="faq-<slug>" and a matching @id on its Question node,
so one answer can be linked and cited on its own.

Verified

  • pnpm check clean, pnpm build prerenders all four pages
  • Home emits 5 nodes across 2 blocks, legal pages 4 — @ids resolve, no
    duplicated Organization
  • dateModified: "2026-09-08" on /privacy, matching the page's own date
  • Live crawler access re-checked: GPTBot, ClaudeBot, PerplexityBot,
    OAI-SearchBot, Googlebot and bingbot all get 200 and the full page — the
    block is in robots.txt only, nothing at the edge
  • https://www.execil.net/ still answers 200, not 301 (SEO: the three items no agent can do #39 item 2, unchanged)

…ts.txt

The code half of pre-launch SEO (#38) covered the search crawlers. This covers
the answer engines, which read different things.

Structured data was two disconnected islands: Organization from the root, a
FAQPage from the home route, neither carrying an @id, so nothing referred to
anything. Now every page emits Organization and WebSite with stable @ids, and
each route adds its own nodes referencing them — the home page a WebPage, the
Service with the Services section as a nested OfferCatalog, and the FAQ; the
legal pages a WebPage with dateModified and a BreadcrumbList. Consumers merge a
graph by @id across script tags, so the publisher is described once.

Every value is derived from messages.json, including Organization.knowsAbout,
which is the specialty list — the thing an engine asked "who bills for
dermatology practices" has to match on.

public/llms.txt restates the page as plain text an answer engine can quote:
services, specialties, the comparison table, the FAQ. Generated by
scripts/llms-txt.mjs and run from `build`, because a second hand-kept copy of
these facts is the same drift the brand assets already taught us about. The
output is committed so a copy change shows up in the diff.

robots.txt names the AI crawlers. Cloudflare's managed robots.txt is prepended
at the edge and disallows GPTBot, ClaudeBot, Google-Extended and five others by
default; the real fix is the dashboard toggle, but repeating a user-agent merges
the groups and an equal-specificity Allow wins the tie-break.

FAQ answers get an id each, so one answer can be cited on its own.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
execil 8239be7 Commit Preview URL

Branch Preview URL
Sep 13 2026, 05:05 AM

@Abuubkar
Abuubkar merged commit 7a8903c into dev Sep 13, 2026
2 checks passed
@Abuubkar
Abuubkar deleted the seo/aeo branch September 13, 2026 05:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant