Skip to content

Frontend chat agent powered by site content #142

Description

@jeffpaul

What problem does this address?

Add an optional experiment that exposes a frontend chat UI capable of answering visitor questions using the site's own public content (and any extra references the site owner provides). Think “chat with my site” using retrieval-augmented responses, integrated with the existing WP AI Client stack.

This experiment will focus outside traditional editor/admin workflows. Looking initially for a simple, extensible example of how WordPress content can power an AI assistant on the public-facing frontend. A frontend chat agent would help visitors quickly find information, showcase how the Abilities API and WP AI Client can serve public UX, and give developers a reference pattern they can build on.

What is your proposed solution?

  • Floating chat bubble (likely via block for site editor integration) that triggers a small chat panel
  • Site owners pick what content gets indexed (post types, filters, pages only, etc.)
  • Optional custom reference text/FAQ entries
  • Answers generated through WP AI Client using configured provider/model
  • Responses cite or link back to relevant content

Potential items needed to build this out:

  1. Content indexing + embeddings
  • Small indexing pipeline: chunk posts/pages, embed passages, store in DB
  • Incremental updates on post changes
  • Basic “reindex” control in settings
  1. Retrieval ability
  • New Ability that accepts a user query and returns top-k relevant chunks with metadata (title, excerpt, permalink)
  • Reusable by other experiments in the future
  1. Chat orchestration
  • Endpoint for sending messages and returning AI output
  • Prompt assembly that combines visitor question + retrieved passages
  • Temporary conversation ID stored client side
  • Server side controller that routes each turn through WP AI Client
  1. Frontend UI
  • Minimal, accessible block (and/or JS component)
  • Sends messages to endpoint
  • Basic error handling and “reset conversation”

Recommended out of scope for v1

  • Admin-only assistants
  • Multi-channel messaging (WhatsApp, Telegram, etc.)
  • Advanced bot workflows or custom logic builders

Decisions to work through

  • Index only public content in v1 to avoid permissions complexity
  • Use a simple local embeddings table first; make storage pluggable later
  • Ship one UI pattern (floating bubble); blocks or template placement can follow later
  • Analytics can be minimal (count + common questions) or deferred entirely

Questions to consider

  • Should embeddings be stored as JSON or a dedicated vector column?
  • Any requirements for streaming responses now, or is single-shot OK for v1?
  • Should we expose a “developer mode” to override the default prompt?

Activity

  1. added this to the Future Release milestone on Dec 8, 2025
  2. the-hercules commented on Jul 13, 2026

    @the-hercules
    Contributor

    @jeffpaul

    Thoughts on the open items

    Storage (JSON vs. Vector column)

    Given how much MySQL environments vary across WordPress hosts, a dedicated VECTOR column (available only in newer MySQL versions) doesn't seem like a practical choice if we want broad compatibility. A custom table with a JSON or LONGTEXT column for storing embeddings feels like the safest option for v1. We can always optimize the storage layer later if native vector support becomes more common.

    Streaming vs. Single-shot

    I think single-shot responses are the right choice for v1. True streaming with Server-Sent Events over standard WordPress REST endpoints can get messy because of server buffering, PHP execution limits, and hosting differences. Starting with a simpler request/response flow lets us focus on the overall UX first, and we can introduce streaming later if it's needed.

    Developer mode (Prompt overrides)

    I don't think we need a dedicated UI for this. Abstract_Ability already exposes the wpai_system_instruction filter, which feels like the idiomatic WordPress way to let developers customize prompts. Anyone who wants full control over the assistant's behavior can hook into that filter.

    Analytics

    We may already get most of what we need through the existing AI_Request_Logging experiment. As long as the new frontend ability sends requests through the standard AiClient, we could attach metadata such as source: frontend_chat and use that for filtering. That would let us defer any custom analytics work until later.

    New areas we should align on

    Since this introduces a new type of workflow for the plugin, there are a few architectural questions worth answering before implementation.

    • Experiment categorization: Today, Experiment_Category only defines EDITOR and ADMIN. Since this is visitor-facing, should we introduce a new category such as FRONTEND or PUBLIC? If so, how should it appear in the settings UI so site owners understand that enabling it affects public traffic?

    • Embeddings fallback: Not every configured connector will support the EMBEDDING_GENERATION capability. Should the experiment refuse to activate unless an embeddings-capable connector is available, or should a non-embedding fallback, such as basic WordPress search or BM25 retrieval, be part of v1?

    • Data handling and abuse: This will be the first experiment where anonymous visitors can trigger AI API calls. The existing Connector_Approval experiment is designed around admin-approved plugin and theme usage, but a public chat interface exposes API usage to unauthenticated traffic. Should v1 include built-in rate limiting or other abuse protection to prevent malicious users from exhausting a site owner's API quota?

  3. priyanshuhaldar007 commented on Jul 14, 2026

    @priyanshuhaldar007
    Contributor

    Great breakdown, @the-hercules. I agree with the overall direction and most of the proposed decisions. I have a few additional thoughts, particularly around the architectural considerations.

    • Storage (JSON vs. VECTOR): I agree that JSON/LONGTEXT is the right choice for v1. Without native vector support, similarity searches will need to happen in PHP after retrieving candidate rows, which is reasonable for small to medium sites but will become a bottleneck as the number of indexed chunks grows. To avoid future refactoring, it would be beneficial to make the retrieval storage layer pluggable from the start so sites using pgvector, MySQL VECTOR, SQLite-vss, Qdrant, or similar solutions can swap storage backends without affecting the rest of the chat system.
    • Streaming vs. single response: I agree that a single response is the right approach for the first version. Even without server-sent events, a typing indicator and optimistic UI can significantly improve perceived responsiveness. It would also be useful to keep the frontend messaging layer abstract enough so streaming can be introduced later without requiring a major rewrite.
    • Experiment categorisation: I strongly support introducing a dedicated FRONTEND/PUBLIC experiment category. Visitor-facing AI features have a very different risk profile from editor or admin experiments because they are exposed to anonymous users and can trigger paid API requests. The settings experience should make this distinction clear, ideally with a confirmation step explaining that enabling the feature exposes AI functionality to visitors and may consume the site's AI quota.
    • Rate limiting and abuse prevention: I see this as a requirement for v1 rather than something to defer. Any unauthenticated endpoint capable of triggering paid AI requests can quickly become a target for abuse. A reasonable baseline would include per-IP and global rate limits with sensible defaults, configurable daily or monthly request limits, nonce or origin validation to reduce off-site abuse, and optional lightweight bot protection such as a honeypot. Starting with conservative defaults is likely the safer approach and helps avoid unexpected costs for site owners

    I also think there are a few areas that have not been discussed yet and are worth considering before implementation.

    • Content freshness: The discussion covers updating embeddings when posts change, but it is equally important to handle deleted, trashed, or unpublished content. Indexed chunks should be removed as soon as content is no longer publicly available; otherwise, the chatbot may continue referencing information that has been intentionally removed. Treating content removal as part of the indexing workflow would help maintain accuracy and avoid potential issues.
    • Response grounding and citations: While retrieval helps reduce hallucinations, it does not eliminate them. A model can still generate unsupported claims while attaching citations that appear legitimate. It may be worth considering whether responses should be limited to retrieved content and return a clear "I couldn't find that information on this site" message instead of relying on general model knowledge.
  4. jeffpaul commented on Aug 5, 2026

    @jeffpaul
    MemberAuthor

    Storage (JSON vs. Vector column)

    I think I'd prefer the vector approach (e.g. #683) which is blocked by the PHP AI Client adding embedding support (and ideally that getting into the WP AI client and a WP core release).

    However, I'll defer on that and other architectural topics for @dkotter's review which is delayed as he's away on leave for a bit.

  5. ColinM-sys commented on Nov 9, 2026

    @ColinM-sys
    Contributor

    Thanks, I'll take frontend chat. I'll start the chat PR after #1252 merges.

    For rate limiting, I'd suggest:

    • A per-IP cap, for example 10 messages a minute and 50 an hour, to stop one visitor from hammering the chat.
    • A daily cap on total chat calls for the whole site, to protect the bill. Per-IP limits don't stop many addresses working together, so this one matters most.
    • No per-session limit, since it's easy to get around by clearing cookies.

    I'll implement the rate limiter after this PR merges, and you decide the approach.

  6. moved this from Backlog to To do in WordPress AI Roadmapon Nov 11, 2026
  7. modified the milestones: Future Release, 1.7.0 on Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions