Skip to content

build a SuperMemo-like Web Importer #264

Description

@hugomarins

What would RichTextNamespace.parseAndInsertHtml be useful for? Would it be able to parse an html site (e.g. https://en.wikipedia.org/wiki/Strait_of_Hormuz) and insert its content in a RemNote-friendly way? I ask this because I am searching for a SuperMemo-like web importer (https://help.supermemo.org/wiki/Web_import), that parses and import the content locally.

Yes, RichTextNamespace.parseAndInsertHtml is exactly the right tool for the final step of building a SuperMemo-style web importer, but you will need to do some intermediate work to get the clean result you want.

According to the SDK, parseAndInsertHtml takes an HTML string and a target RemId (or PluginRem), parses that HTML, and inserts it as a tree of Rem. Under the hood, this function uses RemNote's native paste-parsing logic. It understands semantic HTML tags (like <h1>, <ul>, <p>, <b>, <a>) and converts them into a hierarchical Rem structure with rich text formatting applied.

The Catch: You can't just pass raw website HTML

If you fetch the raw HTML of https://en.wikipedia.org/wiki/Strait_of_Hormuz and pass it directly to parseAndInsertHtml, RemNote will attempt to parse everything. You will end up with hundreds of garbage Rems generated from the site's navigation bars, side panels, footer links, hidden metadata, and script tags. RemNote's parser translates HTML to Rem, but it does not have an "Article Extractor" built-in.

How to build your SuperMemo Web Importer

To successfully replicate SuperMemo's Web Import feature, your plugin should follow a three-step pipeline:

  1. Fetch the HTML: Use fetch() or a proxy (to bypass CORS, which is often an issue with client-side fetching) to get the raw HTML of the target URL.
  2. Extract the Article (The Missing Link): Before handing the HTML to RemNote, you must strip away the boilerplate web noise. You can do this by running the HTML through a library like Mozilla's Readability.js (the same engine that powers Firefox's Reader View). This will give you a clean HTML string containing only the main article's title, headers, text, and images.
  3. Insert into RemNote: Pass that cleaned HTML string to plugin.richText.parseAndInsertHtml(cleanedHtml, targetRem.id).

Alternative Approach: HTML to Markdown

If you find that parseAndInsertHtml interprets certain HTML tags in ways you don't like, another popular approach is to convert the cleaned HTML into Markdown (using a library like Turndown.js) and then use plugin.rem.createTreeWithMarkdown(markdown, parentId). This gives you a bit more manual control over exactly how the hierarchy is constructed before RemNote processes it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions