OWD project: Organise Web APIs

[Rumyra]

Ideally Web APIs should be categorised. This will benefit visibility for users, but also help drive how we organise the standalone demos MDN has. Currently the majority of 'github' demos within the MDN org are Web API ones and they could do with an update & better architecture. However this proposal is the first part to categorisation, which is to start by moving interface docs under their relevant Web API.

For example:

• Web Audio API (Parent folder)
  • AudioNode (Interface)
    • context
    • numberofinputs
    • ...
    • connect()
    • disconnect()

There's a couple of caveats - some things just don't fit in this structure (like fetch) and possibly can be left out. Also probably worth a discussion is tutorial docs, which are usually under the main API folder.

One possible solution is:

• Web Audio API (Parent folder)
  • Interfaces
  • Tutorials

It might be worth considering tutorials which cover more than one API - should these be a part of learn?

Effort

This should be just the first step to categorising Web APIs. However there would be a lot of moving of docs, so it's not a small effort

Dependancies

No changes to the platform should be needed

Community

One approach could be to raise an issue for each API detailing which docs need to be moved. This might encourage community help

Momentum

There's another proposal to create a searchable page for APIs. This is really the platform grounding to begin something like that

Enabling learners

Eventually a better way to discover APIs - however this is not covered by this first part

Enabling professionals

As above

Underrepresented topics / ethical web

N/A

Operational necessities

N/A

Addressing needs of the Web industry

Eventually be able to categorise Web APIs and mirror that with the external examples we have. Thus making it easier for users to discover both the docs & code examples.

[jpmedley]

You implied, but didn't say explicitly that this will change URLs. Does it?

What's the plan for keeping the URLs recorded in BCD in sync?

[Rumyra]

This was really a proposal which I want, but haven't thought too much about the details :)

In regards to bcd, this is data held in the frontmatter anyway, so there shouldn't need to be any change.

[Rumyra]

The MDN content team will be moving repos about in Q2 - originally I wanted to reorg the API docs to drive that work, but I've reversed my decision. Once we've tidied up the repositories containing code examples I'll come back to this.

Tl:dr; not a priority at this time and something the MDN content team might take on moving forward 👍

[wbamberg]

I saw this yesterday and realised I had just the same thought the other day, so I'm writing it up here, to add a bit of detail and justifications to your proposal.

---

Problems

There are three problems with the Web/API docs that this proposal tries to address.

(For background on Web/API sidebars I wrote this: https://discourse.mozilla.org/t/defaultapisidebar-apiref-and-groupdata/40210 a long while ago.)

Web/API sidebars are complicated

The sidebar system for Web/API involves lots of moving parts and things to remember, and things very often go wrong.

You have to:

• remember which sidebar macro to use (DefaultAPISidebar versus APIRef)
• pass the correct argument to the sidebar macro
• have exactly the right data in GroupData.json
• have pages in the right places under top-level pages
• have all the right tags (now obsoleted by page types)

In practice many people get some of this wrong, and the sidebar is then broken to some extent.

Web/API sidebars are too fragmentary

As said above, we have 2 main sidebar types (DefaultAPISidebar and APIRef). DefaultAPISidebar is for API overview pages and guide pages, while APIRef is for everything else, including interface pages. The system will produce:

• a different sidebar for every API (including the overview page and guide pages)
• a different sidebar for every interface (including the interface itself and its members)

Since we have about 100 APIs, and about 1000 interfaces, this means we have about 1100 different sidebars in Web/API.

For example, if I'm learning about Fetch, I might start at the main page (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API, sidebar 1), click on Request (https://developer.mozilla.org/en-US/docs/Web/API/Request, sidebar 2), then on the link to Headers, (https://developer.mozilla.org/en-US/docs/Web/API/Headers, sidebar 3). That's three sidebars in three closely related pages, in quite a small API. I think it would be better if there were a single sidebar that stayed the same as I navigated around Fetch.

There are too many directories at the top level

This is really for maintainers, but it matters still. Currently all API overview pages and all interface pages are at the top level, under web/api: https://github.com/mdn/content/tree/main/files/en-us/web/api . This makes it really hard to work with, and hard for maintainers to see the relations between interfaces.

Solution

What if we did this:

• under Web/API, have only "API Overview" (aka Specifications) pages, like https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
• under each of these, have 2 subdirectories, "Guides" and "Interfaces"
• under Guides, have the guides for this API
• under Interfaces, have a page for each interface, and under there, pages for properties, methods, events, as we have now.

Like:

Web/API/
    Fetch_API/
        index.md
        /Guides
            /Using_the_fetch_api
                index.md
           /etc
               index.md
        /Interfaces
            /Request
                /index.md
                /json
                    index.md
                /etc
                    index.md
            /etc
                index.md

...so we have like 100 pages or so directly under Web/API, one for each API. And under there (almost) all the pages for that API.

Then sidebars are just built like: if you are at a page under Web/API, find your top-level directory (e.g. Fetch_API) and construct a sidebar by iterating through the pages under it. Every page in an API gets the same sidebar, there's no more DefaultAPISidebar/APIRef distinction. I think this is a good thing!

Two problems I can see:

• guides want to appear in a curated order, not alphabetised. For this I think we should steal the concept of "weight" from Hugo and other SSGs. So you have a "weight" front matter item, and items with less weight appear higher in the list.
• items which belong to different interfaces, like https://developer.mozilla.org/en-US/docs/Web/API/Navigator/getBattery. For this, if we can't get that info out of webref, we could just list the things in the API page front matter. I think items like this should appear in both sidebars, but when you are actually on the page, the sidebar should be from the API that contains the interface that owns the item. So for instance, https://developer.mozilla.org/en-US/docs/Web/API/Battery_Status_API should list Navigator/getBattery(), but https://developer.mozilla.org/en-US/docs/Web/API/Navigator/getBattery should show the sidebar for the API that defines Navigator. Maybe. There are other options here too though, and maybe more problems?

This seems so simple I can't believe it would work. No GroupData, only one sidebar, no more getting the right argument to the sidebar, no more 1100 pages at the top level. So what's wrong with it?

One problem as @teoli2003 pointed out is that it basically destroys domxref, which depends on this very flat hierarchy. But I think we should stop using domxref
anyway.

[jpmedley]

I think this is a good plan.

I'm trying to think if there are any edge cases that wouldn't fit into this, but I'm coming up blank. If there are any, it would be good to deal with them now rather than waiting to stumble into them.

You should also have a plan for redirects from the existing structure. People have been linking to the existing structure for more than a decade. That's a lot of links to break in one go.

[Rumyra]

Thanks for the write up @wbamberg :) this was pretty much what I was thinking - I do think there could be a possibility something wouldn't fit with this organisation, but I can't think of it off the top of my head without looking.

It's a toughie with domxref - am i just used to it and when actually markdown links are (almost) as easy... I would forego the macro for this better organisation, especially when it improves other things such as sidebars.

It is a lot of moving... if we do plan to go ahead it would be good to let translated content know too I think

[jpmedley]

Is there really no way to make domxref work? This certainly improves things for readers. But if article locations are less predictable (and I think they are) we're going to be playing whack-a-mole with broken links.

[wbamberg]

Is there really no way to make domxref work?

It could be made to work in theory. AFAIK there's a global namespace for interfaces, so we know that {{domxref("Request")}} would only map to /API/Fetch_API/Interfaces/Request, and there can't be /API/Some_Other_API/Interfaces/Request.

But the implementation would be more complicated.

I'm not really a fan of our xref macros and would replace them all with Markdown links but I think I'm in a minority here.

I do think there could be a possibility something wouldn't fit with this organisation

One issue would be globals, which we recently moved out of interfaces and into the top level, like https://developer.mozilla.org/en-US/docs/Web/API/fetch. But I've started to think this might have been a mistake, because there isn't a single global scope, there are several: window scope, worker scope, service worker scope, etc. And we don't represent that at the moment.

[Elchi3]

Ideally Web APIs should be categorised. This will benefit visibility for users, but also help drive how we organise the standalone demos MDN has.

tl;dr I don't think categories/groups are as important to our audience. The ability to find objects, events, methods, properties, is important, however.

One of the strengths of MDN (vs the specifications, for example) is that MDN merges specifications into a single place. As a web developer, I love this. I'm not so much interested in how the Web API surface is grouped, I don't need to know API modules or groups necessarily. I like the docs, because don't need to browse 500+ specifications (or groups) and collect for myself from various places what the whole API surface has to offer me. It is an abstraction that is often not relevant to me. If I do DOM stuff, I interact with various objects, events, methods, properties that are presented to me. I don't actively choose to only interact with DOM events, or with the HTML DOM, I interact with all of this and I need to see how it plays together with more objects that I stumble upon. If I see a new object that is unknown to me, I don't need to know where it belongs to. I will browse the docs at the familiar URL and then that page fills me in on the broader context of this object and guides me to concepts and the API group this object belongs to. So, to me the categorization is for creating context for the reader, but the category is a helper for that, it doesn't come first necessarily.

tl;dr grouping is hard, group assignments change. quite a few groups is groupdata.json make no sense or overlap.

Coming up with groups that make sense for web developers isn't trivial and the groups might change over time. Some things move from one spec to another and if we tight APIs more strictly to groups, we will sure end up moving things around more as they make more sense with new specs the web platform adds.
There is also often overlap, or you have to decide how small you make a group. WebXR has like 20 specs, we would surely not create 20 groups there. You can't say one group per spec, I don't think. How do we group? is this an editorial choice?
And what about Document, Window, navigator? They are like infrastructure interfaces extended by many API groups. Would you want to fragment the docs of these interfaces by group? /API/Fetch_API/Window/fetch and /API/XHR_API/Window/xhr

If I take a few random specs, I can either group broadly or quite granular. It's not clear to me how I would do it:

Fetch
BackgroundFetch
Request
Response
Push
Worker
SharedWorker
ServiceWorker

Canvas
WebGL
WebGL2
WebGPU
WebXR

DOM core
DOM events
HTML DOM

I also don't like that the api/ namespace is polluted with basically everything. If I would re-do it, I would try to organize it by things. Like api/interfaces/*. Sure, that would be a huge folder still, but then it is clear what goes there (defined by WebIDL).
Conceptual material, like a guide that explains how 10 interfaces and a property on window grouped together make you get a job done around network stuff (fetch API, XHR), can live somewhere else like api/networking/bla, or even in the learning area or so.
And sidebars are helpers to let you find context, too. And alphabetical list of all interfaces isn't useful, of course. In sidebars you also want to present relevant related interfaces, events, etc.

The current sidebar implementation with hand-maintained groups does a decent job, but of course it is hard to maintain and has all the problems that Will outlines.

The sidebar system for Web/API involves lots of moving parts and things to remember, and things very often go wrong.

True and I'm afraid that we bring these problem to the page structures itself if we basically introduce similar grouping ideas there.

So, I think, I'd prefer organizing API pages by type (per WebIDL), have guide material separately, and have a sidebar implementation that can work with a guaranteed structure (by type) and additional input (e.g. from webref) to generate useful links to related interfaces, properties, etc.

[teoli2003]

I also don't like that the api/ namespace is polluted with basically everything. If I would re-do it, I would try to organize it by things. Like api/interfaces/*. Sure, that would be a huge folder still, but then it is clear what goes there (defined by WebIDL). Conceptual material, like a guide that explains how 10 interfaces and a property on window grouped together make you get a job done around network stuff (fetch API, XHR), can live somewhere else like api/networking/bla, or even in the learning area or so.

Note that we could move the overview pages (xxxy_API) from Web/API to Web/API/Overview to do 80% of such a cleaning (there would be only a few global properties/methods in it, and a few dictionaries.

[jpmedley]

Coming up with groups that make sense for web developers isn't trivial and the groups might change over time.

The venn diagram for that would look like a chain if you think about this from the standpoint building a specific app. Animation used WebGL, is used by WebXR, uses sensors, is used by navigation, uses canvases, is used by animation. Anyone want me to chart the branches?