Merge pull request #56 from ptr727/develop

Promote develop to main: the gallery conversion damage, and operations as recurring work

Author: ptr727

TODO.md

@@ -38,6 +38,16 @@ body {
     margin: 1rem auto;
 }
 
+/*
+    The set caption is a flex item like the figures beside it, so without a full-width basis it
+    packs onto the end of the last row and reads as a caption for whichever image it lands next
+    to. Full width puts it on its own row under the set, which is what it describes.
+*/
+.gallery > figcaption {
+    flex: 0 0 100%;
+    text-align: center;
+}
+
 .gallery-cols-1 figure {
     width: 100%;
 }

assets/css/extended/custom.css

@@ -84,6 +84,24 @@ A count is all the check can observe, and two causes reach each direction: it ri
 
 **Both directions read absolute references as well as relative ones.** Hugo writes an absolute URL wherever a template resolves one against the base, which the entry-cover image on every list page does. Reading only rooted paths made those files look linked from nowhere while they were being displayed, and left a broken one unchecked in the other direction. The origin is read from the home page's canonical link rather than assumed, since staging and production build with different base URLs and a hardcoded host would check one environment's output against another's. No canonical link is a hard failure, because a guessed origin inflates the orphan count by exactly the pages that use one.
 
+## The gallery check, which no direction above can reach
+
+Every check above reasons about a URL: whether it renders, whether it resolves, whether anything points at it. Content misplaced **inside** a gallery satisfies all of that. The file exists, the reference resolves, and something links it, so the media surface is green in both directions while the page is laid out wrong. The defect is one of structure, which is why three variants of it survived the conversion and every gate since.
+
+A gallery is a flex row whose column widths come from `.gallery-cols-N figure`. Anything in there that is not a `figure` gets no width from that rule and is rendered as one more item in the row. So the check reads the built pages and fails on any direct child of a gallery container that is not a `figure` or the gallery's own `figcaption`.
+
+The three shapes it found, all of them conversion artifacts, and each verified against the captured live site before being changed:
+
+| Shape in the markdown | What the old platform had |
+| --- | --- |
+| Caption text appended after the last `figure` shortcode's `}}` | `<figcaption class="blocks-gallery-caption">`, a caption for the **set** |
+| A bare `![](…)` image | `<li class="blocks-gallery-item"><figure>` |
+| A linked `[![](…)](…)` image | the same, with an `<a>` **inside** the figure |
+
+**The first shape is why the capture is consulted rather than the markup.** A reviewer reading only the source reasonably suggests moving the text into the last figure's `caption` parameter, which is what a per-image caption would need. The capture shows all eleven were gallery-level captions, so that fix would have attributed a caption for a set of four images to whichever one happened to be last, and it would have looked correct.
+
+The gallery shortcode therefore takes a `caption` of its own and renders the container as a `figure`, since `figcaption` is only valid as a figure's child. The theme already styles `figure > figcaption`, so a set caption needs no rule beyond a full-width flex basis to keep it off the end of the last row.
+
 ## What the orphans are
 
 The count is not a backlog. It opened at 120 and was adjudicated against the captured live site under `blog-capture/mirror/`, which holds a crawl of the old platform including all 328 URLs the contract requires:

checks/README.md

@@ -10,6 +10,7 @@
 import pathlib
 import re
 import sys
+from html.parser import HTMLParser
 from urllib.parse import unquote
 
 # A truncated list would make every assertion below it pass vacuously while the gate stays green.
@@ -37,6 +38,12 @@
 # slack can never accumulate for a later regression to hide in.
 ORPHANED_MEDIA = 98
 
+# Every check above returns a list, and the shared summary called all of them "missing". That is
+# what a URL that did not build is, and it is not what a stray node inside a gallery is: those are
+# present, which is the whole complaint. The default stays "missing" so a check added later reads
+# the way the older ones do unless it says otherwise.
+FAILURE_NOUN = {"gallery": "stray nodes"}
+
 
 def load(name):
     lines = [ln.strip() for ln in (CHECKS / name).read_text().splitlines()]
@@ -189,6 +196,102 @@ def check_orphans(public, refs):
     return orphaned
 
 
+class GalleryScan(HTMLParser):
+    """Collect anything inside a gallery container that is not one of its permitted children.
+
+    A gallery is a flex row of figures, so its column widths are set by `.gallery-cols-N figure`.
+    Anything else landing in there is not laid out by that rule and is rendered as one more item
+    in the row.
+
+    This reads the built HTML, so it sees fewer shapes than the markdown has, and deliberately:
+    three source patterns reached it, a set caption written as text trailing a figure shortcode,
+    a bare markdown image, and a linked one, and they arrive here as a bare text node, a `<p>`
+    wrapping images, and an `<a>` with a `<br>` beside it. Enumerating source patterns would
+    make this a list to extend every time the conversion surprises us again. Naming the one
+    invariant instead, that a gallery holds figures and its own caption, covers the shape nobody
+    has thought of yet, which is how the third of the three was found after the first two.
+    """
+
+    # A void element never closes, so counting it as an open tag desynchronizes the depth for the
+    # rest of the document and every later gallery reads as containing whatever follows it.
+    VOID = {"area", "base", "br", "col", "embed", "hr", "img", "input",
+            "link", "meta", "param", "source", "track", "wbr"}
+    ALLOWED = {"figure", "figcaption"}
+
+    # There is deliberately no handle_startendtag override. HTMLParser's own implementation
+    # forwards a self-closing tag to handle_starttag and then handle_endtag, so `<br/>`, `<br />`
+    # and `<img/>` are already reported and already leave the depth balanced. Adding an override
+    # to "support" them is what would break it, by counting a pair the base class already splits.
+    # Verified on those three spellings and on a self-closing non-void `<figure/>`.
+
+    def __init__(self):
+        super().__init__(convert_charrefs=True)
+        self.findings = []
+        self.saw_gallery = False
+        # None outside a gallery; otherwise the number of elements open within the current one,
+        # so zero means the parser is looking at a direct child.
+        self.depth = None
+
+    def handle_starttag(self, tag, attrs):
+        classes = dict(attrs).get("class", "").split()
+        if self.depth is None:
+            if tag == "figure" and "gallery" in classes:
+                self.depth = 0
+                self.saw_gallery = True
+            return
+        if self.depth == 0 and tag not in self.ALLOWED:
+            self.findings.append(f"<{tag}> as a direct child")
+        if tag not in self.VOID:
+            self.depth += 1
+
+    def handle_endtag(self, tag):
+        if self.depth is None or tag in self.VOID:
+            return
+        if self.depth == 0:
+            # The gallery's own closing tag.
+            self.depth = None
+        else:
+            self.depth -= 1
+
+    def handle_data(self, data):
+        # Whitespace between elements is just the template's formatting.
+        if self.depth == 0 and data.strip():
+            self.findings.append(f"bare text {data.strip()[:60]!r}")
+
+
+def check_galleries(public):
+    """Check that every gallery holds only figures and its own caption.
+
+    Neither the assets nor the orphans check can see this: both ask whether a reference resolves
+    or is reached, and content misplaced inside a gallery resolves and is reached exactly as it
+    would anywhere else. The defect is purely one of structure, so nothing that reasons about
+    URLs can observe it, which is why it survived the conversion and every gate since.
+    """
+    findings, pages = [], 0
+    for path in sorted(public.rglob("index.html")):
+        html = path.read_text(encoding="utf-8", errors="replace")
+        # Cheap reject first, since parsing every built page costs far more than one substring
+        # test and galleries appear on a handful of them. The test is the bare word rather than
+        # `class="gallery`, because minification drops the quotes around a value that does not
+        # need them and says nothing about class order, so the quoted form skips a page whose
+        # markup is merely spelled differently and the gate passes vacuously. This form cannot:
+        # the parser below requires the class token `gallery`, so a page it would find always
+        # contains this string. Matching a page that only mentions the word costs one parse.
+        if "gallery" not in html:
+            continue
+        scan = GalleryScan()
+        scan.feed(html)
+        # Counted from what the parser actually found rather than from the reject above, so the
+        # reported number stays "pages carrying a gallery" and not "pages the word appears on".
+        if not scan.saw_gallery:
+            continue
+        pages += 1
+        rel = str(path.relative_to(public)).replace("\\", "/")
+        findings += [f"{rel}: {finding}" for finding in scan.findings]
+    print(f"gallery: {pages} pages with galleries, {len(findings)} stray nodes inside one")
+    return findings
+
+
 def main(argv):
     if len(argv) != 2:
         sys.exit(f"usage: {argv[0]} <public-dir>")
@@ -198,26 +301,27 @@ def main(argv):
 
     refs = collect_refs(public)
     failures = []
-    for label, missing in (
+    for label, found in (
         ("render", check_render(public)),
         ("media", check_media(public)),
         ("assets", check_assets(public, refs)),
         ("orphans", check_orphans(public, refs)),
+        ("gallery", check_galleries(public)),
     ):
-        if missing:
-            failures.append((label, missing))
+        if found:
+            failures.append((label, found))
 
     if not failures:
         print("\nPASS - the built site honors the URL contract")
         return 0
 
     print()
-    for label, missing in failures:
-        print(f"FAIL {label}: {len(missing)} missing")
-        for item in missing[:20]:
+    for label, found in failures:
+        print(f"FAIL {label}: {len(found)} {FAILURE_NOUN.get(label, 'missing')}")
+        for item in found[:20]:
             print(f"  {item}")
-        if len(missing) > 20:
-            print(f"  ... and {len(missing) - 20} more")
+        if len(found) > 20:
+            print(f"  ... and {len(found) - 20} more")
     return 1
 
 

checks/check-url-parity.py

@@ -37,8 +37,8 @@ I removed the motherboard from the old chassis and installed it in the new SC846
 {{< figure src="/media/2020/01/img%5F5897.jpg?w=1024" alt="" caption="" >}}  
 {{< figure src="/media/2020/01/img%5F5898.jpg?w=1024" alt="" caption="" >}}  
 {{< figure src="/media/2020/01/img%5F5899.jpg?w=768" alt="" caption="" >}}  
-![](/media/2020/02/img_5973.jpg?w=1024)  
-![](/media/2020/02/img_5980.jpg?w=768)  
+{{< figure src="/media/2020/02/img_5973.jpg?w=1024" alt="" caption="" >}}  
+{{< figure src="/media/2020/02/img_5980.jpg?w=768" alt="" caption="" >}}  
 {{< /gallery >}}  
 
 I powered the machine up through remote IPMI KVM, all looked good, and I booted into my Ubuntu Server USB stick so I could SSH into the box, and update the firmware.

content/posts/2020/02/02/recovering-the-firmware-on-a-supermicro-bpn-sas3-846el1-backplane.md

@@ -610,12 +610,12 @@ Here are a few screenshot of the end result:
 
 
 {{< gallery cols="3" >}}  
-[![](/media/2020/06/2020-06-21-1.png?w=1024)](/media/2020/06/2020-06-21-1.png?w=1024)  
-[![](/media/2020/06/2020-06-21-2.png?w=1024)](/media/2020/06/2020-06-21-2.png?w=1024)  
-[![](/media/2020/06/2020-06-21-3.png?w=1024)](/media/2020/06/2020-06-21-3.png?w=1024)  
-[![](/media/2020/06/2020-06-21-4.png?w=1024)](/media/2020/06/2020-06-21-4.png?w=1024)  
-[![](/media/2020/06/2020-06-21-9.png?w=1024)](/media/2020/06/2020-06-21-9.png?w=1024)  
-[![](/media/2020/06/2020-06-21-6.png?w=1024)](/media/2020/06/2020-06-21-6.png?w=1024)  
-[![](/media/2020/06/2020-06-21-7.png?w=1024)](/media/2020/06/2020-06-21-7.png?w=1024)  
-[![](/media/2020/06/2020-06-21-8.png?w=1024)](/media/2020/06/2020-06-21-8.png?w=1024)  
+{{< figure src="/media/2020/06/2020-06-21-1.png?w=1024" alt="" link="/media/2020/06/2020-06-21-1.png?w=1024" caption="" >}}  
+{{< figure src="/media/2020/06/2020-06-21-2.png?w=1024" alt="" link="/media/2020/06/2020-06-21-2.png?w=1024" caption="" >}}  
+{{< figure src="/media/2020/06/2020-06-21-3.png?w=1024" alt="" link="/media/2020/06/2020-06-21-3.png?w=1024" caption="" >}}  
+{{< figure src="/media/2020/06/2020-06-21-4.png?w=1024" alt="" link="/media/2020/06/2020-06-21-4.png?w=1024" caption="" >}}  
+{{< figure src="/media/2020/06/2020-06-21-9.png?w=1024" alt="" link="/media/2020/06/2020-06-21-9.png?w=1024" caption="" >}}  
+{{< figure src="/media/2020/06/2020-06-21-6.png?w=1024" alt="" link="/media/2020/06/2020-06-21-6.png?w=1024" caption="" >}}  
+{{< figure src="/media/2020/06/2020-06-21-7.png?w=1024" alt="" link="/media/2020/06/2020-06-21-7.png?w=1024" caption="" >}}  
+{{< figure src="/media/2020/06/2020-06-21-8.png?w=1024" alt="" link="/media/2020/06/2020-06-21-8.png?w=1024" caption="" >}}  
 {{< /gallery >}}