Park the one-time importer, and make the backfill seed-only - #11
Merged
HarryCordewener merged 2 commits intoJul 31, 2026
Merged
Conversation
The backfill exists to prime one deployment once. Carrying its machinery on main
means four HTML parsers for third-party sites we intend never to fetch again,
compiled and tested in CI for ever -- and a parser that never runs but still
compiles is worse than no parser, because it rots silently and reads as a
supported feature. src/MUI.Import, its tests and tools/live-tintin-import move
to the local import/one-time branch, which also carries the four sources and
three fixes the sources agent added; running the import means checking it out.
The rule was mine and it was too narrow. Yesterday's §7.6 said "the importer is
code and is committed; the imported data is not", written from an instruction
that the initial data is a one-time task and should not live in this GitHub. The
data half was right and the code half was an inference.
What stays is everything the imported rows depend on, because rows outlive
tools:
- migrations/0100_import_provenance.sql. Live rows point at it. A game whose
GENRE came from MudStats says so on its page for as long as that value
stands, and dropping the table turns a provenance chip into an unattributed
fact. A schema is a statement about data that exists, not about which code
is checked out.
- FieldSource.ImportedMeasured/Asserted and IntervalOrigin.ImportedMeasured,
in Catalog already -- §7.5's half-weight grace reads the tier off stored
rows and never off the importer.
- docs/import-sources.md, which is the attribution obligation and the record
of which sources were ingested, which were refused and why. It outlives the
code that acted on it, including the account of the MudStats crawl that went
out before anyone had emailed them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
WalkthroughThe change removes the backfill importer from the main solution, hardens crawler text and banner processing, adds PostgreSQL-backed catalogue selection, and updates measurement-state terminology, internal-field filtering, documentation, and tests. ChangesCatalogue integrity and runtime changes
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Program
participant PostgresData
participant PostgreSQL
participant MainLayout
Program->>PostgresData: ResolveConnectionString()
alt PostgreSQL configured
Program->>PostgresData: AddPostgresCatalogue(connectionString)
Program->>PostgresData: ApplyMigrationsAsync()
PostgresData->>PostgreSQL: Apply pending migrations
MainLayout->>MainLayout: Display measured catalogue
else PostgreSQL not configured
MainLayout->>MainLayout: Display fixture catalogue and demo banner
end
Possibly related PRs
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Comment |
Imported values are gone, and with them the machinery that carried them: the imported_measured / imported_asserted field sources, IntervalOrigin .ImportedMeasured, the import_provenance table, ArchivePolicy's half-weight credit for a third party's reachable history, and the two-tier §7.6 that justified all of it. The backfill contributes a host and a port. Every fact on this site is then measured by this crawler. Three reasons, and the first is the strongest. A game's origin is not one fact. The catalogue will be cross-checked against several directories over time, and any game worth listing appears in more than one of them -- so "imported from MudStats" names whichever fetch happened to run first, not the game, and a provenance chip saying it would present an accident as a fact. There is no honest single-origin field to store. That a game exists is public information. The address of a public MU* is published by its operator to be dialled; recording where we read it adds nothing a reader can use, and it is the part of somebody else's work with the least claim to be ours to republish. And the point is to start with a lot of games and then gather our own data. A history import would fill the heatmaps and reachable strips of exactly the games some other directory was already watching, in a way no reader could tell from our own measurement without reading the fine print. What it costs is stated in §7.5 and §7.6 rather than glossed: every game starts at the archive floor on the day we find it, every heatmap starts empty, and the day-one site knows each game's address and one probe's worth of everything else. That is the intended shape. IntervalOrigin survives as a one-member enum and availability_interval.origin as a column. If another party's measurements are ever ingested, an undifferentiated total would already be in the table and could not be split back apart -- a column is cheap and the distinction is not recoverable after the fact. The base migrations are edited rather than undone by a later one, because nothing is deployed yet. docs/import-sources.md keeps the survey as it was taken: it is the evidence for which sources are worth reading at all, and it says what each would give if this decision is ever revisited. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
HarryCordewener
changed the title
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/import-sources.md`:
- Around line 258-260: Update the import command examples in the live-import
documentation to use the parked tools/live-tintin-import project path instead of
tools/live-import, and state that these commands must be run from the branch
containing that project.
In `@docs/screenshots/README.md`:
- Line 51: Update the fenced code block in the README plain-output example to
specify the text language tag, ensuring the opening fence satisfies markdownlint
MD040.
- Around line 61-64: Update the mui-crawl reproduction command in the README to
replace both live seed hosts with operator-supplied <host:port> placeholders,
and remove the literal ellipsis arguments so every --seed value is valid. Leave
the MUI_POSTGRES example unchanged.
In `@src/MUI.Catalog/Views.cs`:
- Around line 73-74: Update the XML documentation remarks near the reachability
description to replace “uptime” with the approved “reachable”/“unreachable”
terminology, preserving the existing meaning and wording otherwise.
In `@src/MUI.Crawl/BannerText.cs`:
- Around line 106-108: Update SkipEscape to handle ESC sequences with
intermediate bytes: when the byte after ESC is in 0x20–0x2F, consume that
intermediate byte and the subsequent final byte before returning. Preserve
existing handling for other ESC forms so Flatten excludes the complete sequence
and BannerFingerprint.Of and TelnetProbe.LooksUnfinished receive correct data.
In `@src/MUI.Web/Data/PostgresData.cs`:
- Line 5: Convert the namespace declarations in src/MUI.Web/Data/PostgresData.cs
at lines 5-5 and tests/MUI.Web.Tests/SilenceIsNotEvidenceTests.cs at lines 5-5
to file-scoped namespaces, using MUI.Web.Data and MUI.Web.Tests respectively,
and preserve four-space C# indentation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 8af66c84-69c0-45c5-bcdb-0d7ec2d0b130
⛔ Files ignored due to path filters (6)
docs/screenshots/01-home-feeds.pngis excluded by!**/*.pngdocs/screenshots/02-games-listing.pngis excluded by!**/*.pngdocs/screenshots/03-game-page.pngis excluded by!**/*.pngdocs/screenshots/04-plain-mode.pngis excluded by!**/*.pngdocs/screenshots/05-archive.pngis excluded by!**/*.pngdocs/screenshots/06-mobile-game.pngis excluded by!**/*.png
📒 Files selected for processing (79)
.github/workflows/ci.yml.gitignoreCLAUDE.mdMUIndex.slnxREADME.mddocs/import-sources.mddocs/screenshots/README.mddocs/specs/2026-07-30-mu-directory-design.mdmigrations/0100_import_provenance.sqlsrc/MUI.Catalog/Persistence/FieldRegistry.cssrc/MUI.Catalog/Persistence/NpgsqlGameQueries.cssrc/MUI.Catalog/Views.cssrc/MUI.Crawl/BannerText.cssrc/MUI.Crawl/MsspReport.cssrc/MUI.Crawl/ProbeOptions.cssrc/MUI.Crawl/TelnetProbe.cssrc/MUI.Crawl/WireText.cssrc/MUI.Discovery/BannerFingerprint.cssrc/MUI.Discovery/Identity.cssrc/MUI.Import/DirectoryFetcher.cssrc/MUI.Import/EtiquettePlanner.cssrc/MUI.Import/HistorySink.cssrc/MUI.Import/IDirectorySource.cssrc/MUI.Import/IImportWriter.cssrc/MUI.Import/ImportEtiquette.cssrc/MUI.Import/ImportIdentity.cssrc/MUI.Import/ImportPipeline.cssrc/MUI.Import/ImportProvenance.cssrc/MUI.Import/ImportReport.cssrc/MUI.Import/ImportRunner.cssrc/MUI.Import/ImportTier.cssrc/MUI.Import/ImportedAvailabilityWriter.cssrc/MUI.Import/ImportedGame.cssrc/MUI.Import/MUI.Import.csprojsrc/MUI.Import/MeasuredHistorySink.cssrc/MUI.Import/NpgsqlImportProvenanceStore.cssrc/MUI.Import/PolitenessGate.cssrc/MUI.Import/RobotsPolicy.cssrc/MUI.Import/ServiceCollectionExtensions.cssrc/MUI.Import/SourceAttribution.cssrc/MUI.Import/Sources/HtmlLine.cssrc/MUI.Import/Sources/MsspCrawlerTable.cssrc/MUI.Import/Sources/MudStatsSource.cssrc/MUI.Import/Sources/TinTinMsspCrawlerSource.cssrc/MUI.Web/Components/ActivityHeatmap.razorsrc/MUI.Web/Components/ActivitySummary.cssrc/MUI.Web/Components/Layout/MainLayout.razorsrc/MUI.Web/Components/PlainText.cssrc/MUI.Web/Components/ReachSeries.cssrc/MUI.Web/Data/PostgresData.cssrc/MUI.Web/Program.cssrc/MUI.Web/wwwroot/app.csstests/MUI.Crawl.Tests/ProbeSessionTests.cstests/MUI.Crawl.Tests/WireTextTests.cstests/MUI.Crawler.Tests/CrawlCyclePostgresTests.cstests/MUI.Import.Tests/AttributionTests.cstests/MUI.Import.Tests/EtiquetteTests.cstests/MUI.Import.Tests/Fixtures/mudstats-index.htmltests/MUI.Import.Tests/Fixtures/mudstats-world-4dimensions.htmltests/MUI.Import.Tests/Fixtures/tintin-mssp-mudlist.htmltests/MUI.Import.Tests/HistoryTierTests.cstests/MUI.Import.Tests/ImportAgainstPostgresTests.cstests/MUI.Import.Tests/ImportPipelineTests.cstests/MUI.Import.Tests/ImportTierTests.cstests/MUI.Import.Tests/MUI.Import.Tests.csprojtests/MUI.Import.Tests/RobotsAndRateLimitTests.cstests/MUI.Import.Tests/Sources/MudStatsSourceTests.cstests/MUI.Import.Tests/Sources/TinTinMsspCrawlerSourceTests.cstests/MUI.Import.Tests/Support/FakeHttp.cstests/MUI.Import.Tests/Support/Fixture.cstests/MUI.Import.Tests/Support/Harness.cstests/MUI.Import.Tests/Support/InMemoryStores.cstests/MUI.Import.Tests/Support/ManualTimeProvider.cstests/MUI.Import.Tests/Support/PostgresFixture.cstests/MUI.Web.Tests/PlainParityTests.cstests/MUI.Web.Tests/SilenceIsNotEvidenceTests.cstests/MUI.Web.Tests/ThreeStatesTests.cstools/live-tintin-import/Program.cstools/live-tintin-import/live-tintin-import.csproj
💤 Files with no reviewable changes (48)
- tests/MUI.Import.Tests/Fixtures/mudstats-index.html
- src/MUI.Import/MUI.Import.csproj
- src/MUI.Import/DirectoryFetcher.cs
- tools/live-tintin-import/live-tintin-import.csproj
- src/MUI.Import/ImportProvenance.cs
- tests/MUI.Import.Tests/Support/Harness.cs
- tests/MUI.Import.Tests/EtiquetteTests.cs
- tests/MUI.Import.Tests/Support/InMemoryStores.cs
- src/MUI.Import/IDirectorySource.cs
- src/MUI.Import/ImportedAvailabilityWriter.cs
- src/MUI.Import/ServiceCollectionExtensions.cs
- tests/MUI.Import.Tests/HistoryTierTests.cs
- tests/MUI.Import.Tests/RobotsAndRateLimitTests.cs
- src/MUI.Import/ImportTier.cs
- src/MUI.Import/Sources/MudStatsSource.cs
- src/MUI.Import/HistorySink.cs
- src/MUI.Import/ImportedGame.cs
- src/MUI.Import/Sources/TinTinMsspCrawlerSource.cs
- src/MUI.Import/EtiquettePlanner.cs
- tests/MUI.Import.Tests/ImportTierTests.cs
- src/MUI.Import/Sources/HtmlLine.cs
- src/MUI.Import/RobotsPolicy.cs
- tests/MUI.Import.Tests/ImportPipelineTests.cs
- src/MUI.Import/ImportPipeline.cs
- tests/MUI.Import.Tests/Sources/MudStatsSourceTests.cs
- tests/MUI.Import.Tests/Support/ManualTimeProvider.cs
- tools/live-tintin-import/Program.cs
- tests/MUI.Import.Tests/Support/PostgresFixture.cs
- MUIndex.slnx
- tests/MUI.Import.Tests/Support/FakeHttp.cs
- src/MUI.Import/ImportIdentity.cs
- tests/MUI.Import.Tests/Fixtures/mudstats-world-4dimensions.html
- tests/MUI.Import.Tests/Support/Fixture.cs
- src/MUI.Import/ImportEtiquette.cs
- src/MUI.Import/IImportWriter.cs
- src/MUI.Import/MeasuredHistorySink.cs
- src/MUI.Import/ImportRunner.cs
- tests/MUI.Import.Tests/Fixtures/tintin-mssp-mudlist.html
- .github/workflows/ci.yml
- tests/MUI.Import.Tests/ImportAgainstPostgresTests.cs
- tests/MUI.Import.Tests/Sources/TinTinMsspCrawlerSourceTests.cs
- tests/MUI.Import.Tests/AttributionTests.cs
- src/MUI.Import/Sources/MsspCrawlerTable.cs
- src/MUI.Import/PolitenessGate.cs
- src/MUI.Import/NpgsqlImportProvenanceStore.cs
- src/MUI.Import/SourceAttribution.cs
- src/MUI.Import/ImportReport.cs
- tests/MUI.Import.Tests/MUI.Import.Tests.csproj
Comment on lines
+258
to
+260
| ```bash | ||
| dotnet run --project tools/live-import -- --list | ||
| dotnet run --project tools/live-import -- --source MudStats --cache /var/tmp/mudstats --dsn "Host=…" |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Fix the project path in the import commands.
These commands invoke tools/live-import, but the PR parks the one-time importer as tools/live-tintin-import and removes importer execution from main. Following these commands from the main checkout will fail because the project is absent. State the branch requirement and use the parked project path.
Proposed update
+Run these commands from the local `import/one-time` branch:
-dotnet run --project tools/live-import -- --list
-dotnet run --project tools/live-import -- --source MudStats --cache /var/tmp/mudstats --dsn "Host=…"
+dotnet run --project tools/live-tintin-import -- --list
+dotnet run --project tools/live-tintin-import -- --source MudStats --cache /var/tmp/mudstats --dsn "Host=…"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| ```bash | |
| dotnet run --project tools/live-import -- --list | |
| dotnet run --project tools/live-import -- --source MudStats --cache /var/tmp/mudstats --dsn "Host=…" | |
| Run these commands from the local `import/one-time` branch: | |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/import-sources.md` around lines 258 - 260, Update the import command
examples in the live-import documentation to use the parked
tools/live-tintin-import project path instead of tools/live-import, and state
that these commands must be run from the branch containing that project.
| was decoration — and the plain surface carried the same wrong sentence, in the same words, which is | ||
| what makes it a real parity check rather than a second implementation. | ||
|
|
||
| ``` |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add a language tag to the fenced block.
Line 51 starts a fenced block without a language. Use text for the plain-output example so markdownlint MD040 passes.
🧰 Tools
🪛 markdownlint-cli2 (0.23.1)
[warning] 51-51: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/screenshots/README.md` at line 51, Update the fenced code block in the
README plain-output example to specify the text language tag, ensuring the
opening fence satisfies markdownlint MD040.
Source: Linters/SAST tools
Comment on lines
+61
to
+64
| ```bash | ||
| mui-crawl --connection "…" --seed mush.pennmush.org:4201 --seed mud.kharkov.org:3000 … | ||
| MUI_POSTGRES="…" dotnet run --project src/MUI.Web | ||
| ``` |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Do not commit live seed hosts in the reproduction command.
Line 62 publishes mush.pennmush.org:4201 and mud.kharkov.org:3000 as crawl targets. Use operator-supplied <host:port> placeholders instead. Remove the literal … arguments because they are not valid seed values.
Proposed documentation change
-mui-crawl --connection "…" --seed mush.pennmush.org:4201 --seed mud.kharkov.org:3000 …
-MUI_POSTGRES="…" dotnet run --project src/MUI.Web
+mui-crawl --connection "<postgres-connection>" --seed "<host:port>" --seed "<host:port>"
+MUI_POSTGRES="<postgres-connection>" dotnet run --project src/MUI.WebBased on learnings: do not commit real-host seed lists; keep live crawl inputs local.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| ```bash | |
| mui-crawl --connection "…" --seed mush.pennmush.org:4201 --seed mud.kharkov.org:3000 … | |
| MUI_POSTGRES="…" dotnet run --project src/MUI.Web | |
| ``` |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/screenshots/README.md` around lines 61 - 64, Update the mui-crawl
reproduction command in the README to replace both live seed hosts with
operator-supplied <host:port> placeholders, and remove the literal ellipsis
arguments so every --seed value is valid. Leave the MUI_POSTGRES example
unchanged.
Source: Learnings
Comment on lines
+73
to
+74
| /// which is the one thing this site may never do: a game found an hour ago would have 167 hours | ||
| /// of a perfect week's uptime described as downtime. Reachability has its own strip, measured |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use “reachable” instead of “uptime” in the XML documentation.
The new remarks use “uptime” at Line 74. Replace it with “unreachable” wording.
Proposed wording
-/// a game found an hour ago would have 167 hours
-/// of a perfect week's uptime described as downtime. Reachability has its own strip, measured
+/// a game found an hour ago would otherwise be described as unreachable for 167 hours of the
+/// week. Reachability has its own strip, measuredAs per coding guidelines: Use “reachable,” never “uptime,” in schema, APIs, code, and user-facing copy.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/MUI.Catalog/Views.cs` around lines 73 - 74, Update the XML documentation
remarks near the reachability description to replace “uptime” with the approved
“reachable”/“unreachable” terminology, preserving the existing meaning and
wording otherwise.
Source: Coding guidelines
Comment on lines
+106
to
+108
| // Anything else two-byte: ESC 7, ESC =, ESC ( B and friends. | ||
| return i; | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -e
file=$(fd -t f 'BannerText\.cs$' . | head -n 1)
printf '%s\n' "FILE=$file"
cat -n "$file" | sed -n '1,140p'
printf '\n-- usages --\n'
rg -n -C 3 'BannerText\.(Flatten|SkipEscape)|BannerFingerprint|LooksUnfinished|SlightBannerLength' --glob '*.{cs,md}'
printf '\n-- related tests --\n'
rg -n -C 3 'Flatten|Escape|ANSI|CSI|OSC|BannerFingerprint' --glob '*Tests*.cs' --glob '*.cs'Repository: SharpMUSH/MUIndex
Length of output: 4111
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '-- C# files --'
git ls-files '*.cs' | sed -n '1,240p'
printf '\n-- symbols and references --\n'
rg -n -C 4 --glob '*.cs' 'Flatten|SkipEscape|BannerFingerprint|LooksUnfinished|SlightBannerLength'
printf '\n-- project/docs references --\n'
rg -n -C 3 'BannerFingerprint|SlightBannerLength|BannerPatience|§7\.3|Flatten|charset|ISO-2022|VT100' --glob '*.cs' --glob '*.md' --glob '*.json' --glob '*.yml' --glob '*.yaml'
printf '\n-- test project files --\n'
git ls-files | rg '(^|/)(test|tests|.*Test.*)/|Tests?\.csproj$|\.Tests?\.cs$'Repository: SharpMUSH/MUIndex
Length of output: 8060
🏁 Script executed:
#!/bin/bash
set -e
python3 - <<'PY'
def skip_escape(banner, start):
i = start + 1
if i >= len(banner):
return start
if banner[i] == '[':
i += 1
while i < len(banner):
if '@' <= banner[i] <= '~':
return i
i += 1
return len(banner) - 1
if banner[i] == ']':
i += 1
while i < len(banner):
if banner[i] == '\a':
return i
if banner[i] == '\x1b' and i + 1 < len(banner) and banner[i + 1] == '\\':
return i + 1
i += 1
return len(banner) - 1
return i
def flatten(banner):
out = []
pending_space = False
i = 0
while i < len(banner):
ch = banner[i]
if ch == '\x1b':
i = skip_escape(banner, i)
elif ch.isspace():
pending_space = bool(out)
elif ord(ch) < 32 or ord(ch) == 127:
pass
else:
if pending_space:
out.append(' ')
pending_space = False
out.append(ch)
i += 1
return ''.join(out)
cases = {
'designate ASCII': '\x1b(BMUD',
'designate DEC': '\x1b)0MUD',
'two-byte 7': '\x1b7MUD',
'two-byte =': '\x1b=MUD',
'truncated intermediate': '\x1b(',
'truncated ESC': '\x1b',
'CSI SGR': '\x1b[31mMUD',
'OSC BEL': '\x1b]0;title\aMUD',
'OSC ST': '\x1b]0;title\x1b\\MUD',
}
for name, value in cases.items():
print(f'{name}: {flatten(value)!r}')
PYRepository: SharpMUSH/MUIndex
Length of output: 326
🏁 Script executed:
#!/bin/bash
set -e
printf '%s\n' '-- BannerFingerprint.cs --'
cat -n src/MUI.Discovery/BannerFingerprint.cs
printf '\n-- TelnetProbe outline and relevant sections --\n'
ast-grep outline src/MUI.Crawl/TelnetProbe.cs --match 'class TelnetProbe' --view expanded
rg -n -C 8 'LooksUnfinished|SlightBannerLength|BannerPatience|BannerText' src/MUI.Crawl/TelnetProbe.cs src/MUI.Crawl/ProbeOptions.cs
printf '\n-- BannerFingerprint tests --\n'
cat -n tests/MUI.Discovery.Tests/BannerFingerprintTests.cs
printf '\n-- crawl project references --\n'
rg -n -C 4 'BannerText|Flatten|LooksUnfinished|SlightBannerLength' src/MUI.Crawl src/MUI.Discovery tests/MUI.Crawl.Tests tests/MUI.Discovery.TestsRepository: SharpMUSH/MUIndex
Length of output: 25250
Fix SkipEscape to consume ESC sequences with intermediate bytes.
When the byte after ESC is in 0x20–0x2F, consume it and the following final byte. Otherwise, ESC ( B leaves B in Flatten output. This corrupts BannerFingerprint.Of and can change TelnetProbe.LooksUnfinished length.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/MUI.Crawl/BannerText.cs` around lines 106 - 108, Update SkipEscape to
handle ESC sequences with intermediate bytes: when the byte after ESC is in
0x20–0x2F, consume that intermediate byte and the subsequent final byte before
returning. Preserve existing handling for other ESC forms so Flatten excludes
the complete sequence and BannerFingerprint.Of and TelnetProbe.LooksUnfinished
receive correct data.
| using MUI.Catalog.Persistence; | ||
| using Npgsql; | ||
|
|
||
| namespace MUI.Web.Data; |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use file-scoped namespaces in both new C# files.
src/MUI.Web/Data/PostgresData.cs#L5-L5: Change tonamespace MUI.Web.Data;.tests/MUI.Web.Tests/SilenceIsNotEvidenceTests.cs#L5-L5: Change tonamespace MUI.Web.Tests;.
As per coding guidelines, “Use file-scoped namespaces and four-space indentation in C#.”
📍 Affects 2 files
src/MUI.Web/Data/PostgresData.cs#L5-L5(this comment)tests/MUI.Web.Tests/SilenceIsNotEvidenceTests.cs#L5-L5
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/MUI.Web/Data/PostgresData.cs` at line 5, Convert the namespace
declarations in src/MUI.Web/Data/PostgresData.cs at lines 5-5 and
tests/MUI.Web.Tests/SilenceIsNotEvidenceTests.cs at lines 5-5 to file-scoped
namespaces, using MUI.Web.Data and MUI.Web.Tests respectively, and preserve
four-space C# indentation.
Source: Coding guidelines
This was referenced Jul 31, 2026
HarryCordewener
changed the base branch from
main
to
feat/real-data-and-referral-crawl
July 31, 2026 15:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #9 — review that first; this branch is #9 plus two commits.
1. The importer is not carried on
mainThe backfill exists to prime one deployment once. Carrying its machinery here means four HTML parsers for third-party sites we intend never to fetch again, compiled and tested in CI for ever. A parser that never runs but still compiles is worse than no parser: it rots silently and reads as a supported feature.
src/MUI.Import,tests/MUI.Import.Testsandtools/live-tintin-import→ the localimport/one-timebranch, which also carries everything from the now-closed #10. PR #10 is closed and its remote branch deleted, so nothing importer-shaped is on GitHub.2. The backfill takes addresses, and nothing else
Imported values are gone, and with them everything that carried them:
FieldSource.ImportedMeasured/ImportedAsserted,IntervalOrigin.ImportedMeasured, theimport_provenancetable,ArchivePolicy's half-weight credit for a third party's reachable history, and the two-tier §7.6 that justified it.Three reasons, strongest first:
What it costs, in the spec rather than glossed: every game starts at the archive floor on the day we find it, every heatmap starts empty, and the day-one site knows each game's address plus one probe's worth of everything else. That is the intended shape.
IntervalOriginsurvives as a one-member enum, andavailability_interval.originas a column: if another party's measurements are ever ingested, an undifferentiated total would already be in the table and could not be split back apart.Base migrations are edited rather than undone by a later one, because nothing is deployed.
docs/import-sources.mdkeeps the survey as it was taken — it is the evidence for which sources are worth reading at all, and says what each would give if this is ever revisited.Also, on the parked branch
ContactedMaintainerfor MudStats defaulted totruewith a comment stating the maintainer had been approached. Nobody had emailed them; a 143-page crawl went out on the strength of it. Default isfalseagain and the claim moves to a caller who can make it (--contacted MudStats).CLAUDE.mdgains it as a "Never".Five suites, 633 tests, zero warnings.
🤖 Generated with Claude Code
https://claude.ai/code/session_01NrGKmKcRCGktyhRTFbQDMk