Repository navigation
principle-test-behavior-not-implementation: listed matchers do not all pass when imports return undefined #474
Description
Observed at commit 2eb7ed4: pstack/skills/principle-test-behavior-not-implementation/SKILL.md lines 15-18.
The skill's check is "would the test still pass if every function it imports returned undefined?" and it lists, as "Five shapes that still pass when every imported function returns undefined", assertions such as:
toBeDefined,toBeTruthy,toBeInstanceOf,toBeGreaterThan(0)toEqual([]),toHaveLength(0)
Problem
Several of these do not pass when the value is undefined:
expect(undefined).toBeDefined() // fails
expect(undefined).toBeTruthy() // fails
expect(undefined).toBeInstanceOf(Foo) // fails
expect(undefined).toBeGreaterThan(0) // throws (received value must be a number)
expect(undefined).toEqual([]) // fails
expect(undefined).toHaveLength(0) // throws (received value must have a length property)They are weak assertions, but the stated reason ("still passes when every import returns undefined") is false for them, which can lead an agent following the rule to delete tests that do fail when the function breaks, or to distrust the check. (not.toThrow, toBeUndefined, not.toHaveBeenCalled and a missing expect do pass.)
Suggested fix
Describe the principle by what the test can detect rather than by matcher name: ask whether the assertion would still pass if the function returned a wrong but plausible value (an empty array, true, a different object). Either remove the matchers that fail on undefined from the list, or move them to a separate "weak but not vacuous" group with the accurate explanation, and keep the undefined claim only for the matchers that truly pass.
Activity
deanrie commentedon Oct 3, 2026
Confirmed by running each matcher against
undefinedin vitest:toBeDefined,toBeTruthy,toBeInstanceOf,toBeGreaterThan(0),toEqual([])andtoHaveLength(0)all fail (the last two throw), whilenot.toThrow,toBeUndefined,not.toHaveBeenCalledandnot.toBe(wrongValue)pass — exactly your split.#487 already restructures the list into "vacuous: still passes on
undefined" vs "weak but not vacuous", matching those results. I've reviewed it and flagged it for merge. One thing it leaves as-is is the opening check sentence, which still describes only theundefinedstand-in; I've noted that it should also mention the "wrong but plausible value" test you proposed, so the check covers the weak group too.mingleiw commentedon Oct 16, 2026
This looks resolved on main. #633 (merged Oct 14) restated
principle-test-behavior-not-implementationas two imagined edits and removed the matcher list. Main at f713f14 no longer namestoBeDefined,toBeTruthy,toHaveLength, ortoEqual([]), and the "still pass when every imported function returnsundefined" sentence is gone.#476 and #487 edit that removed list, and both now conflict with main. If you agree, this issue and those two PRs can be closed.