Skip to content

Improves handling of   characters - #20950

Merged
FAMarfuaty merged 5 commits into
release/21.8from
fix/nbsp-handling
Dec 21, 2023
Merged

FAMarfuaty merged 5 commits into
release/21.8from
fix/nbsp-handling

Conversation

@mhkuu

@mhkuu mhkuu commented Dec 15, 2023 •

Copy link
Copy Markdown
Contributor

Context

  • In this PR, we improve handling of   characters for position-based highlighting in general and the keyphrase density assessment in particular. Before this PR, if an   character where to enter the Paper, it would incorrectly be parsed early into a space character, and hence be problematic when highlighting.
  • For search-based highlighting, removing the   character as we did hitherto does not work, but we resolve that when actually applying the marks in the corresponding Premium PR.

Summary

This PR can be summarized in the following changelog entry:

  • Fixes an unreleased bug where   characters would be handled incorrectly when using the highlighting feature in Elementor.
  • [yoastseo] Avoids early conversion of the HTML entity   to its reserved character when running the analysis and adds #nbsp; as a word delimiter.
  • [shopify-seo] Avoids early conversion of the HTML entity   to its reserved character when running the analysis and adds #nbsp; as a word delimiter.

Relevant technical choices:

  • We copy the approach for other HTML entities (see this PR), and convert   to #nbsp; before parse5 can convert it to a space, and mess up our position information.
  • We add #nbsp; as a word delimiter.
  • Note that the current solution generates two Mark objects when the keyphrase is separated by   characters. In mergeConsecutiveAndOverlappingMarkings for consecutive markings, so there we need to add a special rule for &nbsp. However, this method needs to be refactored anyway, this is the corresponding TD task: https://github.com/Yoast/lingo-other-tasks/issues/296

Test instructions

Test instructions for the acceptance test before the PR gets merged

This PR can be acceptance tested by following these steps:

WordPress

Shopify

NOTE: testing in Shopify is mostly for impact check

  • Create a product
  • Add the following paragraphs to the product description
  • Add   character somewhere in the text (you can use the text editor when adding the character)
  • Add a transition word in the first sentence of the second paragraph. (It's been added in the paragraphs provided above)
  • Add   characters in the first sentence of the second paragraph (it's the sentence with the transition words).
  • Open up the Readability analysis.
  • Confirm that transition words assessment finds the sentence with the relevant transition word and highlights it properly when activating the eye marker.
  • Confirm that paragraph too long assessment finds both paragraphs and highlights them properly when activating the eye marker.
    • Feedback: 🔴 Paragraph length: 2 of the paragraphs contain more than the recommended maximum of 150 words. Shorten your paragraphs!
  • Add a keyphrase that appears in the first sentence of the second paragraph.
    • For the provided paragraphs, use the keyphrase paragraphs
  • Confirm that the keyphrase density assessment recognizes the keyphrase and highlights it properly.
    • 🔴 Keyphrase density: The keyphrase was found 18 times. That's way more than the recommended maximum of 14 times for a text of this length. Don't overoptimize!
  • Confirm that the keyphrase distribution assessment recognizes the keyphrase, but does not highlight it, as per Sentences containing   are not highlighted in Consecutive sentences and Keyphrase distribution assessments #19776
    • 🟢 Keyphrase distribution: Good job!

Relevant test scenarios

  • Changes should be tested with the browser console open
  • Changes should be tested on different posts/pages/taxonomies/custom post types/custom taxonomies
  • Changes should be tested on different editors (Block/Classic/Elementor/other)
  • Changes should be tested on different browsers
  • Changes should be tested on multisite

Test instructions for QA when the code is in the RC

  • QA should use the same steps as above.

QA can test this PR by following these steps:

Impact check

This PR affects the following parts of the plugin, which may require extra testing:

  • Affects handling of   characters in position-based highlighting (so currently only the keyphrase density assessment.

UI changes

  • This PR changes the UI in the plugin. I have added the 'UI change' label to this PR.

Other environments

  • This PR also affects Shopify. I have added a changelog entry starting with [shopify-seo], added test instructions for Shopify and attached the Shopify label to this PR.

Documentation

  • I have written documentation for this change.

Quality assurance

  • I have tested this code to the best of my abilities.
  • During testing, I had activated all plugins that Yoast SEO provides integrations for.
  • I have added unit tests to verify the code works as intended.
  • If any part of the code is behind a feature flag, my test instructions also cover cases where the feature flag is switched off.
  • I have written this PR in accordance with my team's definition of done.
  • I have checked that the base branch is correctly set.

Innovation

  • No innovation project is applicable for this PR.
  • This PR falls under an innovation project. I have attached the innovation label.
  • I have added my hours to the WBSO document.

Fixes https://github.com/Yoast/plugins-automated-testing/issues/1244
Fixes https://github.com/Yoast/plugins-automated-testing/issues/1247

… in general and the keyphrase density assessment in particular
@mhkuu mhkuu added the changelog: non-user-facing Needs to be included in the 'Non-userfacing' category in the changelog label Dec 15, 2023
@mhkuu mhkuu added this to the 21.8 milestone Dec 15, 2023
@mhkuu mhkuu added the innovation Innovative issue. Relating to performance, memory or data-flow. label Dec 15, 2023
@mhkuu
mhkuu marked this pull request as ready for review December 15, 2023 13:37
@agnieszkaszuba agnieszkaszuba modified the milestone: 21.8 Dec 19, 2023
@FAMarfuaty FAMarfuaty added the Shopify This PR impacts Shopify. label Dec 21, 2023
@FAMarfuaty

Copy link
Copy Markdown
Contributor

Acceptance testing:

By following the testing instruction in https://github.com/Yoast/wordpress-seo-premium/pull/4167, I get the expected results. With the exception of the known (and acceptable) issues reported by Marina in this comment. We decided that this behaviour is acceptable on the note that that is behaviour would be fixed when we migrate the Paragraph length assessment to use the new HTML Parser.

Additionally, there is a known issue where highlighting cannot be applied to text containing <code> tags. This is also a problem in the current released version and in other editors as well. With the reasonings above, I'll merge this PR and the accompanying PR in premium

@FAMarfuaty
FAMarfuaty merged commit 928f849 into release/21.8 Dec 21, 2023
@FAMarfuaty
FAMarfuaty deleted the fix/nbsp-handling branch December 21, 2023 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog: non-user-facing Needs to be included in the 'Non-userfacing' category in the changelog innovation Innovative issue. Relating to performance, memory or data-flow. Shopify This PR impacts Shopify.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants