[HOLD for payment 2024-04-15] [$500] Characters missing when typing message in compose box

@mallenexpensify

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

Coming from here where we were able to document reliable reproduction steps.

Version Number:
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @mallenexpensify
Slack conversation: here

Reproduction Steps

Open Script Editor (Mac)
New Document > Paste the following script

on run
	set textToType to "JUMP "
	repeat
		emulateTyping(textToType)
		delay 1
	end repeat
end run

on emulateTyping(textToType)
	tell application "System Events"
		repeat with i from 1 to count of characters of textToType
			set currentChar to character i of textToType
			keystroke currentChar
			delay 1
		end repeat
	end tell
end emulateTyping

Press the play button (run the script)
Focus the desktop newDot app composer
Wait until you observe that a letter from the word 'JUMP' is being omitted after being typed.

Expected result:
The word 'JUMP' needs to be typed with consistency in the composer. ignore the first word it is because we had a delay when focusing the input.

Actual result:
After a few tries character gets omitted and the word becomes inconsistent. i.e. JUMP JUMP JMP

Screen.Recording.2024-02-22.at.11.35.41.AM.mov

Platforms:

Which of our officially supported platforms is this issue occurring on?

Screenshots/Videos

View the other issue.

Additional Deliverable

From here

FWIW the issue also occurred when editing the message, I think that should also be fixed in this post.

View all open jobs on GitHub

Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0104493dfd4225a65e
Upwork Job ID: 1763648489851805696
Last Price Increase: 2024-03-13
Automatic offers:

getusha | Reviewer | 0
tienifr | Contributor | 0

@johncschuster

Triggered auto assignment to @johncschuster (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

@johncschuster

@johncschuster, I'll take this, I have context from the issue this came form, and the bug.

Proposal

Please re-state the problem that we are trying to solve in this issue.

Characters missing when typing message in compose box

What is the root cause of that problem?

Race conditions occur when updating the composer value by typing and updating the value from the Onyx draft.

In the main composer, the SilentCommentUpdater update can be triggered when the user types at a debounce interval.

App/src/pages/home/report/ReportActionCompose/SilentCommentUpdater/index.tsx

Lines 34 to 36 in b7a230e

    
               updateComment(comment ?? ''); 
        
           }, [prevCommentProp, prevPreferredLocale, prevReportId, comment, preferredLocale, reportID, updateComment, value, commentRef]);

When editing chat, this is the culprit that causes it to behave similarly to the main composer.

App/src/pages/home/report/ReportActionItemMessageEdit.tsx

Lines 129 to 131 in b7a230e

    
               } 
        
               setDraft(Str.htmlDecode(draftMessage)); 
        
           }, [draftMessage, action]);

The problem is that when typing a comment, the value from the composer will be more recent than the updates above, which means the above updates might use outdated value.

What changes do you think we should make in order to solve the problem?

Ensure the updated value is not outdated by using Composer's most recent value when the user is actively typing.
Track the current user's typing status with the useTypingStatus hook:

const useTypingStatus = ({delay = 1000}: UseTypingStatusProps = {}): UseTypingStatusReturn => {
    const isTypingRef = useRef(false);

    const setTypingFalse = useCallback(() => {
        const debouncedFunc = lodashDebounce(() => {
            isTypingRef.current = false;
            /* The +500 difference in the setTypingFalse debounce function is added
            to ensure that the typing status is not set to false immediately right after 
            the user starts typing delay ended. It allows a small buffer period after the delay 
            before considering the user as not typing anymore. */
        }, delay + 500);
        debouncedFunc();
    }, [delay]);

    const handleUserTyping = useCallback(() => {
        const throttledFunc = lodashThrottle(() => {
            if (!isTypingRef.current) {
                isTypingRef.current = true;
            }
            setTypingFalse();
        }, delay);
        throttledFunc();
    }, [delay, setTypingFalse]);

    return {isTypingRef, handleUserTyping};
};

In ComposerWithSuggestion, we can initiate track typing within onChangeText

   const onChangeText = useCallback(
        (commentValue: string) => {
            handleUserTyping();
            updateComment(commentValue, true);
.............
        },
        [updateComment, handleUserTyping],
    );

Pass the isTypingRef to the SilentCommentUpdater.

                <SilentCommentUpdater
                    reportID={reportID}
                    value={value}
                    updateComment={updateComment}
                    commentRef={commentRef}
                    isTypingRef={isTypingRef}
                />

and use commentRef.current when the user is typing

        const newComment = isTypingRef.current ? commentRef.current : comment;
        updateComment(newComment ?? '');
    }, [prevCommentProp, prevPreferredLocale, prevReportId, comment, preferredLocale, reportID, updateComment, value, commentRef, isTypingRef]);

When editing ReportActionItemMessageEdit.tsx, we can update this code to utilize textInputRef.current?.value during active typing.

    useEffect(() => {
        if (ReportActionsUtils.isDeletedAction(action) || (action.message && draftMessage === action.message[0].html)) {
            return;
        }

        const newDraft = (isTypingRef.current ? textInputRef.current?.value : draftMessage) ?? '';

        setDraft(Str.htmlDecode(newDraft));
    }, [draftMessage, action, isTypingRef]);

Trigger the typing tracking here.

                            onChangeText={(newText) => {
                                handleUserTyping();
                                updateDraft(newText);
                            }} // Debounced saveDraftComment

Or perform early return when isTypingRef.current === true

Branch to test

What alternative solutions did you explore? (Optional)

Alternative 1

We can remove the shouldDebounceSaveComment feature and its function debouncedSaveReportComment. Though it functions, it will revert us to the pre-optimized state, resulting in noticeable lag as the method will be executed for each character.

GMT20240229-024116_Clip_Wildan.M.s.Clip.02_29_2024.mp4

Alternative 2

What if we set the trailing to false and leading to true?

It means the debounced function will be invoked immediately when the event occurs, and then it will be debounced for subsequent calls.

It will not miss a keystroke, but the onyx saveReportComment data will not be properly updated, as it will not call the method at the end of the debounce trailing phase.

The issue illustrated in this video

GMT20240229-031351_Clip_Wildan.M.s.Clip.02_29_2024.mp4

Better solution

We can trigger the saveReportComment function in both the leading and trailing phases to ensure setState doesn't missed important invocation.

Change this lines:

App/src/pages/home/report/ReportActionCompose/ComposerWithSuggestions/ComposerWithSuggestions.tsx

Lines 322 to 328 in 3e74ddf

    
           const debouncedSaveReportComment = useMemo( 
        
               () => 
        
                   lodashDebounce((selectedReportID, newComment) => { 
        
                       Report.saveReportComment(selectedReportID, newComment || ''); 
        
                   }, 1000), 
        
               [], 
        
           );

To

....

    const debouncedSaveReportComment = useMemo(
        () =>
            lodashDebounce(
                (selectedReportID, newComment) => {
                    Report.saveReportComment(selectedReportID, newComment || '');
                }
                , 1000, { leading: true, trailing: true }),
        [],
    );

Also implement this modification in src/pages/home/report/ReportActionItemMessageEdit.tsx for debouncedSaveDraft

Test with 'JUMP' at 1-second typing interval.

GMT20240229-023310_Clip_Wildan.M.s.Clip.02_29_2024.mp4

Test with paragraph at 100 millisecond typing interval

GMT20240229-022734_Clip_Wildan.M.s.Clip.02_29_2024.mp4

Proposal

GMT20240229-024116_Clip_Wildan.M.s.Clip.02_29_2024.mp4

I would like to explain why your solution seems fail to reproduce, but not mitigate the bug:

Specifically note from the docs the following:
If leading and trailing options are true, func is invoked on the trailing edge of the timeout only if the debounced function is invoked more than once during the wait timeout.

If you type 1 character per second, and the wait is 1 second, it will now never call the debounce. I think we see a few of those occurrences in your video.

If we review this report https://ellenaua.medium.com/throttle-debounce-behavior-lodash-6bcae1494e03 it shows how the debounce also now no longer reliable runs at the 1 second interval as is required to actually reproduce the bug with the script.

@jeremy-croff

@jeremy-croff Thanks for your comment. To help readers stay focused, I filter my console log with the 'wildebug' keyword. This is my personal debug code.

@jeremy-croff

@jeremy-croff Thanks for your comment. To help readers stay focused, I filter my console log with the 'wildebug' keyword. This is my personal debug code.

Thank you and my apologies, I edited my wording a bit in the same train of thought. I also deleted my original proposal and have a new one to consider:

Proposal

Please re-state the problem that we are trying to solve in this issue.

We are missing characters when typing

What is the root cause of that problem?

We have an async report draft saving lifecycle to the onyx store, that will rerender the component with what's in the store.
At the same time we have an async setTimeout, with the debounce which can reset the draft text in the store
We have a state variable in the Composer component with the current text value that we set here

These three compete with what's on the screen, what should get debounced, and what's in the store.

To reproduce the bug, we we maximize the draft saving lifecycles, to trigger rerenders, which occur each time you wait >1 sec because the debounce, Let's say you type a character before that change is reflected on the input 1. a debounce will save it, but it gets wiped while it repaints from onyx. And after the repaint 2. changeText is detected and now the debounce will just simply save the text with the missing character.

I was able to isolate the related code early on here in the original thread.

What changes do you think we should make in order to solve the problem?

When reviewing a solution, I wanted to decouples these 3 factors,
I came up with the following:

one source of truth for what we try to save as a draft, which is what's in the components state variable, not in a debounced memory
Example: useEffect(() => _.debounce((selectedReportID) => Report.saveReportComment(selectedReportID, value || ''), 1000), [value]);
note: value is the existing state variable that we set on text update here or directly from the onyx store here

Job added to Upwork: https://www.upwork.com/jobs/~0104493dfd4225a65e

@getusha

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[@getusha] The PR that introduced the bug has been identified. Link to the PR:
[@getusha] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
[@getusha] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
[@getusha] Determine if we should create a regression test for this bug.
[@getusha] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.
[@mallenexpensify] Link the GH issue for creating/updating the regression test once above steps have been agreed upon: https://github.com/Expensify/Expensify/issues/388366

@tienifr

Contributor: @tienifr paid $500 via Upwork
Contributor+: @getusha paid $500 via Upwork

I created the TestRail GH with the test steps below from the PR, comment if there are better steps.

@getusha, please complete the rest of the BZ checklist above. thx

Tests

Open Script Editor (Mac)
New Document > Paste the following script

on run
	set textToType to "JUMP "
	repeat
		emulateTyping(textToType)
		delay 1
	end repeat
end run

on emulateTyping(textToType)
	tell application "System Events"
		repeat with i from 1 to count of characters of textToType
			set currentChar to character i of textToType
			keystroke currentChar
			delay 1
		end repeat
	end tell
end emulateTyping

Press the play button (run the script)
Focus the desktop newDot app composer
Wait until you observe that a letter from the word 'JUMP' is being omitted after being typed.
The word 'JUMP' needs to be typed with consistency in the composer. ignore the first word it is because we had a delay when focusing the input.

@getusha

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

[@getusha] The PR that introduced the bug has been identified. Link to the PR: #25758
[@getusha] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: https://github.com/Expensify/App/pull/25758/files#r1573699887
[@getusha] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: N/a, this is hard to reproduce and catch. i don't think we need other actions to be taken other than adding the tests steps.

Navigation Menu

Repository navigation

[HOLD for payment 2024-04-15] [$500] Characters missing when typing message in compose box #37356

Reproduction Steps

Platforms:

Screenshots/Videos

Additional Deliverable

Proposal

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

What alternative solutions did you explore? (Optional)

Alternative 1

Alternative 2

Proposal

Proposal

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

76 remaining items

Tests

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development


	updateComment(comment ?? '');
	}, [prevCommentProp, prevPreferredLocale, prevReportId, comment, preferredLocale, reportID, updateComment, value, commentRef]);

	}
	setDraft(Str.htmlDecode(draftMessage));
	}, [draftMessage, action]);

	const debouncedSaveReportComment = useMemo(
	() =>
	lodashDebounce((selectedReportID, newComment) => {
	Report.saveReportComment(selectedReportID, newComment \|\| '');
	}, 1000),
	[],
	);

[HOLD for payment 2024-04-15] [$500] Characters missing when typing message in compose box #37356

Description

Reproduction Steps

Platforms:

Screenshots/Videos

Additional Deliverable

Activity

melvin-bot commented on Feb 27, 2024

mallenexpensify commented on Feb 27, 2024

wildan-m commented on Feb 29, 2024

Proposal

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

What alternative solutions did you explore? (Optional)

Alternative 1

Alternative 2

jeremy-croff commented on Feb 29, 2024

Proposal

wildan-m commented on Feb 29, 2024

jeremy-croff commented on Feb 29, 2024

Proposal

Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?

What changes do you think we should make in order to solve the problem?

melvin-bot commented on Mar 1, 2024

76 remaining items

melvin-bot commented on Apr 8, 2024

mallenexpensify commented on Apr 17, 2024

Tests

getusha commented on Apr 21, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions