Skip to content

[RNMobile] Using deferredEventEmitter to wait for the Editor to be mounted before issuing replaceBlock signal - #13699

Merged
mzorz merged 29 commits into
developfrom
rnmobile/stories-bridge-deferred-event
Feb 5, 2021
Merged

mzorz merged 29 commits into
developfrom
rnmobile/stories-bridge-deferred-event

Conversation

@mzorz

@mzorz mzorz commented Dec 31, 2020 •

Copy link
Copy Markdown
Contributor

Should fix #13662 and some other crashes and issues related to OOM with Gutenberg and StoryComposer
see #13662 (comment)

Related gutenberg-mobile PR wordpress-mobile/gutenberg-mobile#2949
Related gutenberg PR WordPress/gutenberg#27944
Related jetpack PR Automattic/jetpack#18540
Related stories library PR Automattic/stories-android#632

Problem

The way we'd do things normally when editing an existing Story block and coming back to Gutenberg was by means of calling startActivityForResult and then once a new block was created by the composer, set is as the result of the action, and have it processed by onActivityResult() in EditPostActivity -> GutenbergEditorFragment.
What we'd do there was just call replaceBlock, using its blockId.

The problem with this is, when in a low memory situation, when we go to the StoryComposer the system might choose to kill the Editor activity, and as such when we come back from the StoryComposer a new instance of the Editor is created, and with this new load this time the blockId no longer matches.

Hence, we need a way to tell which block is which, so to know which block we need to replace in the original Post.

Not only that, but also depending on when things happen, things may lead to unexpected behavior or crashes such as the one referenced in this PR.

Options

  1. First I thought of using one of the mediaFiles ids in the media collection in the Story block, but what if the user removes the slides we were referencing when editing? Or even worse, what if they completely replace all slides with new ones? We would again lose any reference to the original block.

  2. I then attempted injecting a transient blockEditingId among the mediaFiles attributes, which I'd use only for the purpose of matching the original Post block and whatever new slides (even if it were all new slides in the edited Story block) the user may have introduced while editing. This way, we can always rest assured the match was going to be possible, but it felt dirty in a way (getting a new attribute only for this purpose didn't seem quite right).

  3. A third option suggested by @bjtitus was to calculate a hash on the original block, so when the Gutenberg editor reloads the Post when coming back from the StoryComposer, it would be able to calculate it again on the block that needs to be replaced and see if there's a match. If there's a match then we definitely are talking about the same block.

Solution

I went with this latter option, but it also meant a new signal was needed in the bridge, to let the native side of things that a Story block has just been loaded. When this signal reaches the native side, it's time to compare hashes and do call the replaceBlock method accordingly.

Lifecycle

The best way I found to try this all is by turning Don't keep activities to ON in the developer options.

  1. User opens an existing Story post, and taps on the Story block to edit. onRequestMediaFilesEditorLoad() gets called in GutenbergEditorFragment, and before launching the StoryComposer we save the calculated mExternallyEditedBlockOriginalHash` hash to ourselves for later.
  2. User plays in the Story Composer and either cancels or taps on the check mark. If they cancel, we just use a simple flag in mStoryEditingCancelled to handle even listener registration (more on that later). If they tap the check mark:
  3. the new replacing block is built (it's a string), and set as the StoryComposerActivityResult. At the same time, the FrameSaveService gets triggered and saves the slides with temporary ids, also broadcasting progress events all along (**)
  4. we get back to the Editor but, because it was gone, a new instance of the EditPostActivity is created, and we need to wait for Gutenberg to be ready.
    (*) You'll notice at this point no spinning progress wheel is shown while Gutenberg gets ready - this is because we're using a mEditorDidMount variable for configChanges (such as rotation) which we also keep in onSaveInstanceState but in the very special case of Dont Keep Activities we do keep that value but we don't keep the GutenbergEditorFragment - hence it needs to be reloaded but the spinning wheel will be missing. I don't think this is the case for a real OOM situation, but noting it here just in case you think we should address this too.
    (**) a special note re: events handling. Several events in the StoriesEventListener are meant to actually have an impact and update the actual Story block contents (for example, change a mediaFile's id, or updating the URL for a slide from a local file to a remote URL), whereas some other events are used only to paint save or upload progress visually on the block to keep the user informed.

Given the new EditPostActivity and Gutenberg need to be loaded first before we can throw events at it, we wait until everything is loaded in order to start processing those events which were posted by the FrameSaveService and the UploadService. Because timing is of essence in the lifecycle of events (i.e. it only makes sense to send a mediaId change from localId to RemoteId when there is a localId to replace in the first place), we need to process events in the same order in which these are happening and broadcasted. How do we handle things to not miss events? We're using a trick here by using sticky events and only registering EventBus when we're ready to process those events, as opposed to start listening right away when EditPostActivity gets created. This way, we can receive the events in order, and we can also make the modifying callback calls in order to update the Post's content properly.

  1. in this new instance of EditPostActivity, we go through onActivityResult -> call GutenbergEditorFragment's onActivityResult and then we keep the new block structure as given by the StoryComposer to us, then we call replaceStoryEditedBlock, which will not have any effects given the blockId does not exist anymore (but we leave it there, because this call will actually perform its purpose 99% of the time).

At this point we have nothing else to do but wait for Gutenberg to be mounted and ready.

  1. When Gutenberg is fed a Post, it loads each block and assigns each block a blockId. Then for Story blocks, a new signal has been added called mediaFilesBlockReplaceSync which basically triggers itself whenever a Story block is loaded and ran first time, and is received on the native side of the bridge through the onMediaFilesBlockReplaceSync callback method.

  2. now imagine that a Post gets loaded. The post still contains the old content, so we want to make sure the Story block is the one the user originally wanted to update, and replace it with the new block. Then only once the Story block gets replaced, we can then proceed and let the events be thrown at it, so Gutenberg can properly modify and change the ids and URLs as suited. onMediaFilesBlockReplaceSync has this with a flag that gets flipped, because when we replace the block we'll get a new call on onMediaFilesBlockReplaceSync, like this:

7.1. original Post gets loaded, so the Story block sends the mediaFilesBlockReplaceSync signal to the bridge.
7.2 we receive it, calculate the hash and compare it to the hash we saved in step 1 above.
7.3. if it matches, then we go ahead and ask Gutenberg to replace the block with the new block as given by the StoryComposerActivity. At this point we also don't want any events sent to Gutenberg until the block gets actually replaced, so we pause the StoriesEventLIstener by calling mEditorFragmentListener.onReplaceStoryEditedBlockActionSent(); (see internally, it'll pause the listener)
7.4. replacing the block in turn triggers a new call mediaFilesBlockReplaceSync signal to be sent to the bridge once the new block gets loaded, so we land on the same method again. To avoid running the same code we flip the flag mStoryBlockReplacedSignalWait.
7.5. at this point we can trigger events at it again, which we do with mEditorFragmentListener.onReplaceStoryEditedBlockActionReceived();

                        if (mStoryBlockReplacedSignalWait) {
                            // in case we were expecting a fresh block replacement sync signal, let the fragment listener
                            // know so it can process all of the pending block save / update / upload events
                            mStoryBlockReplacedSignalWait = false;
                            mExternallyEditedBlockOriginalHash = null;
                            mEditorFragmentListener.onReplaceStoryEditedBlockActionReceived();
} else {
...

7.6 finally, events get processed and the Post gets updated accordingly. Special mention to StoryFrameMediaUploadedEvent which is a locally defined event to the StoriesEventListener. The trick here is to listen to UploadService events which may be happening out of our control, and post this local event (with postStoryMediaUploadedEvent()) so it gets enqueued by EventBus until it's listened back to in the StoriesEventListener when the listening pause is lifted :)

To test

To test (without don't keep activities):

  1. create a Story post with at least 1 frame, 1 added view
  2. look for it in the Posts list and open it
  3. tap on the Story block to select, then tap again to edit
  4. observe the Story creator fires up and you can make edits. Do make some visible edits and tap the checkmark to save
  5. observe the edits are replaced in the Story block, it gets saved and then uploaded.
  6. tap UPDATE to upload the changes to the remote

To test (with don't keep activities):

  1. create a Story post with at least 1 frame, 1 added view so you can edit later
  2. now set don't keep activities to ON (if you do this before creating your Story, an unrelated issue prevents the Post from saving its content correctly)
  3. look for the post created in step 1 in the Posts list and open it
  4. tap on the Story block to select, then tap again to edit
  5. observe the Story creator fires up and you can make edits. Do make some visible edits and tap the checkmark to save
  6. observe the edits are replaced in the Story block

Do this many times and confirm it works :)

PR submission checklist:

  • I have considered adding unit tests where possible.
  • I have considered adding accessibility improvements for my changes.
  • I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

…ng information when editor activity gets killed while editing on the Story composer
@peril-wordpress-mobile

peril-wordpress-mobile Bot commented Dec 31, 2020 •

Copy link
Copy Markdown

You can trigger optional UI/connected tests for these changes by visiting CircleCI here.

@peril-wordpress-mobile

peril-wordpress-mobile Bot commented Dec 31, 2020 •

Copy link
Copy Markdown

You can test the changes on this Pull Request by downloading the APK here.

mzorz added 8 commits January 22, 2021 10:12
…ts when the Gutenberg editor is ready and the right target block is replaced
refactored and put Stories specific code in StoriesEventLIstener, also wrapped calles in updateAndSavePostAsync() to make sure the Post gets updated with the UI and saved before moving on to processing events.
The StoriesEventListener now has an internal event that is fired and only processed when the listener is not paused.
Finally added method onStoryMediaSavedToRemote to interface StorySaveMediaListener, to make sure to process new mediaId and remote URL on the Post when a Gutenberg instance is not yet mounted when the upload successful event is returned.
…a Double from RN depending on conversinos there
…ue after the first time editing is cancelled
@mzorz mzorz modified the milestones: 16.6 ❄️, 16.7 Jan 25, 2021
@mzorz mzorz added Gutenberg Editing and display of Gutenberg blocks. gutenberg-mobile labels Jan 25, 2021
@mzorz
mzorz marked this pull request as ready for review January 25, 2021 23:13
@mzorz
mzorz requested a review from jd-alexander January 26, 2021 00:06

@jd-alexander jd-alexander left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes @mzorz I really enjoyed reading the discussion to understand the mental model you created to solve this problem. I have taken a look at the code and so far I approve of the approach and the path taken to solve the problem. I tested the solution by toggling the "Dont keep activities" in developer mode and the editor's behavior remained intact so indeed the solution is battle-tested and ready for low memory devices who may find themselves in need of this solution for recovery when editing story content.

I have left some comments and I will be taking a look at the other repositories so I can properly connect the dots across all the repositories involved in making this possible. Once that's done, I will take another look at the code here. Again, thanks for such a thoughtful solution. It's truly appreciated.

I know this is a fix that is targeting an issue in production so I will try my best to wrap up the other PR reviews in a timely manner so that this solution can be deployed to the upcoming release. I am being a bit cautious to ensure I truly understand the different paths being modified here as they are touching critical flows for the story creation process and again, the solution spans multiple paths and repositories so care is necessary.

Comment on lines +503 to +506
if (((HashMap<String, Object>) mediaFile).get("id") instanceof Double) {
Double originalValue = (Double) ((HashMap<String, Object>) mediaFile).get("id");
// now set it back to String to normalize with temporary ids
((HashMap) mediaFile).put("id", String.valueOf(originalValue.longValue()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mzorz I would normally hardcode id value into a constant but that's totally not required. I just spotted it and decided to share 😜

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I totally get what you say and I'd definitely go for a constant when code is self-contained - in this situation I personally find it easier to read the "id" field right there on the spot given it is shared with the JavaScript counterpart so, when following code I have one less level of abstraction / indirection to check (as I can spot the "id" key right away when looking at both codebases, as opposed to checking a constant, figuring out what the constant value is, then trying to mentally match it to what the JS counterpart is doing). For this reason I'd prefer to leave it, but happy to change it otherwise, what do you think @jd-alexander ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it can remain. Now when I think about the main reason my mind told me to put it in a constant would be if we needed to replace the value we would only make this change in one place but now I can't think of any reason where id itself would be changed since it's a universal way of identifying objects so in that sense, for readability and overall consistent it can remain. Thanks for expounding @mzorz

@jd-alexander jd-alexander left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes @mzorz LGTM 🚢

@mchowning

Copy link
Copy Markdown
Contributor

👋 @mzorz ! Thanks so much for writing up such a great PR description and including many useful comments in the code. It's been super helpful to me as I work on a similar issue.

...we need a way to tell which block is which, so to know which block we need to replace in the original Post

For my issue, I was thinking about using the "index" of the block in the block list on the js-side as the identifier. In other words, passing the block's index down from js with the media selection request and then passing it back up with the selected media item in order to identify the relevant block to add the item to. I haven't implemented this or anything yet, and just wanted to touch base and make sure you didn't consider (and dismiss) that option because of some problem you identified.

I'm considering this different approach because the hash-based approach gives the same identifier to all empty blocks of the same type, which means the "first" empty block results in a matching hash regardless of the specific empty block that triggered the media selection. This is a bit of an edge case, but it feels like more of a concern for regular image blocks to me since it seems more likely that a user would have multiple empty image blocks in a post.

Absolutely no worries if you don't have anything to add here. I barely remember what I did last month, much less a year ago. 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Gutenberg Editing and display of Gutenberg blocks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IllegalArgumentException: Required value was null

3 participants