Repository navigation
feat: offline support - #1527
Conversation
…act-native into vishal/offline-support-with-hydration
…tream/stream-chat-react-native into vishal/offline-support-with-hydration-1
…/stream-chat-react-native into vishal/offline-support-with-hydration-1
999750e to
0a89925
Compare
0a89925 to
8742a26
Compare
…act-native into vishal/offline-support-with-hydration-1
…-react-native into vishal/offline-support-with-hydration-1
fc70d25 to
361a65e
Compare
…-react-native into vishal/offline-support-with-hydration-1
…hub.com/GetStream/stream-chat-react-native into vishal/offline-support-with-hydration-1
…-react-native into vishal/offline-support-with-hydration-1 # Conflicts: # package/jest-setup.js # package/src/components/Attachment/Gallery.tsx # package/src/components/Avatar/Avatar.tsx # package/src/components/Avatar/GroupAvatar.tsx
santhoshvai
left a comment
There was a problem hiding this comment.
Just some minor comments.. awesome work @vishalnarkhede
| { | ||
| "editor.formatOnSave": true | ||
| } | ||
| "editor.formatOnSave": false |
There was a problem hiding this comment.
I disagree.. temporary I guess?
There was a problem hiding this comment.
Yes yes, only temporary!!
|
I believe we need to change the PR title and commit messages to something appropriate before merging? |
e8dddab to
9aa56e2
Compare
35aaee1 to
ea828c3
Compare
|
Lint issues are fixed now. |
|
Hey @vishalnarkhede, I think it will make sense to keep one of our sample apps (SA/TSM) without offline support to monitor regression. |
|
Let's verify that we don't have any regression. This means that if the user will NOT opt-in to offline support and WILL NOT install |
| @@ -0,0 +1,44 @@ | |||
| import Database from 'better-sqlite3'; | |||
|
|
|||
There was a problem hiding this comment.
What is the idea behind creating a JS file over TS?
| }; | ||
|
|
||
| it('should create tables on first load if offline feature is enabled', async () => { | ||
| const { getByTestId } = render( |
There was a problem hiding this comment.
can you also expect that the DB tables are not existing before the rendering?
| </Chat>, | ||
| ); | ||
|
|
||
| const expectCIDsOnUIToBeInDB = (queryAllByA11yRole) => { |
There was a problem hiding this comment.
Can we move all expect funcs to a utils file?
| /* eslint-disable no-underscore-dangle */ | ||
| import React from 'react'; | ||
| import { Text, View } from 'react-native'; | ||
|
|
There was a problem hiding this comment.
I know you might be annoying fighting TS in a test file, yet we agreed that when we create new test files, they'll be TS ones over JS
| expect(matchingReadRows.length).toBe(1); | ||
| expect(matchingReadRows[0].unreadMessages).toBe(0); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
This test file is way too long, can you consider splitting it:
- by creating files that have to do with certain domains/topics etc.
- moving setup/expectation funcs to a different utils file
- using it.each where applicable (this will also reduce code duplications)
| uri: | ||
| !imageProp || | ||
| imageProp.includes(randomImageBaseUrl) || | ||
| imageProp.includes(randomSvgBaseUrl) | ||
| ? imageProp?.includes(streamCDN) | ||
| ? imageProp | ||
| : `${randomImageBaseUrl}${ | ||
| name ? `?name=${getInitials(name)}&size=${size}` : '' | ||
| }` | ||
| : getResizedImageUrl({ | ||
| height: size, | ||
| url: imageProp, | ||
| width: size, | ||
| }), | ||
| }} |
There was a problem hiding this comment.
Can we please refactor this in the future?
…-react-native into vishal/offline-support-with-hydration-1
…hub.com/GetStream/stream-chat-react-native into vishal/offline-support-with-hydration-1
* feat: wip jump to a message after using the new around messages api * refactor: scroll to first unread channel * refactor: loading more recent messages should not rely on isUpToDate flag anymore * fix: set noMoreMessages flag to false when we are jumping to a message * fix: scroll to bottom button should not rely on isUpToDate flag * fix: show scroll to bottom button using the new hasNoMoreRecentMessages prop * fix: replace more isUpToDate calls with hasNoMoreRecentMessagesToLoad * feat: add documentation for hasNoMoreRecentMessagesToLoad * feat: update stream-chat-js to 6.3.0 * chore: remove unused import of Text * fix: readd the removed console log of error * feat: add documentation for loadChannelAroundMessage * chore: remove unneeded comment * chore: move scrollToDebounceTimeoutRef to the top of messageList * fix: remove unused variables * fix: update jest snapshot for the new prop addition for messageList * fix: remove unused useEffect in messageList * fix: fix the failing message list test * fix: channel unread indicator was showing up even when marked read * fix: scroll to bottom indicator must not be always shown on open * Revert "fix: fix the failing message list test" This reverts commit d605744. * fix: sometimes the channel was not loaded to the latest message * feat: add maxToRenderPerBatch prop to the messageList this reduces the scrollToIndex failure rate * fix: when scrolling to a non rendered item, highlight was not shown * fix: update snapshot to fix unit test * fix: scroll to messageId didnt work if scrollToFirstUnreadMsg is set * fix: ensure setTargetedMessage happens after messages array is copied * fix: simplified channel logic of messageId scroll * fix: remove unnecessary console log * docs: create a new version of docs for v5 (#1596) * docs: upgrade helper guide (#1605) * docs: cut a new version of docs for v5 * docs: upgrade helper guide * docs: drop v5 changes from v4 docs * docs: add reason for cameraroll fork replacement * docs: make removing cameraroll to be non-optional Co-authored-by: Santhosh Vaiyapuri <santhoshvai@gmail.com> * feat: offline support (#1527) * feat: offline support * refactor: drop un-used function pragmaUserVersion * refactor: reverting changes to ChannelPreview * fix: fixing migration code issue * refactor: drop limits from queries * refactor: handling of message.read events and reads entity * refactor: added members to separate table * refactor: relative explicit query with left join to enrich users * refactor: cleanup and reorganizing * refactor: added index and foreign key constraints * refactor: changes to be compatible with js client * test: mocking sqlite * refactor: added missing fields on channels table * refactor: changes to be in sync with event from js client * refactor: renaming table and cleanup of columns * refactor: moving all quick-sqlite logic to separate class * test: added simple test for offline support * refactor: handle member related events * refactor: added usage of sync api * refactor: cleaning up un-necessary packages * refactor: fixed issue with update message foreign key constraint failure * refactor: cleanup * refactor: cleanup * tests: writing tests for offline feature * refactor: removed dev-menu package and usage * refactor: remove unused initDevMenu import * refactor: remove un-used flipper packages * docs: adding code comments * refactor: removing unnecessary generics * refactor: cleanup of mapper functions * refactor: renaming offlineChannelsActive to staticChannelsActive * refactor: removed metro config changes * buiild: fixed issue with manual release script * refactor: cleanup of handleEventToSyncDB * refactor: fixing lint issue * fix: removed redundant usage of useStreami18n hook * fix: issue with message not displayed in static state * build: updating stream-chat version * fix: issue with appendWhereClause and undefined value * fix: issue with members state initialization * build: do not format on save * fix: issue mapping memberCount to api response * build: upgrading stream-chat version * fix: reposition image error indicator * refactor: use network error handling to useLoadingImage * fix: image loading issue after network is recovered * fix: logic for sync api and lint issues * feat: point cameraroll dependency to the stream fork version * fix: message.new event handling * refactor: remove unnecessary client as hook dep * refactor: code review changes * refactor: moving offline support logic to Chat component * refactor: fix lint and typescript issues * refactor: handle quick-sqlite not being installed in better way * docs: lint fixes Co-authored-by: Mads Røskar <madshvero@gmail.com> Co-authored-by: stevegalili <galiliziv@gmail.com> Co-authored-by: Santhosh Vaiyapuri <santhoshvai@gmail.com> * build: fix the doc paths * docs: update upgrade helper doc * docs: update upgrade helper doc with logout logic changes * docs: change rc to beta in version label * docs: add pod install command to upgrade helper * build: update target branch name * fix: flipper database plugin setup * fix: handling of createSelectQuery function * build: upgrade better sqlite and quick sqlite dev dep * fix: open the selected image when pressed in the image grid (#1585) * refactor!: rename gallery state objects to match their data * fix: open the selected image when pressed in the image grid * style: reorder object keys to be alphabetic * chore: rename image grid props * fix: rename setters to match new names * test: update snapshots * test: move bettersqlite test utils to src/test-utils * test: gitignore locally generated database files * style: reorder imports * style: remove redundant whitespace * refactor: export store apis * fix: set an initial image index for the ImageGallery (#1639) * docs: update upgrade helper guide * docs: rename setImages and setImage * docs: fix broken mdx imports * docs: remove unintended newline in import * docs: update upgrade helper guide * feat: upgrade stream-chat to 7.0.0-offline-support.4 * docs: update upgrade helper guide * docs: offline support docs * ignore intelliJ files. * Update broken external URLs * Update broken internal links * docs: replace cameraroll to our fork in install steps * docs: upgrading v4 dopcs * build: connection docusaurus gh action to develop and main * feat: remove cameraroll fork and use v5 original cameraroll * feat: upgrade stream-chat to v7 stable Co-authored-by: Santhosh Vaiyapuri <santhoshvai@gmail.com> Co-authored-by: Vishal Narkhede <vishalnarkhede.iitd@gmail.com> Co-authored-by: Mads Røskar <madshvero@gmail.com> Co-authored-by: Mads Røskar <mads.roskar@getstream.io> Co-authored-by: nash0x7e2 <mail@neevash.dev>
* feat: wip jump to a message after using the new around messages api * refactor: scroll to first unread channel * refactor: loading more recent messages should not rely on isUpToDate flag anymore * fix: set noMoreMessages flag to false when we are jumping to a message * fix: scroll to bottom button should not rely on isUpToDate flag * fix: show scroll to bottom button using the new hasNoMoreRecentMessages prop * fix: replace more isUpToDate calls with hasNoMoreRecentMessagesToLoad * feat: add documentation for hasNoMoreRecentMessagesToLoad * feat: update stream-chat-js to 6.3.0 * chore: remove unused import of Text * fix: readd the removed console log of error * feat: add documentation for loadChannelAroundMessage * chore: remove unneeded comment * chore: move scrollToDebounceTimeoutRef to the top of messageList * fix: remove unused variables * fix: update jest snapshot for the new prop addition for messageList * fix: remove unused useEffect in messageList * fix: fix the failing message list test * fix: channel unread indicator was showing up even when marked read * fix: scroll to bottom indicator must not be always shown on open * Revert "fix: fix the failing message list test" This reverts commit d605744. * fix: sometimes the channel was not loaded to the latest message * feat: add maxToRenderPerBatch prop to the messageList this reduces the scrollToIndex failure rate * fix: when scrolling to a non rendered item, highlight was not shown * fix: update snapshot to fix unit test * fix: scroll to messageId didnt work if scrollToFirstUnreadMsg is set * fix: ensure setTargetedMessage happens after messages array is copied * fix: simplified channel logic of messageId scroll * fix: remove unnecessary console log * docs: create a new version of docs for v5 (#1596) * docs: upgrade helper guide (#1605) * docs: cut a new version of docs for v5 * docs: upgrade helper guide * docs: drop v5 changes from v4 docs * docs: add reason for cameraroll fork replacement * docs: make removing cameraroll to be non-optional Co-authored-by: Santhosh Vaiyapuri <santhoshvai@gmail.com> * feat: offline support (#1527) * feat: offline support * refactor: drop un-used function pragmaUserVersion * refactor: reverting changes to ChannelPreview * fix: fixing migration code issue * refactor: drop limits from queries * refactor: handling of message.read events and reads entity * refactor: added members to separate table * refactor: relative explicit query with left join to enrich users * refactor: cleanup and reorganizing * refactor: added index and foreign key constraints * refactor: changes to be compatible with js client * test: mocking sqlite * refactor: added missing fields on channels table * refactor: changes to be in sync with event from js client * refactor: renaming table and cleanup of columns * refactor: moving all quick-sqlite logic to separate class * test: added simple test for offline support * refactor: handle member related events * refactor: added usage of sync api * refactor: cleaning up un-necessary packages * refactor: fixed issue with update message foreign key constraint failure * refactor: cleanup * refactor: cleanup * tests: writing tests for offline feature * refactor: removed dev-menu package and usage * refactor: remove unused initDevMenu import * refactor: remove un-used flipper packages * docs: adding code comments * refactor: removing unnecessary generics * refactor: cleanup of mapper functions * refactor: renaming offlineChannelsActive to staticChannelsActive * refactor: removed metro config changes * buiild: fixed issue with manual release script * refactor: cleanup of handleEventToSyncDB * refactor: fixing lint issue * fix: removed redundant usage of useStreami18n hook * fix: issue with message not displayed in static state * build: updating stream-chat version * fix: issue with appendWhereClause and undefined value * fix: issue with members state initialization * build: do not format on save * fix: issue mapping memberCount to api response * build: upgrading stream-chat version * fix: reposition image error indicator * refactor: use network error handling to useLoadingImage * fix: image loading issue after network is recovered * fix: logic for sync api and lint issues * feat: point cameraroll dependency to the stream fork version * fix: message.new event handling * refactor: remove unnecessary client as hook dep * refactor: code review changes * refactor: moving offline support logic to Chat component * refactor: fix lint and typescript issues * refactor: handle quick-sqlite not being installed in better way * docs: lint fixes Co-authored-by: Mads Røskar <madshvero@gmail.com> Co-authored-by: stevegalili <galiliziv@gmail.com> Co-authored-by: Santhosh Vaiyapuri <santhoshvai@gmail.com> * build: fix the doc paths * docs: update upgrade helper doc * docs: update upgrade helper doc with logout logic changes * docs: change rc to beta in version label * docs: add pod install command to upgrade helper * build: update target branch name * fix: flipper database plugin setup * fix: handling of createSelectQuery function * build: upgrade better sqlite and quick sqlite dev dep * fix: open the selected image when pressed in the image grid (#1585) * refactor!: rename gallery state objects to match their data * fix: open the selected image when pressed in the image grid * style: reorder object keys to be alphabetic * chore: rename image grid props * fix: rename setters to match new names * test: update snapshots * test: move bettersqlite test utils to src/test-utils * test: gitignore locally generated database files * style: reorder imports * style: remove redundant whitespace * refactor: export store apis * fix: set an initial image index for the ImageGallery (#1639) * docs: update upgrade helper guide * docs: rename setImages and setImage * docs: fix broken mdx imports * docs: remove unintended newline in import * docs: update upgrade helper guide * feat: upgrade stream-chat to 7.0.0-offline-support.4 * docs: update upgrade helper guide * docs: offline support docs * ignore intelliJ files. * Update broken external URLs * Update broken internal links * docs: replace cameraroll to our fork in install steps * docs: upgrading v4 dopcs * build: connection docusaurus gh action to develop and main * feat: remove cameraroll fork and use v5 original cameraroll * feat: upgrade stream-chat to v7 stable Co-authored-by: Santhosh Vaiyapuri <santhoshvai@gmail.com> Co-authored-by: Vishal Narkhede <vishalnarkhede.iitd@gmail.com> Co-authored-by: Mads Røskar <madshvero@gmail.com> Co-authored-by: Mads Røskar <mads.roskar@getstream.io> Co-authored-by: nash0x7e2 <mail@neevash.dev> docs: update upgrade-helper for v5 docs: update stream-chat version in docs docs: make v5 docs stable
Whats Changed:
First version of offline support has been added to the SDK:
syncapi and WS events to keep DB in sync with backend.Screen Recordings
Following screen recordings display the loading time of chat on Slow 3G network
Screen.Recording.2022-07-15.at.20.32.37.mov
Screen.Recording.2022-07-15.at.20.34.15.mov
Description
JS client changes
To support offline storage, js client needs some tweaking as well. The changes are done in this PR - GetStream/stream-chat-js#1011
How to enable offline support
Chat component accepts a prop named
enableOfflineSupportwhich can be used to toggle this feature.When do we store data?
client.queryChannels()orchannel.query()call is made, js client will emit a local event namedchannels.queriedand send the channel state (from api) as part of event payload. Within RN SDK we will listen to this event and always store the channels from payload in DB. This way we don't need to add store channels after every call to channel.watch() from SDK. And also customer might have a custom implementation of props such asonMessageNew,onAddedToChannelon ChannelList, where they might be doing a channel.watch() call.syncapi to fetch all the missed events and run all the events through event handlers for storage. https://github.com/GetStream/stream-chat-react-native/blob/35aaee1f2e3c2ec2a0d9767bcc86a577a9393f50/package/src/components/Chat/hooks/useConnectionRecovered.tsNOTE: Upsert queries to DB are batched as much as possible to lower the overhead of interaction between JS and sqlite.
How do we store data?
react-native-quick-sqlitepackage which uses fast bindings via JSI.QuickSqliteClientstatic class, which is basically an interface to deal with quick sqlite package.schema.ts. We store the data in 6 different normalised table -channels,messages,members,users,reactions,reads. Tables are basically mapped version of data/state received in API response from server. Mapper functions help with converting the API response to storable format e.g.,mapMessageToStorablechannelQueriestable stores channel ids and corresponding stringified version of filters + sortuserSyncStatustable stores the datetime stamp of last time database was synced. This value is set aftersyncapi is called. Please check this slack thread for discussionHow do we use offline stored data?
client.hydrateActiveChannels(channels)to hydrate the client with API response generated from local DB. This function will also return instances of Channel class, which can be used in our UI components.How do we handle migration?
Well in short we don't, instead we simply drop and recreate all the tables if there are any changes to schema. We simply need to manually bump
dbVersiononQuickSqliteClienteverytime we make any changes to schema. This version number will be compared againstuser_versionpragma to decide weather to recreate tables or not. You can find the logic hereChanges that customers will need on App level
react-native-quick-sqlitepackageawait client.connectUser(user, token)to succeed before rendering any Chat components. But now after this release, they will need to only callclient.connectUser(user, token)but NOT wait for it to succeed, before rendering Chat component. But they must call connectUser before rendering components.QuickSqliteClient.resetDatabase()after/beforeclient.disconnectUser()call.Chatcomponent in codebase.enableOfflineSupportprop on Chat componentTasks