feat: Persistent cache #3721

AmarTrebinjac · 2024-10-24T22:01:18Z

Changes

Adds capability of persisting queries in localStorage using persistQueryClient.
This PR also includes moving the Squads from Boot to the persisted cache in local storage

Events

Did you introduce any new tracking events?

Experiment

Did you introduce any new experiments?

Manual Testing

Caution

Please make sure existing components are not breaking/affected by this PR

WT-2201
WT-2231

Preview domain

https://wt-2201.preview.app.daily.dev

working persistent cache

vercel · 2024-10-24T22:01:22Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated (UTC)
daily-webapp	✅ Ready (Inspect)	Visit Preview	Oct 25, 2024 11:40am

1 Skipped Deployment

Name	Status	Preview	Updated (UTC)
storybook	⬜️ Ignored (Inspect)		Oct 25, 2024 11:40am

AmarTrebinjac · 2024-10-25T11:45:24Z

packages/shared/src/lib/persistedQuery.ts

+});
+
+export const queryClientPersister = createSyncStoragePersister({
+  storage: typeof window !== 'undefined' ? window.localStorage : undefined,


Recommended way by TKDodo (core maintainer) to handle window being undefined in SSR

We actually have globalThis?.window you probably want to use that one.

AmarTrebinjac · 2024-10-25T11:46:24Z

packages/shared/src/lib/persistedQuery.ts

+  persister: queryClientPersister,
+  dehydrateOptions: {
+    shouldDehydrateQuery: ({ queryKey }) => {
+      return persistedKeys.includes(queryKey[0] as RequestKey);


shouldDehydrateQuery gets called for each query. Keys that match persistedKeys list will be persisted in localStorage

So adding more we should just use useQuery on the normal client and by this it will define which keys are persisted?
That's pretty neat actually.

AmarTrebinjac · 2024-10-25T11:47:57Z

packages/shared/src/components/sidebar/Section.tsx

 }

+const ItemsSkeleton = () => {


Created a quick loading animation for the squad section in the sidebar, as squads may not always be loaded at render time

It's probably good to check how many squads the average normal user has to avoid shifts for the mass

I did a query on db, looks like only ~10.000 has more then 5, so probably not bad default

found a better query, i think

select count(*) from public.user u where (select count(1) from source_member sm where sm."userId" = u.id) > 5

9,009

and more then 1 is 55,435

so probably 1 is the best 😆

rebelchris

Wow nice work!
Got some questions/remarks, but overall super neat work

rebelchris · 2024-10-25T11:59:07Z

packages/extension/package.json

@@ -20,6 +20,7 @@
    "@dailydotdev/shared": "*",
    "@tanstack/react-query": "^4.36.1",
    "@tanstack/react-query-devtools": "^4.35.3",
+    "@tanstack/react-query-persist-client": "^4.36.1",


It will conflict with: #3718
Just to note we might have to choose how to release

yeah, also looking at your comments in that PR, I will have to change some syntax here too when it comes to queryKeys and queryFns

rebelchris · 2024-10-25T12:00:05Z

packages/shared/src/components/sidebar/Section.tsx

 }

+const ItemsSkeleton = () => {


It's probably good to check how many squads the average normal user has to avoid shifts for the mass

rebelchris · 2024-10-25T12:02:20Z

packages/shared/src/hooks/squads/useSquad.ts

+};
+
+export const useSquads = (): UseSquads => {
+  const { isFetched: isBootFetched, user } = useContext(AuthContext);


Why do we care if boot is fetched?
Basically if you have a user you can go right?

rebelchris · 2024-10-25T12:03:40Z

packages/shared/src/hooks/useLeaveSquad.ts

-  const { deleteSquad: deleteCachedSquad } = useBoot();
+  const { mutate } = useMutation(leaveSquad, {
+    onSuccess: () => {
+      persistedQueryClient.invalidateQueries({ queryKey: [RequestKey.Squads] });


Can we also mutate the persistent client instead of just invalidating it?

Hm, not quite sure I understand the question. 😅

You mean use the setQueryData function to update the cache instead of refetching?

rebelchris · 2024-10-25T12:04:29Z

packages/shared/src/lib/persistedQuery.ts

+});
+
+export const queryClientPersister = createSyncStoragePersister({
+  storage: typeof window !== 'undefined' ? window.localStorage : undefined,


We actually have globalThis?.window you probably want to use that one.

rebelchris · 2024-10-25T12:05:00Z

packages/shared/src/lib/persistedQuery.ts

+  persister: queryClientPersister,
+  dehydrateOptions: {
+    shouldDehydrateQuery: ({ queryKey }) => {
+      return persistedKeys.includes(queryKey[0] as RequestKey);


So adding more we should just use useQuery on the normal client and by this it will define which keys are persisted?
That's pretty neat actually.

rebelchris · 2024-10-25T12:06:11Z

packages/webapp/pages/_app.tsx

+      <PersistQueryClientProvider
+        client={persistedQueryClient}
+        persistOptions={persistedQueryClientOptions}
+      >


This will then underlying resolve the regular one as well?

The persistedQueryClient is just the regular queryClient with some additional options, and the provider has some additional utilities, so everything that is not related to persisting should be working as it always did

capJavert · 2024-10-25T15:00:17Z

package-lock.json

@@ -0,0 +1,38 @@
+{
+  "name": "daily-apps",


we use pnpm, you probably accidentally run npm install, so this file should be removed

capJavert

Looks pretty good from quick test 👏 , will look again after react-query is merged!

capJavert · 2024-10-25T15:18:46Z

packages/shared/src/hooks/squads/useSquad.ts

+  const { data: squads, isLoading } = useQuery(
+    [RequestKey.Squads],
+    () => getSquads(user?.id),
+    {
+      enabled: isBootFetched && !!user,
+    },
+  );


One thing I noticed is that if I create a new squad on another device, or another tab not sharing local storage the cache will never be updated, probably only after stale time goes by. It should always invalidate in the bg I think.

sshanzel · 2024-10-29T15:55:59Z

Do we have the information on how much time the cache is stored and how we clear the cache for when the user changes account (probably doesn't happen often, just something to consider if easy) as it might blow up the storage (for us at least)?

Maybe let's add some general details about this feature/references we can read to help the team understand how to use it properly.

sshanzel · 2024-11-25T13:32:13Z

As we have moved to v5, and this is still referencing the v4, it might be good for us to park this in the meantime as there can be additional work with the dependencies plus the actual feature. Thoughts? cc @dailydotdev/web-team

AmarTrebinjac added 7 commits October 23, 2024 22:30

install packages

3e74d49

working persistent cache

Add loader function to SquadSection

181f08e

Rewrite boot functions to hooks with useQuery

462f1e2

Update formatting

a260088

Merge branch 'main' into introduce-persistent-cache

55240bd

remove comment

5735d04

Add comment

e3f4dce

AmarTrebinjac changed the title ~~Wt 2201~~ feat: Persistent cache Oct 24, 2024

vercel bot deployed to Preview – daily-webapp October 24, 2024 22:03 View deployment

fix type errors

3b6bda0

vercel bot deployed to Preview – daily-webapp October 24, 2024 22:10 View deployment

fix additional type errors

acbf83e

vercel bot deployed to Preview – daily-webapp October 24, 2024 22:14 View deployment

AmarTrebinjac added 2 commits October 25, 2024 01:14

Set mutation cache

c3c3882

Update companion app

6db2343

vercel bot deployed to Preview – daily-webapp October 24, 2024 23:18 View deployment

Update squad edit page

77eef05

vercel bot deployed to Preview – daily-webapp October 25, 2024 09:07 View deployment

Sort squads by name

889f9b1

vercel bot deployed to Preview – daily-webapp October 25, 2024 09:20 View deployment

Added default retry properties

b4a773e

vercel bot deployed to Preview – daily-webapp October 25, 2024 10:41 View deployment

Update squad join

fb5b149

vercel bot deployed to Preview – daily-webapp October 25, 2024 11:03 View deployment

Add missing dependency

80ad810

vercel bot deployed to Preview – daily-webapp October 25, 2024 11:06 View deployment

Linting fixes

3f5be5f

vercel bot deployed to Preview – daily-webapp October 25, 2024 11:25 View deployment

Remove unneeded changes to companion app

32cf049

vercel bot deployed to Preview – daily-webapp October 25, 2024 11:40 View deployment

AmarTrebinjac commented Oct 25, 2024

View reviewed changes

AmarTrebinjac marked this pull request as ready for review October 25, 2024 11:55

AmarTrebinjac requested a review from a team as a code owner October 25, 2024 11:55

AmarTrebinjac requested review from rebelchris, omBratteng, ilasw, capJavert, sshanzel and nensidosari October 25, 2024 11:55

rebelchris requested changes Oct 25, 2024

View reviewed changes

capJavert reviewed Oct 25, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Persistent cache #3721

feat: Persistent cache #3721

AmarTrebinjac commented Oct 24, 2024 •

edited

Loading

vercel bot commented Oct 24, 2024 •

edited

Loading

AmarTrebinjac Oct 25, 2024

rebelchris Oct 25, 2024

AmarTrebinjac Oct 25, 2024

rebelchris Oct 25, 2024

AmarTrebinjac Oct 25, 2024

rebelchris Oct 25, 2024

capJavert Oct 25, 2024

capJavert Oct 25, 2024

capJavert Oct 25, 2024

rebelchris left a comment

rebelchris Oct 25, 2024

AmarTrebinjac Oct 25, 2024

rebelchris Oct 25, 2024

rebelchris Oct 25, 2024

rebelchris Oct 25, 2024

AmarTrebinjac Oct 25, 2024

rebelchris Oct 25, 2024

rebelchris Oct 25, 2024

rebelchris Oct 25, 2024

AmarTrebinjac Oct 25, 2024

capJavert Oct 25, 2024

capJavert left a comment

capJavert Oct 25, 2024

sshanzel commented Oct 29, 2024

sshanzel commented Nov 25, 2024

feat: Persistent cache #3721

Are you sure you want to change the base?

feat: Persistent cache #3721

Conversation

AmarTrebinjac commented Oct 24, 2024 • edited Loading

Changes

Events

Experiment

Manual Testing

Preview domain

vercel bot commented Oct 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rebelchris left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

capJavert left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sshanzel commented Oct 29, 2024

sshanzel commented Nov 25, 2024

AmarTrebinjac commented Oct 24, 2024 •

edited

Loading

vercel bot commented Oct 24, 2024 •

edited

Loading