Thoughts on the refactor

[carlbarrdahl]

Hi!
I wanted to write down some thoughts on the upcoming refactor and the reasoning behind the recommendations and suggestions I put forth. I'm adamant about these because I've seen many projects struggle with their code-bases collapse under complexity.

Use an existing component library

Creating component libraries is difficult and complex. One little change in a component could mean having to go over the whole site to change where the component is being used. It also means writing and maintaining documentation and tests.

By using an existing component library, focus can be put on creating actual value. Existing component libraries are also well-documented which makes it easier to onboard new developers. Many developers are also used to the conventions found in them in how to use design tokens.

Consistency in design is easier to achieve by using design tokens (font sizes, colors, spacing, etc). Less clutter in folders due to fewer files.

A DX/components library can be created which wraps for example ChakraUI with different theme files for the different products. Dark/Light mode is also implemented and easy to configure.

Many component libraries have excellent utility functions which makes making minor adjustments to components in a consistent way easy. For example: <Text fontSize="sm" marginRight={2} /> changes cont size and adds margin to the right side of the element from a spacing configuration in the theme file. This removes the need to create all those styled wrappers with one or two lines of adjustments.

From my experience, component libraries that are being developed parallel to the app struggle in both. The component library is constantly tweaked to work with the app and the app has to be updated for all these changes. The documentation is rarely up-to-date and the developers get frustrated. The component libraries are useful and powerful because of their stable api.

All that time that goes into writing the components, testing them and documenting them is better spent in working on features.

Simplicity is compounding.

ReactQuery for async data fetching, states and caching

Describe a consistent way of handling async requests. React-Query solves caching and automatically batches requests. It can be used for all async such as http, graphql, smart contracts, etc.

It makes triggering contracts easy and composable. Could look something like this:

function useProposalVote(proposalId) {
  const { data: voter } = useAccount()
  const contract = useContract(contractTypes.proposal)
  return useMutation([proposalId, "vote-action"], async ({ amount, vote }) => {
    return contract.vote(proposalId, vote, amount, voter)
  })
}

This hook will return an data, error, loading and the function to trigger the call. It makes the code-base consistent in how the developer can expect the returned object to look like.

Data is cached pointing to a key and can easily be invalidated to trigger a refetch. A cache plugin can be added or written to store in localStorage.

For example a some data is immutable and only needs to be fetched once. The plugin would cache this in localStorage or IndexDB automatically.

function useBlock(number) {
  const provider = useProvider()
  return useQuery(["block", number], () => provider.getBlock(number))
}
function useIpfsHash(hash) {
  const ipfs = useIpfs()
  return useQuery(["ipfs", hash], () => ipfs.get(hash))
}

To do this in SWR takes a lot of boilerplate. If someone has a good example please share!

React-Hook-Forms for forms

Again, well-documented library to point new developers to. It is performant and supports schemas such as joi and zod.

It's excellent at keeping state and handling dynamic forms where new fields are added and removed. Example with schema and and fields below:

const schema = z.object({
  description: z.string().min(1, { message: "Required" }),
  actions: z.array(
    z.object({ contract_name: z.string() })
  ),
});

function ProposalCreateForm({}) {
  // Setup form and schema validation
  const { control, register, handleSubmit, formState } = useForm({
    defaultValues: defaultFormValues,
    resolver: zodResolver(schema)
  });
  // Dynamic array for actions
  const { fields, append, remove } = useFieldArray({ control, name: "actions" });
  
  return (
    ...
      <Button onClick={() => append({})}>Add action</Button>
      {fields.map((field, i) => (
        <>
          <Input {...register(`actions.${i}.contract_name`)} />
          <Button onClick={() => remove(i)}>-</Button>
        </>
      ))}
  )
}

Storybook

Storybook is great for fast development of components in isolation. It makes it easy to mock data and switch between different states the component can be in.

The developer shouldn't have to manually reproduce the different states of the app, some might be more difficult than others, for example error states.

Storybook also acts as a great overview of all the features and components in the site and helps with QA. There's also a huge ecosystem of plugins to integrate testing.

The storybook can in addition to the component primitives have features listed in the sidebar. For example the ProposalCreateWizard component. It would contain the complete flow of that feature and any outbound requests will be mocked.

Testing

React Testing Library + MSW to mock requests and even RPC. This makes it simple to test interactive components and the async hooks can be put closer to the leaf nodes in the component tree. By putting the data closer to the component which actually uses it we can avoid or minimize prop drilling.

Utilities

React-Use hooks for localStorage, global state, clipboard, throttling, lists, queues etc

Folder structure

Pages, layouts, modules, components.

A page is a Route component. It's wrapped by a Layout which contains header, a container for content, a footer and all that is relevant to the page. It usually contains the header with Authentication-related components.

The Page imports the different modules appearing on the page.

A module is a collection of components, hooks and utils related to a feature. It can be a route like Guilds but also Auth. The module should not be dependent on any other module and the folder can easily be lifted out.
It's a piece of feature built from components and hooks. It could be a GuildsList or ProposalCreateWizard. It can be good to use the naming convention {thing}{action}{type} so all the relevant files are grouped.

The components folder contain all the shared components. This is the folder that would be a seperate component library but could start by just wrapping the external library component.

Example structure

/components
  /Button
  /Card
  /FormLabel
  /FormInput
/layouts
  /LandingPageLayout.tsx
  /DefaultLayout.tsx
/modules
  /Auth
    /components
      /Avatar
      /ConnectButton
      /NetworkSelector
      /Web3Provider
    /hooks
      /useAccount
      /useConnect
      /useNetwork
    /utils
      /truncateAddress.ts
  /Guilds
    /components
      /GuildsGrid
      /GuildsGridItem
      /GuildsToolbar
      /ProposalCreateWizard
      /ProposalCreateForm
      /ProposalSelectTypeCard
      /ProposalView
      /ProposalVoteCard
    /hooks
      /useGuildsList.ts
      /useProposalCreate.ts
      /useProposal.ts
    /utils
      /mapGuildData.ts
      /formatRelativeTime.ts

Here's an example of what a component might look like.

function useProposalVote(proposalId) {
  const { data: voter } = useAccount()
  const contract = useContract(contractTypes.proposal)
  return useMutation([proposalId, "vote-action"], async ({ amount, vote }) => {
    return contract.vote(proposalId, vote, amount, voter)
  })
}

function useProposal(id) {
  // Using mutate allow us to lazily call with hash as parameter
  const ipfs = useIpfs()
  const contract = useContract(contractTypes.proposal)
  return useQuery(["proposal", id], async () => {
    const proposal = await contract.getProposal(id)
    const content = await ipfs.get(proposal.contentHash)
    return { proposal, content }
  })
}

function ProposalView({ proposalId }) {
  const { data, error, loading } = useProposal(proposalId)
  const { proposal, content } = data
  return (
    <Layout>
      <Heading fontSize="2xl">{proposal.title}</Heading>
      <Markdown>{content}</Markdown>
      <ProposalVote state={proposal.state} proposalId={proposalId} />
    </Layout>
  )
}

function ProposalVote({ proposalId }) {
  const { register, handleSubmit } = useForm({
    defaultValues: { amount: 0, vote: null }
  })
  const [{ data, error }, vote] = useProposalVote(proposalId)

return (
  <Card as="form onSubmit={handleSubmit(vote)}>
    <Heading>Cast your vote</Heading>
    <Input {..register("amount")} />
    <Radio {..register("vote")}  value={1} />
    <Radio {..register("vote")}  value={2} />
    <Button variant="primary" type="submit">Vote</Button>
  </Card>
}

I built geranimo's design of the Proposal page in a project I've been playing around with:

https://github.com/carlbarrdahl/lens-projects/blob/main/modules/proposals/components/ProposalView.tsx#L27

[rossneilson]

Amazing write up Carl! For those that missed it we had an hour and a bit discussion going through this following a meeting earlier in the day also where we discussed our approach to the the UI refactor.
We are at the point where this is kind of the last point we can effectively change many of our approaches so this is an important discussion to have as we consolidate guild's UI elements and get ready to double down on them in DXvote.

Some quick context:
We have guilds which is a tool but we are also supporting DXvote style architecture (primarily used by us but to be made for others as we upgrade it). These systems work different and DXvote is far more complex and capable but the interfaces between them are really very similar.

Using Chakra UI

The most contentious topic on this list, this was something long-discussed at the start of guild work and again now. The original arguments against were mostly based on my previous experiences using poorly built component libraries that were bloated and large getting more in the way of development than aiding it.
Anyways we now are basically splitting up our components into a stateless component library anyways and so we face the decision of whether to continue with what we have or jump ship to chakra.

The question is really what is more effort for us:
Moving over to Chakra would involve changing all of the existing components we already have and adjusting chakra to fit into our existing design not built around chakra. We would be using this for primitives that chakra gives us but would still have build our own custom components in the structure of our own library due to wanting to reuse those higher-level components. The benefit would be that it is maintained externally and more familiar to new developers just onboarding.

Our own library would be work for us to build, test and document. But it can meet our exact requirements without fiddling and would potentially help enforce those habits and best practices anyways. The hope is that the primitives wont be changing much from this point on also.

I think there is still the potential for us to decide here but sentiment leaned towards our own components.

React-query

SWR has been working well for us so far and I don't think the change to react-query gives enough value.

Fortunately due to its similarity its maybe one of the parts we could conceivably change later down the road. I think the one difference potentially useful was how we can use react-query for more "POST" like requests that we may be doing more of with ceramic and orbit db (although I think we can deal with this)

React hook forms

We already have a PR with this, definitely will be using :P

Storybook

If we are going down the route of our own component library then this is likely worth the effort as the project grows and has plans to further grow. We can implement this as we go through the refactor issues.

Testing

We have examples of good unit tests implemented by Lulli but we never really continued, this will be part of the finished definition for the refactor and enforced beyond. As for more integration like unit tests I think we'll go with proper unit tests for this. We can start more on this now that we have full flows through the UI and the dev script using Cypress @MiltonTulli

Utils

Think we can add these as we go, already have a lot of the ones mentioned.

Structure

We discussed this a lot this morning. Since we have a strange case where we essentially have two applications (guilds and DXvote) but they will both look mostly the same to users interacting with them.
Because of this we need to split components into stateful and stateless to reuse as much as possible. We cant just reuse page layouts for both as this would result in huge prop drilling and messy code. So we have a component folder with all stateless components that accept well typed props that are then used by stateful wrappers that include raw data fetching, processing and eventual passing down to the reused components.

[rossneilson]

Next steps:
I think a good next step for this would be for us all to walk through a mobbing / pair programming session for a group of components to get a grasp on what would be the steps for subsequent issues that we can all work on to quickly deliver this initial refactor to then allow the next steps.
This would include, moving the files in the new structure, separating the logic, defining types, testing and documenting with storybook. We can maybe do this next week.