Configuring projects/entrypoints

[g-elwell]

Hey @ampersarnie

I see you've posed a question about whether or not to use a config file before and this kinda aligns with something that's been on my mind as I've been using this package.

Instead of looking inside a predefined group of directories for projects with package.json files, would it be possible to enable a way to define your project paths upfront using either a build-tools.json or an entry into your main package.json?

I'm imagining something like this:

projects: [
  'plugins/my-plugin',
  'mu-plugins/my-other-plugin',
  'themes/my-theme',
  'themes/my-theme/blocks/my-block'
]

You'd still require a package.json and src/entrypoints inside each project root and the absence of a defined set of projects would mean the build tools assume you're working on a standalone project and look for a src/entrypoints in your root directory.

Although this is extra config, I think some explicitness here would provide more clarify on what the build tools are doing within a project and might also also solve #17 re: building block assets in a way that matches the direction WordPress is moving towards.

[ampersarnie]

@g-elwell This very much falls in line with the config and targetDirs example in #1. What you've got here is almost an extension of that so I do think this sort of config would be beneficial and help bring more flexibility to our work. Happy to put this in a backlog but would end up being part of a larger config approach using something like Cosmiconfig (I'm not so sure of having the config in package.json but those are different details).

[g-elwell]

@ampersarnie this could be seen as an extension of targetDirs but the distinction I'd make is to be explicit about the exact projects you want to build instead of which directories you want to include (assuming targetDirs would be based on the way the build tools currently work which is to find all projects residing within a set group of directories).

[ampersarnie]

@g-elwell Oh absolutely, they are different but I mean extension in the terms of directory and project management for the build tools. They'd likely have to live under different properties, etc.

One thing we'd need to figure out is priority orders, etc as we might hit some issues or expectations when working with a new property for managing these and/or general directories with targetDir.

Question for you, is your main motivation for this to tackle #17?

[g-elwell]

@ampersarnie my thinking is more along the lines that this property would replace the current method of finding projects within specific directories, as well as the different behaviour depending on whether or not you're running from inside wp-content.

I think that being un-opinionated about directory structure and explicit about projects would make it easier to understand what's going on (especially for new users) on top of providing extra flexibility over what you want to build.

This would enable #17 but my motivation for bringing it up is more based around my experience getting up and running with the build tools and helping others understand them, especially when building multiple projects from a wp-content folder. I'm curious whether this approach could replace a bunch of logic in the build process and simplify some things on that side too whilst keeping the same functionality for users

[ampersarnie]

@ampersarnie my thinking is more along the lines that this property would replace the current method of finding projects within specific directories, as well as the different behaviour depending on whether or not you're running from inside wp-content.

I think that being un-opinionated about directory structure and explicit about projects would make it easier to understand what's going on (especially for new users) on top of providing extra flexibility over what you want to build.

This would enable #17 but my motivation for bringing it up is more based around my experience getting up and running with the build tools and helping others understand them, especially when building multiple projects from a wp-content folder. I'm curious whether this approach could replace a bunch of logic in the build process and simplify some things on that side too whilst keeping the same functionality for users

Then this brings in a whole different angle and set of ideas to what you're proposing. The build-tools wasn't trying to be un-opinionated for a few reasons, and was actually trying to be the opposite. Previously we've had many issues where numerous projects and sites have had different structures, tooling, etc that can and have brought confusion when going back or onboarding someone. I don't want to have that happen again and it seems to me what you're describing here will risk that.

As this isn't built to be only run from wp-content, we're still going to need to logic to be able to check where we're running this from and to check the directory structures. So I don't think this will simplify any of the code as all we'll be doing is removing the directory look-up to a property look up. Even then there are elements of that which we'll need for your suggestion and there's no reason why we couldn't, or shouldn't have a fallback to cater for that project lookup. I don't believe a config should be required and more compliment or customise where it is required, so getting rid of one system to have something partially config driven isn't something I want do here.

Your proposal only really tackles the Site Wide setup (which is what I assumed you were discussing), and doesn't take into account managing Individual Projects from both root and the project itself. These are also situations that we need to account for any means that the logic and process we're using isn't likely to go away just yet.

My proposal is this (to follow yours); we implement a config that can include a project listing that will be used if it (the config) does exist as you opened with. However, if it does not exist it will fallback to the current automated method of looking for projects to build. We may also add in additional automated lookups to search deeper than themes/theme to also allow for similar to your example of themes/my-theme/blocks/my-block. The build priority order would then be;

1. Defined projects (e.g. build-tools build my-plugin,my-theme)
2. Config if it exists (e.g. build-tools build).
3. Automated check for wp-content and seek for entrypoints. (e.g. no config build-tools build)
4. Automated check for entrypoints. (e.g. build-tools build from wp-content/plugins/my-plugin)

This would still allow for the flexibility but also allow us to have a drop-in build-system via npx that doesn't require any config if needed.

[g-elwell]

I'm only suggesting to be un-opinionated about where the projects are located specifically, you'd still need to follow the required conventions around directory structure inside each project so I don't think this opens the door for projects to deviate from one another in any significant way (correct me if I'm wrong here).

This should also work with individual projects as well as a site-wide set up. If you only install the build tools and provide no path-list/config, then it should treat your root folder as an individual project and look for src/entrypoints within it.

If you do provide a config (in whichever form this ends up being) then the list of paths you provide will be parsed to look for a package.json and src/entrypoints inside of each directory.

So the process would become:

1. Defined projects using the full path (e.g. build-tools build plugins/my-plugin,themes/my-theme or build-tools build with a config file present)
2. No defined projects, seek for entrypoints (e.g. build-tools build)

No need to check whether you're inside wp-content or specifically traverse a predefined list of directories.

Is that any different to what you thought I was suggesting?

[ampersarnie]

It opens up potential for people to assign random directories for something that's not a block, etc.

That doesn't really change much of how I read it. This change still seems like it would be unnecessarily forcing people to define and maintain a projects config when there is no need to. We can already find those projects without the need to do that. What it does offer, and what I was interested in, is the ability to expand the structure to have and control those blocks within a theme or plugin, treating the current processes as default behaviour.

I don't think dropping the current lookups for another point of maintenance is the right way to go. If anything it should be optional if someone chooses to use it.

I'm not sure why you want to drop that feature when we can look at implementing some of your suggestion to attain both? or update the current to look for block paths too?

[g-elwell]

I'm not sure how we'd implement only some of this - incorporating an optional config would still open the door to assign any directory as a project. Having this work alongside the current lookup process would also mean you'd have both explicit and implicit sets of projects to consider inside a repo and (as you said) also adds some complexity around what takes priority.

This is why my mind is leaning towards using a single source of truth for projects in the form of a required project-path config. Yeah the list would need to be maintained, but it would also make it easier to understand what's going on inside a repo when you run the build tools. You still keep the rest of the abstractions such as shared configs, strict (internal) project structure, unified commands to watch/build, etc.

I also completely understand if the door should remain fully closed around assigning any directory as a project. In this case I think that creating some conventions and looking for block paths automatically would be another way to achieve #17 and remain more in line with how the build tools operate currently. The struggles I've encountered around getting set up with the build tools and learning their conventions could also be solved another way via updated documentation, additional error messages, etc.