The Ring Buffer Workflow

Some people that I collaborate with find it hard to follow how I work and sometimes are throwing words at me because they don't get the benefits of working in such a way. I take this as an opportunity to write down how it works and maybe convince the one or the other developer of the advantages:

So, I call this ring-buffer workflow. That is, because I am basically seeing my local work as a multidimensional ring buffer with a funnel at the input side.

Every change enters at the right side (the funnel). It possibly moves around in the buffer for quiet some time - and finally comes out linearly at the left side.

I said, multi-dimensional, because there are different ways to look at the ring buffer:

1. Stability

   On a large scale, the buffer is separated into three areas: stage phase, dev phase and experimental.

   These are ordered like that from left to right.

2. Logical change sets

   A feature or bug fix that I deem to be able to stand on its own. This is not necessarily an atomic change.

   Each logical change set will become a branch (And when I'm on GitHub, a pull request).

   Usually, I do have a local source tree for every branch I'm actively working on. However, I am working in none of those. This is because I have an additional source tree, which represents the funnel on the right side of the buffer.

3. Individual changes

   These are commits actually.

   I try to create commits as atomic as possible. Sometimes my fingers are too fast and larger things escape into a single commit. I don't get hard feelings when that happens, but I try to avoid it.

   But this also means that I have usually up to 10 active (uncommited) changes in my funnel working tree. I massively use git add -p to get them in.

The most stable changes in the ring buffer are located at the left side, which is usually based onto the development branch. Going from here to the right, every logical change set is based upon the previous one. Logical change sets may change their position inside the buffer. This is especially the case, when I encounter a missing feature in some work that is still on the right side - but that feature is actually unrelated (I.e. I have to change a build system tool while developing a new feature).

Only the most left change can ever be merged upwards to the development branch.

I have two pointers into the middle of the buffer. These pointers separate the stability of the buffer. The first (leftmost) one, "stage pointer", points to the end of the stage phase. Everything left of that pointer is just there for peer-review or for my further testing. The second one, "dev pointer", points at the end of the dev phase. Everything between the two pointers is a logical change under active development. Everything right of the dev pointer is experimental work that I might move farther left or just drop completely.

I try hard to have at most one branch between the dev pointer and the funnel. But it's unavoidable at times to have more there. However, I'm totally happy if the dev pointer points directly to the funnel.

As all changes that I commit land in the funnel first. I have to regularly sort them into the right branches. There are different ways to accomplish this. The easiest one is to go to the individual branches and git fetch from the funnel.

When graduating commits to more left positioned change sets or even across one of the pointers, I can make a build of the working tree that contains the buffer just up to that change. But mostly, these source trees just aid me in rebasing the change sets (cd to a different directory and finding what-is-this-merged-upon is far easier than keeping track of all that manually).

The PRs I regularly publish are an image of that ring buffer, usually excluding the funnel. But in a submodules based project like MacGitver, I just use the right most branch as funnel. It's just a convenience, really.

The attentive reader may have noticed, that this is a micro version of how most open source projects do releases of final products. Yes, that is exactly what it is meant to be like - with all the advantages this brings: Additional local testing; Modifiable features (even without the great workflows github gives); Bugfixing before the change set hits anybody else. But most of all, it doesn't disrupt anybody else's work.

Note: If you're an agile developer, you will find this workflow impossible to adhere to. That is because agile development is about fast-integration. This is about quality, a concept that is absolutely orthogonal to agile development (in my book)...

Sascha

PS: I hope to get the time someday to illustrate this with some pictures.