Lazy "new"

[spoerri]

It would be nice to banish empty commits, and to enable users not to have to care about them in the form of @- vs @. It would also be nice to be a little more opinionated and automatic for "squash workflow" vs "edit workflow". For example, users shouldn't have to care about whether to use new or edit for modifying vs viewing historical commits. #3129 #3660

Therefore, let's make new be lazy in the sense that a commit/change is only created when the working copy is actually modified.

I.e. for:

1. jj new [someExistingRev or not]
2. ...add, modify or delete a file in the working copy...
3. ...any jj command...

This proposal is to create the commit at 3 instead of 1.

Other niggles: would probably also want to go ahead and create the commit for jj describe modifications as well as working copy modifications. Might also want to rename new to something like "get", especially if there's openness to employ commit instead of new in one use case.

(And fun: to one way of looking at it, this would make the old "open" vs "closed" be a property of the working copy! The working copy would open either via edit or on a commit which it just created, and would close with new, commit, or edit of a different commit.)

[ilyagr]

let's make new be lazy in the sense that a commit/change is only created when the working copy is actually modified

I do sometimes want to jj edit a commit, especially if it's a WIP commit with no children. I'm not sure how to fit this in with a "lazy" new.

Also, this was just before my time, but jj did formerly have an open/closed concept, and then it was simplified away. See #321 (comment). I never learned precisely how that worked, but I imagine it was at least somewhat similar to what you are thinking.

[arxanas]

What's the practical ramification of this?

I always use jj new. If I'm investigating a historical commit, I might want to make changes/do builds/etc., but not be sure if I've left changes in the working copy. I might want to run e.g. jj diff --from X --to @- to determine something about the commit I'm investigating. If I've made working copy changes, then I might get silently very-slightly-wrong results with lazy-new. Of course I can run jj status before all commands like that, but the point is to reduce the amount of context I have to check before various operations.

[spoerri]

OK, I hear. Yes, lazy-new would remove the ability to easily and reliably exclude changes that might have crept into the working copy, which is useful for when you're just going to abandon the working copy anyway. Is it just for jj diff?

If that's sufficiently useful, there are ways to support it without such a global impact on the design.

I always use jj new

Yes, the whole motivation for this proposal is that "new" is the recommended mode of operation. If we were happy with edit workflow, we could go do fine with neither "eager" nor "lazy" new; we could go with "explicit only" new.

It does raise another point in the design space: instead of creating an empty commit, we could make the working copy readonly. The only supported ways to make it writable would be jj new and jj edit. Forked a different discussion for that: #4047

[arxanas]

Is it just for jj diff?

In practice, I believe I use it for a variety of commands, like jj rebase, jj split, jj restore, etc., without wanting to check jj status first.

That is, I think there are a lot of workflows that I personally use that depend on @ being "stable" in a certain sense.

If we were happy with edit workflow, we could go do fine with neither "eager" nor "lazy" new; we could go with "explicit only" new.

As far as I can tell, there are in fact two factions of users, where one strongly prefers jj new by default, and one strongly prefers jj edit by default, and neither subsumes the other.

• In that you couldn't forgo one and express all of your typical operations with it using the other one.
• I don't think lazy-new subsumes either one, either?

It does raise another point in the design space: instead of creating an empty commit, we could make the working copy readonly. The only supported ways to make it writable would be jj new and jj edit.

In a sense, this is kind of like preserving a staging area idea, and not really committing the working copy changes to the commit graph until an explicit "commit" operation of some sort.

While I think this idea would still be executed better in jj than in Git, since we would still treat the working copy as a normal kind of commit and use all the same tools to operate on it, we really want to reduce the number of logical states and transitions that the user has to think about.

• What's wrong with Git? A conceptual design analysis (2013) gives some of the examples and arguments towards reducing the number of states/transitions.
  • It doesn't include extra other state like ongoing rebases/merges in the figures.

If that's sufficiently useful, there are ways to support it without such a global impact on the design.

My feeling so far is that the jj-new model is indeed a local maximum in the design space, but I don't think I draw the same conclusions about how to evolve the design as you?

• It's not necessarily ideal, and one has to think about @ vs @- maybe more often than one would like, but I think limiting the number of design concepts here has been worth the reduction in complexity.
• My intuition is that an approach to change the UI around the existing jj concepts would be more successful than literally changing the jj semantics.
  • For example, the suggestion of adding a specific symbol to indicate @ has come up several times. It adds no new expressivity to jj itself, but maybe it turns out to be a significant UI improvement in practice?
• But I don't see that lazy-new or default-read-only addresses the design problems in the right way.

---

If we change the jj design, since I think the current design occupies a local maximum, I suspect that we would need to do a less incremental step in the design space to arrive at a better solution.

• My own proposal was hierarchical commits,
  • which generalizes the "open/closed" concept from $n \in {0, 1}$ to $n \in \mathbb{Z^+}$.
• While we may never implement hierarchical commits, the interesting thing would be tackling the problem of grouping commits arbitrarily.
  • Under hierarchical commits, we would always create new working copy commits, even when "editing" historical commits, and the problem becomes grouping them meaningfully instead.
  • But the same grouping problems exist today. For example, branches, topics, and stacks are all solutions that aim at the problem of grouping commits meaningfully.

What we can take away is the UI problem of whether the resulting working-copy commit is grouped with the parent commit or not.

• This problem manifests itself in both
  • how we physically render it in the commit graph — quite important when it comes to onboarding users and building their intuition,
  • and also how we address the commit versus its hierarchical ancestors programmatically.
    • Currently @ is the only such option.
    • There is not currently a way to delineate/separate the working-copy commit from its parent commit in a semantically-meaningful way in the revset language, or, really, the jj design itself. It's all just convention, for which we generally want to use @- to address the parent commit, but not always.
• [Tangential] To express grouping in the hierarchical commits model,
  • we would require an operation in terms of some $n$ expressing hierarchy:
    • it would be some kind of term meaning "the chunk of this commit graph that spans from the previous 'fork point' to the next 'merge point'",
    • and the ability to express " $n$ = how many levels of hierarchy to traverse upward when finding the desired fork- and merge-points".
  • jj new X would be like adding a new commit after X inside its parent commit, and addressing that new commit with $n = 0$.
  • jj edit X would be like adding a new commit inside X, but continuing to address X by using $n = 1$.
    • Realistically, I think users of an "edit workflow" might prefer to use $n = 0$ in many cases.
  • jj lazy-new X would be something like adding the new commit inside X (like jj edit), but operations use $n = 0$ in the specific case when jj wants to "instantiate" the lazy commit with the working-copy changes, or $n = 1$ otherwise.
    • Critically, the user could still choose to explicitly address the working-copy commit with $n = 0$ or parent commit with $n = 1$ if they so desired, which would support my workflows mentioned previously.
    • The design complexity is now addressing commits, not creating them. The commit graph is now "stable" in the sense I alluded to.
  • Branching becomes a matter of groups with $n > 1$.

To summarize:

• If we're going to change something, then we may prefer to change the entire way that jj organizes/groups commits, rather than handle just the case of the working-copy commit differently.
• The entire workflow may well be best addressed a UI problem, not a fundamental design problem.

[arxanas]

Another idea is to have some kind of symbol that means "the working-copy commit if non-empty, or the parent commit if empty". Then people who use the "lazy-new" workflow can enjoy that today. Actually, I bet we could express that right now with a revset alias? (We just wouldn't have a cool symbol for it.) Then the main problem is fixing most commands to operate on that (or some other user-definable revset that means "the thing I'm working on"). @spoerri I'm curious if that addresses your workflow issues?

[spoerri]

symbol that means "the working-copy commit if non-empty, or the parent commit if empty" [...] addresses your workflow issues?

maybe... i was really hoping to reduce complexity, so adding features without replacing anything is a risky way to achieve that.

you probably saw in the other discussion that i'm settling on jj commit as the problem.

i've been conflating "central" and "leaf" empty commits. i'm coming around to the idea that leaf empty commits don't really have much conceptual impact. OK, it's true that users can't simply do jj checkout somHistoricalRev; jj branch create X but jj checkout somHistoricalRev; jj branch create X -r somHistoricalRev isn't so bad if you don't want to think about @ and @-; you have to know which revision you checked out anyway. And once you switch to another commit, those leaves are fleeting, so to say.

i'm more concerned about standardizing on "finished" commits being only those with child commits. assuming the advance branches stuff goes the right away, maybe all i want is a calling convention for jj git push that requires neither running jj commit nor specifying weird parameters.

[joyously]

This proposal is to create the commit at 3 instead of 1.

Despite you suggesting to rename new, I think the command should actually do its job when invoked. No one can predict the order or timing of commands to come. The next command might be from a different user, months after, or a tool, milliseconds after. Making a new commit is the expected result, or the command would not be used. It should be consistent, or it muddies up the mental model.
Is there some penalty for an empty commit?

[spoerri]

IMHO not aggressively creating commits is the correct mental model! I didn’t lead with that, lest I cause a tumult.

From a practical perspective, empty commits are not the end of the world. So far I’ve just seen trivial issues around @- and the correct semantics for jj next. To be clear, I don’t mean literally jj new but any command that creates a new commit in advance. jj commit is the most egregious. This is the sort of thing that increasingly many things come to rely on.

It’s more important to avoid a design that relies on empty commits because they are a smell in the model. We employ empty commits now as a replacement for “the parent commit is closed”, but the real problem with open/closed is that it doesn’t belong in the commits at all. It belongs in the working copy.

Multiple working copies ala workspaces might demonstrate: you can edit a commit in one workspace and view it in another. So having an empty commit as its child doesn’t accurately represent reality.

Or maybe we don’t even need to look at such an esoteric case; it’s just wrong to have users create new commits in order to view historical revisions. (Of course it’s also unfortunate to use jj edit for that.) IMHO jujutsu has great potential to “scale down” and be simple for newbies. So it’d be lovely to be able tell users “you can open up any revision with: jj get, and if you make any changes jujutsu will automatically and safely save them.” It’s hard to promote jujutsu as a simplified git if some of the most basic use cases require explaining revsets and squash vs edit workflows. Meet the new revision control same as the old revision control. Anyway, sorry to rant 😄

[joyously]

avoid a design that relies on empty commits because they are a smell in the model. We employ empty commits now as a replacement for “the parent commit is closed”, but the real problem with open/closed is that it doesn’t belong in the commits at all. It belongs in the working copy.

I don't think you have described the entire model. In my understanding, no commit is "closed", while the working copy as commit simplifies the interface and reduces possibility of lost work.

you can edit a commit in one workspace and view it in another.

If it's two workspaces, it's not the same commit? I can easily see how you can be editing a commit and view it in another window (no extra workspace needed). That's always the nature of multitasking. I just don't see what the contents have to do with it.

it’s just wrong to have users create new commits in order to view historical revisions.

I agree. I don't think they have to, however. There are other commands besides new.

So it’d be lovely to be able tell users “you can open up any revision with: jj get, and if you make any changes jujutsu will automatically and safely save them.”

I think this would be a big problem. I wrote up a similar sounding issue while I was testing out gg and it seemed very fragile if a novice could be browsing the repo, leaving a trail of changes from editor autosaves.

It’s hard to promote jujutsu as a simplified git if some of the most basic use cases require explaining revsets and squash vs edit workflows.

It's not a simplified git. The most basic use case doesn't need revsets or squash or edit either. Most basic is change files, then jj new. You don't even have to describe it. You can always simply march forward and ignore most of the commands.

[scott2000]

I personally prefer the current model because it feels very consistent since @ is always present and always refers to the working copy (meaning that it's empty if there are no changes made yet). I also wouldn't like the commit graph to change due to filesystem changes.

Maybe some of these issues could be resolved more easily by just adding a new revset called # which just means @-? So the "checked-out" commit would just be # always, and the working-copy commit would remain as @, and you could just configure jj log to hide @ if there's no changes so that you'd only see #. This has the benefit that all of the commands would still work the same as they do now (e.g. jj describe could still mean jj describe @).

So basically when you check out a commit, it would show that commit as # in the log (and the empty @ would be hidden). Then if you make changes, now @ would show up on top of #. But behind the scenes, the working copy would still always be accessible using @ regardless of if it's visible in the log.

[spoerri]

Can we look with fresh eyes?

Here's two versions of a jj intro doc, one for "eager new" and the other for "lazy new". I tried to make them both appropriately simple and pithy, and I'd be happy to improve either. Obviously it'd be silly to update them for partisan reasons 😄

eager new	lazy new
jj automatically updates the current "change" object to correspond to the "working copy".	jj automatically updates the current "change" object to correspond to the "working copy". jj automatically creates a new change when you modify the working copy, if the working copy was already associated with a historical change. This prevents you from accidentally editing a historical change.
Use the "new" command to retrieve a historical change into the working copy, or if you'd like to separate the changes you're about to make from the ones you just made. Use "commit" to tell jj that you're finished working on a change. Use "next" and "prev" to retrieve historical changes relative to the change currently associated with the working copy.	Use the "get" command to retrieve a historical change into the working copy. Use "new" if you'd like to separate the changes you're about to make from the ones you just made. This will also happen automatically any time you expose the current change, e.g. via git push or branch create. IMHO we wouldn't even bother having aliases for "jj get @+" and "jj get @-".
All of these commands create a new change and associate it with the working copy. Any command which abandons a change also creates a new change instead of deleting it. The "rebase" command is one example. This prevents you from accidentally editing a finished change. Therefore, if you're working with a finished change in your working copy, you should specify "-r @-" to commands that otherwise default to "-r @".
When you want to push your work to github, use "jj git push -c @-" (Make sure that you have run jj new or jj commit once after you've made your changes.)	When you want to push your work to github, use "jj push -c"
For the case in which you really do want to edit a change directly, use the "edit" command.	For the case in which you really do want to edit a historical change directly, use the "edit" command.

You'll notice that the line talking about lazy goes right after the line talking about connecting change and working copy, implying that they are similar features.

Also notice that the doc for lazy doesn't need any concept of a "finished" change. Unlike in the discussion above it keeps the "new" command and gets rid of the "commit" command.

IMHO this simplicity of intro doc corresponds to a simplicity of underlying model.