Better use of GitHub

[TiagoCavalcante]

@rocky @mmatera we currently have many branches and PRs, I'm afraid that some improvements get forgotten or that new contributors think that it's hard to contribute to Mathics.

I also think that we could make better use of some GitHub features, mainly the following:

1. Replace current PRs' merge strategy with rebase or squash (IMHO I think that rebase would fit better with Mathics), this would reduce the number of merge commits
2. Require linear history for PRs, this would also make the Git history easier to follow
3. Use automatic relases (by the way, it has been a long time since last release)
4. Mark archived repositories as archived (mathics/Mathics has not seen any update by a long time)

And one extra thing: rename master to main

[rocky]

Replace current PRs' merge strategy with rebase or squash (IMHO I think that rebase would fit better with Mathics), this would reduce the number of merge commits

I agree that some of the merges to master are sloppy. The problem I see about a blanket rule here is whether to use squash or rebase depends on:

1. The committer's workstyle
2. The non-committer developer's habits desire/need to dig down into why the code is the way it is.

I tend to put a lot of care into the PR, and I hope you won't find a lot of garbage commits. My style tests to use a lot of git commit --amend and git push --force to a branch. That style is accommodates a developer who what's to see in great detail what is going on. For a developer that doesn't use that, squash may reduce volume, but only a little...

On the other hand, when one uses merge and doesn't rebase, and each commit has low information content, then squash is great because probably no one cares about the commit that added a print statement and the commit that removed. Or enumerating all of the the failed attempts to fix a problem.

The problem with rebasing which is what I tend to do, is that it can be more work upfront by the person doing the commit. And it might make change the linear github history?...

With respect to 2. I am not totally sure what you mean. I think a downside of the rebase

strategy is that you don't have linear history in github. However in git browsers outside of github there are two dates associated with a commit.

For example:

But please elaborate what you mean here.

3. Use automatic rel[e]ases (by the way, it has been a long time since last release)

The problem we have right now is that the API is changing. I think the next release will be 6.0 because of all of the API changes. (But in my opinion all of these are needed and when Forms and Boxing done is complete, probably there will be more there as well.)

The implication here is that there are times when we have to update some combination of Mathics-core, Mathics-Django, mathicsscript, mathics-scanner, pymathics-graph, pymathics-natlang, and mathics-backend-threejs, and mathics-omnibus in coordination.

(This is a downside of splitting things up into several github repositories. In my opinion this disadvantage is offset by the advantage of everyone not piling on everyone else all of the time for largely independent things. But right now we still have a fair bit of this in mathics-core.)

So in my view were we to have automatic release and were people to use them, we would be inundated with problems regarding the fact that the latest version Mathics-X doesn't work with Mathics-Y and we would have to be more stringent about keeping the versions listed in the setup.py or package.json up to date.

Right now I can't take on any more work towards administration. But if you want to set up something in a limited basis - (are you doing this on mathics-threejs-backend?) then have at it. Or if someone else wants to do this let me know.

I am aware that it has been a long time since release. I would have like to have had a release by now. However things just keep changing too fast. You may have noticed that I am struggling to keep up with the look at the various PRs. In the olden days there were weeks and months where nothing was happening.

Maybe we should have a moratorium on largish changes starting December and spend the entire month of December, getting a release out for the first of next year. (Before a release I would like to do the usual linting and going over standards kinds of things. Currently it is apply() to eval() where appropriate , URL links were we can. We still have probably have mismatched <dd> and dt indentation mismatches and hard line breaks in documentation. Maybe using flake8 and isort over various and other linting.

I haven't been doing that sort of thing because again there is so much other more foundational things going on.

4. This is sort of a political thing. We are not the original maintainers. If you get buy off from them, I am happy to flick the switch.

5. As for master versus main, main is a github kind of things. Is there evidence that this is now current practice? When I created a new repository recently, I was informed by github that "trunk" (as in the SVN name for master) is also possible.

[TiagoCavalcante]

But please elaborate what you mean here.

Basically this would require PRs to don't contain merge commits

So in my view were we to have automatic release and were people to use them, we would be inundated with problems regarding the fact that the latest version Mathics-X doesn't work with Mathics-Y and we would have to be more stringent about keeping the versions listed in the setup.py or package.json up to date.

Automatic releases only write a new page in Releases when a new non-alpha/beta tag is created, so it saves a bit of effort.

Thinking now automatic releases is not a good name, maybe automated releases would be a better name?

Right now I can't take on any more work towards administration. But if you want to set up something in a limited basis - then have at it.

Yes, I'd like to make a PR with these changes.

are you doing this on mathics-threejs-backend?

Yes, I was investigating yesterday how to go a bit further, in mathics-threejs-backend I'd like to have both npm and GitHub releases automated.

Is there evidence that this is now current practice?

I got surprised by the fact that I found no information about this on-line (in thruth I found reasons to not rename master to main).

[rocky]

With respect to requiring a linear history, I am willing to try it. @mmatera - your thoughts and comments?

Automatic releases only write a new page in Releases when a new non-alpha/beta tag is created, so it saves a bit of effort.

Right now, that is not the part that requires a lot of effort. (Testing the release an going over the release notes to make things consistent, concise and more readable is. In other words, quaility over quantity. And even here I have messed up on releases.)

Right now, releases have information from CHANGES.rst copied in as markdown, and copies of the wheels, Is there a way to do that? I know that zips are created, but Python package tarballs are a little bit different than zipping everything that is git checked out.

With respect to automatic releases, from my perspective it just exposes the inconsistencies quicker than I have time to generally fix right now. In other words, often I know there is API breakage, but no one is complaining about it so I can let it slip for a little while to get all of the repositories consistent. When the API breaks less often (and right now I don't see that happening on the horizon soon), that this would make more sense.

Finally, I haven't heard a package distro person asking for this.

Remember how gitpod was going to save the day and add new developers? While it may be the case that it helps people, right now there is no evidence that it has added any developers.

Yes, I was investigating yesterday how to go a bit further, in mathics-threejs-backend I'd like to have both npm and GitHub releases automated.

Then please let's wait until you have first done this on mathics-threejs-backend as opposed to still investigating.

Speaking of mathics-threejs-backend and releases:

• What's the status of getting this or Mathics-Django fixed so that they work together with the latest release of mathics-threejs-backend. In my opinion that is important and something you have complete control over.
• How about upgrading to a newer mathics-threejs-backend release like r146 (as opposed to r138 as in the github repository and r135 in mathics django

[TiagoCavalcante]

Right now, releases have information from CHANGES.rst copied in as markdown, and copies of the wheels, Is there a way to do that?

Do you mean, automatically update CHANGES.rst?
if yes, then it is possible, to not pollute CHANGES.rst with all commit messages we could select only those with a fix: , builtin: , ... prefix.

I know that zips are created, but Python package tarballs are a little bit different than zipping everything that is git checked out.

Packaging can also be automated, I could ask to vicp74 how he builds the AUR package to create a GitHub action for that.

With respect to automatic releases, from my perspective it just exposes the inconsistencies quicker than I have time to generally fix right now.

Not sure if I understand what you mean, but I'm not proposing releases that happen every x time, I'm proposising to use the GitHub action named automatic releases, which writes the release based on the commit messages whenever a new tag is pushed.

Then please let's wait until you have first done this on mathics-threejs-backend as opposed to still investigating.

I'll do that today.

What's the status of getting this or Mathics-Django fixed so that they work together with the latest release of mathics-threejs-backend. In my opinion that is important and something you have complete control over.

I'll fix until weekend, I just want to finish the new primitive text before, BTW, could you give me a help there?

How about upgrading to a newer mathics-threejs-backend release like r146 (as opposed to r138 as in the github repository and r135 in mathics django

Same as above.

[rocky]

I'll fix until weekend, I just want to finish the new primitive text before, BTW, could you give me a help there?

I will look at Mathics3/mathics-threejs-backend#109 when I get a chance.

(From Mathics' perspective, getting the current release to run which has been broken for 6 months or so is more pressing than adding new features that Mathics isn't going to be able to use because it is stuck in the past.)

[mmatera]

@rocky @mmatera we currently have many branches and PRs, I'm afraid that some improvements get forgotten or that new contributors think that it's hard to contribute to Mathics.

Indeed, at this point, with the API broken and continually changing, it is hard.

I also think that we could make better use of some GitHub features, mainly the following:

1. Replace current PRs' merge strategy with rebase or squash (IMHO I think that rebase would fit better with Mathics), this would reduce the number of merge commits

Usually I use the "squash and merge", after a rebase. Sometimes, when rebasing gets too complicate, I copy the changed files over a fresh branch based on the master branch, and redo the PR over it.
I think that the main problem here is that to allow more tidy PRs, we need a tidier, less entangled API, in a way that any small change requires to make changes scattered along all the project.

2. Require linear history for PRs, this would also make the Git history easier to follow

Not sure how to do that, but if you have guidelines for it, I would try to follow them.

3. Use [automatic relases ](https://github.com/marketplace/actions/automatic-releases) (by the way, it has been a long time since last release)

Also, not sure how to do that. In any case, I would prefer to postpone the next release until we have finish to disentangle the API.

4. Mark archived repositories as archived (mathics/Mathics has not seen any update by a long time)

I agree with @rocky, that is not our repository.

And one extra thing: rename master to main

[rocky]

@TiagoCavalcante I just changed Mathics-Django default branch back to "master" from "main".

After trying out having the default branch be "main" instead of "master", I am not seeing an advantage but only a disadvantage. The problem is that all of the other repositories default to "master".

So what invariably happens is that I push to "master" in Django and get confused as to why I am not seeing the new code. In the last docker update I probably spent an hour trying to figure out what was up here.

I suppose I learned more about debugging Docker, but I'd rather have done learned that lesson some other time.