Date: Thu, 19 Nov 2020 14:00:36 -0500 From: Dan Langille <dan@langille.org> To: Marc Branchaud <marcnarc@gmail.com> Cc: freebsd-git@freebsd.org Subject: Re: Monitoring commits on all branches Message-ID: <BE9B078D-633B-4340-9A79-03B0FA60C431@langille.org> In-Reply-To: <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com> References: <197541CC-FEA7-4B4C-936E-66A5625BB64C@langille.org> <3c9f6285-ae7c-1062-2dd3-42f8c953a230@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Nov 19, 2020, at 11:16 AM, Marc Branchaud <marcnarc@gmail.com> = wrote: >=20 > On 2020-11-18 8:49 p.m., Dan Langille wrote: >> How can a repo be monitored for commits on all branches? >> I know how to ask a given branch: do you have any commits after = foo_hash? >> How do I: >> * get a list of all commits since foo_hash >=20 > A quick a note about Warner's reply: >=20 >> git log $hash..HEAD >=20 > "HEAD" is just a git nickname for "whatever you have currently = checked-out" (which can be a branch, a tag, or "detached" commit SHA = ID). Mathieu mentioned: git log $foo_hash...branch_name That was the first time I've seen that used. All previous suggestions = were HEAD, not branch_name. But they are all the same in this context? >> * know which branch each of those commits was on (e.g. master, = branches/2020Q4) >=20 > Unfortunately you'll find most normal git advice to be a bit = frustrating with the FreeBSD repos, because FreeBSD doesn't work the way = most people use git. Specifically, the FreeBSD project does not ever = merge branches (in the git sense of the word "merge"). Things would be = very, very much easier if the FreeBSD project were to use git-style = merging. I believe there are discussions underway about adjusting the = whole MFC process for the git world. I admit that part of my motivation = in writing this message is to provide grist for that mill. >=20 > Fortunately even without git-merged branches, there are still git = tools that help, though they're not as precise as one would like. >=20 > Let's look at a concrete example with the beta ports git repo (which I = just cloned), and compare the 2020Q4 and main branches. I'll start with = some overall exploration, then address your specific question. >=20 > There are 298 commits in the 2020Q4 branch. I know this because > git merge-base origin/main origin/branches/2020Q4 > tells me where 2020Q4 branched off of main: commit 5dbe4e5f775ea2. = And > git rev-list 5dbe4e5f775ea2..origin/branches/2020Q4 | wc -l > says "299". (The "rev-list" command is a bare-bones version of "log" = that only lists commit SHA IDs.) [examples snipped] I followed that. I took the merge information as background and good-to-know, because = FreshPorts won't be doing any merging. It just needs a good "git = checkout" working copy. Sorry for such a long reply. * How FreshPorts extracts data FreshPorts is only interested in a snapshot of the repo with respect to = a given commit. It works on the 'repo as a whole' to extract values = from the ports which were affected by that commit. Case in point: a = commit to a parent port might affect any or all of the child ports. All = the child ports need to be refreshed. I am quickly concluding that FreshPorts must decide in advance what git = branches it will pay attention to. At present, it follows all branches. * FreshPorts (without git) uses email to create XML When moving FreshPorts from subversion to git, one of the goals was to = avoid relying on email to know that a commit has occurred. That is how = FreshPorts has always worked. The email (from the CVS commit) was parsed = and XML created. This code was updated for SVN. The XML is then used to = load the commit into the FreshPorts database which then drives the = website contents. When I started the GIT conversion, there was no commit email. "git log = $foo_hash...HEAD" is how FreshPorts knows what commits to process. One positive aspect of email approach: it identified the branch. So far, = I can't see how I can process the repo as a whole and see every commit = and know what branch it was on. * Polling git It is beginning to sound like the FreshPorts git code for detecting = incoming commits will be: Every N minutes, do this: for each repo in REPOS for branch in BRANCHES cd to the directory for that repo git checkout branch git log $branch_last_hash...HEAD for each of those commits process the commit end for end for end for At present, the REPOS and BRANCHES are: * freebsd BRANCHES=3D"master branches/2020Q4 branches/2020Q3 = branches/2020Q2 ...etc" * freebsd-ports BRANCHES=3D"master stable/12 stable/11" * freebsd-doc BRANCHES=3D"master" Some might ask: * Why not just master and latest-quarterly for freebsd-ports? * Because commits to older branches sometimes occur (or at least I = thought I saw one once) Commit hooks might also help, but I'm not sure if that will make things = easier or complicate everything * When new branches arrive It is vital that FreshPorts remain automated as much as it can be. At = present, under SVN, I might have to fix things perhaps 5 or 6 times a = year, usually because a commit did not get processed. Keeping that in mind, I do not yet know how to handle the following = situations: A new branch comes out. * Automation might be possible for ports quarterly branches * FreshPorts has to know there is a new branch * BRANCHES needs to be updated * I don't see that it can be automated for stable/* * need to handle 'git checkout branch' when branch does not exist? * Once branch exists, how do you find out about the commits when you = have no starting point for 'git log'? Right now, a new quarterly branch is noticed when the first commit email = comes through. FreshPorts then does an 'svn co' for that branch. I'm hoping someone has good ideas for my edge cases. Thank you. =E2=80=94=20 Dan Langille http://langille.org/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BE9B078D-633B-4340-9A79-03B0FA60C431>