From owner-freebsd-git@freebsd.org Mon Sep 23 18:51:16 2019 Return-Path: Delivered-To: freebsd-git@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AF2A8FC932 for ; Mon, 23 Sep 2019 18:51:16 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46cYLq4Mhjz4Mw5 for ; Mon, 23 Sep 2019 18:51:15 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qt1-x841.google.com with SMTP id o12so18466030qtf.3 for ; Mon, 23 Sep 2019 11:51:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=egX4S2MMjGk7foAaHDXxCyQOxX+9/bEZ9MZuOWBkggE=; b=W+NxHgaPMt/XFaFHMelPLg7W7Ms10WKScwOA1Ae9zLnEVu3WoEK7q3kq4wx94pBldo 5v4ON3Dlc7mmiNEmUihL5VV0a8yPg6rEK4JlTTQ/kRnDmexUEyklYIr4yUd7MMSK18fx Iu4RxC6B7Fm9AEkl+itUW3S5no5b3vf4GTiz2htnld77SAZon0mehR4vSzcGdm7dodIq 8+Ut8ytlEVxrKQXPHrw9L7d7oRoGTEckXqbrahZbo1JTX1V5MmQ34qnZsiBrYQTW3R4/ LjcetFO4xZGS/wol7v+IWlwD47SjOgluwM5W8S0Nvw8DrOr5drDxI7wCIC1LHfk0OuDs zbGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=egX4S2MMjGk7foAaHDXxCyQOxX+9/bEZ9MZuOWBkggE=; b=RrfD0dlxG1vVCpOJfAEbe1VKHO7o1M5VcVl14K47LMFn1NFIlS7lZ7zDTSv5YtPVKC 7iNSh/HMvPuklbfhzf73azH8NU8TCJgeAmKGsq9eJ2cIY5epsEotcam2oc9NyEuI+Kw8 oT7gu2mz9zXvxfJK6uYm3bNUXeNizEagSIIUYxCgLjt+OJ+xup6pvvwJUS2xycJ24BV3 ZxPB0dj3QPY+f+4XOmNhhRZXnMDWyvFv5e/RyvE9E0YnTz1g/baIKqvwq3tLL+fyTSNj SNU4TfnryHhx9PrySJ8gNpwIXOXR9YB2yOtvcmIH2TswrKdXLSY6vA52ZagzwbXvTc1s y1/A== X-Gm-Message-State: APjAAAXgaot5U9FsoGMz4GV2222ep45ap2yGIMD9lqNHpiq3pLgm21tc XXbTbayAVGws9XLbSnsUogh0sSBW7+w= X-Google-Smtp-Source: APXvYqyZBFn0yld6wBf+AZ2+6gOBr6577QZZKdFGh/l9XAC3DDNCQS99ngwum4bQdI2EC3LCro54Kw== X-Received: by 2002:a0c:ea92:: with SMTP id d18mr696164qvp.112.1569264674582; Mon, 23 Sep 2019 11:51:14 -0700 (PDT) Received: from mutt-hbsd ([63.88.83.120]) by smtp.gmail.com with ESMTPSA id z200sm6096124qkb.5.2019.09.23.11.51.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 23 Sep 2019 11:51:13 -0700 (PDT) Date: Mon, 23 Sep 2019 14:51:13 -0400 From: Shawn Webb To: Ulrich Sp??rlein Cc: Sean Chittenden , freebsd-git@freebsd.org Subject: Re: Service disruption: git converter currently down Message-ID: <20190923185113.dyvxxn36gvj4dtu5@mutt-hbsd> References: <20190923183424.ebnghzf67mx56aom@mutt-hbsd> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="yuhiezjj5qjvvmb7" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD mutt-hbsd 13.0-CURRENT-HBSD FreeBSD 13.0-CURRENT-HBSD r352488+2a20025064d4-c272097(hardened/current/master) HARDENEDBSD-13-CURRENT amd64 X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0xFF2E67A277F8E1FA User-Agent: NeoMutt/20180716 X-Rspamd-Queue-Id: 46cYLq4Mhjz4Mw5 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=W+NxHgaP; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::841 as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org X-Spamd-Result: default: False [-4.58 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-git@freebsd.org]; DMARC_NA(0.00)[hardenedbsd.org]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; RCVD_IN_DNSWL_NONE(0.00)[1.4.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; SIGNED_PGP(-2.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; IP_SCORE(-0.48)[ip: (2.49), ipnet: 2607:f8b0::/32(-2.64), asn: 15169(-2.20), country: US(-0.05)] X-BeenThere: freebsd-git@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion of git use in the FreeBSD project List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Sep 2019 18:51:16 -0000 --yuhiezjj5qjvvmb7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Sep 23, 2019 at 08:42:10PM +0200, Ulrich Sp??rlein wrote: > Am Mo., 23. Sept. 2019 um 20:34 Uhr schrieb Shawn Webb > : > > > > Hey Ulrich, > > > > I appreciate your hard work in maintaining the git mirror. Work like > > this can sometimes go unthanked. I want to take a moment to show > > appreciation for you and the FreeBSD project in maintaining the git > > mirror. > > > > I do have a few concerns with what was stated in your email. I've > > written my concerns inline. I hope this discussion is a positive one, > > wherein upstream and downstream can effectively come to a conclusion. > > > > On Mon, Sep 23, 2019 at 08:16:25PM +0200, Ulrich Sp??rlein wrote: > > > Am Mo., 23. Sept. 2019 um 19:51 Uhr schrieb Sean Chittenden > > > : > > > >> > > > >> Please note however, that more "garbage" metadata escaped from SVN= into > > > >> github, meaning 3rd parties have a hard time re-running the conver= sion and > > > >> making sure that it matches SVN down to the metadata (i.e. timesta= mps). > > > >> > > > >> Eventually, this will have to be re-rolled and a new "master" bran= ch will > > > >> be force-pushed into github. There's no timeline for this yet. > > > > > > > > > > > > Wait, what? Can you elaborate? > > > > > > > > Discussion of a force-push to github has occurred a few times and b= een explicitly ruled out because most of our corporate citizens use github = to integrate changes from FreeBSD. Rerolling master was universally reject= ed when we socialized wanting to do this due to the level of disruption thi= s would cause. The feedback was that this would be a high-cost, low-value = operation. In the tradeoffs of purity vs pragmatism, pragmatism wins every= time (that is the FreeBSD way). > > > > > > > > -sc > > > > > > > > > This is not just about pragmatism and the disruption it would cause is > > > vastly overblown by people who don't seem to know much about the git > > > storage model. > > > > > > There *is* garbage metadata in the published version on github, there > > > *is* a disclaimer on https://wiki.freebsd.org/GitWorkflow since > > > forever, and the cost of switching from 1 published branch to another > > > is literally: > > > > > > - git diff origin/broken_master mybranch > mybranch.patch > > > - git checkout -b fixed_branch origin/fixed_master > > > - patch < mybranch.patch > > > > Such a workflow breaks historical accuracy. Instead of `git annotate` > > showing the history properly, it's now based on an "epoch commit". > > Sure such a commit brings the branch to a working condition, but at > > the cost of history. >=20 > Is there really that much value in having "git blame" work in that enviro= nment? > My mental model is of short-lived branches that get upstreamed, so I'm bi= ased > towards this not being all that big of a problem (for some at least). >=20 > > > > > > It should also be possible to merge both broken and fixed master into > > > your branch (at the exact same SVN revision in time) and then you can > > > follow fixed_master going forward. You'll schlepp around double the > > > commit history, but not tree objects. > > > If you want to retain history, you can upstream the changes prior to > > > the switch > > > > I so wish that were possible for certain downstream projects. We're > > unable to upstream the majority of our work. To argue "upstream your > > work and you won't be affected" is to choose an argument that does not > > reflect the reality of a growing portion of FreeBSD's downstream > > consumers: the inability to work effectively with upstream. >=20 > :/ >=20 > I'm 80% sure that you can just merge both branches and things will be fine > (though the exact incantation will surely be black magic). I'd love to > try this on > an actual repo though, I don't have the time to craft some test repo to v= erify > this assumption, and then find out that other repos are different). HardenedBSD's github repo has existed since 2013, with branches stemming from that work existing still today. Perhaps HardenedBSD is somewhat in a special case: we aim to provide the BSD community with a clean-room reimplementation of publicly-documented parts of the grsecurity patchset. With FreeBSD not taking the same approach, we will have very long-lived branches. For example, our hardened/current/master branch follows FreeBSD's HEAD and syncs every six hours. Meaning, we maintain our patches, resolving whatever few merge conflicts arrive. The hardened/current/master branch was created so many years ago, I've forgotten when it was actually created (perhaps in 2013?) Though HardenedBSD's cause for existence may be a special case, this problem can be viewed in a general fashion. I'm confident HardenedBSD is not alone in facing issues of these types. Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 Tor+XMPP+OTR: lattera@is.a.hacker.sx GPG Key ID: 0xFF2E67A277F8E1FA GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9 3633 C85B 0AF8 AB23 0FB2 --yuhiezjj5qjvvmb7 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAl2JFBkACgkQ/y5nonf4 4fo53Q/8DGpBwo2owUaJJSoGfVPpymG/gsucxtNdDa6XPC1+43Dggport4To+XcU kjPLrSaq8Z3LWVwsRNFdPn1D6+XnwYYupwJwZCheWe9CoV1Z0COsLJQRnVwRkkTu o0chDU62yd1dcRUbXc/48m7SE/2V/WrZpqj9/9grKGecx9if/PotcHL/+R/0Wa2W 5zCKURXcHYmUXvqCj3EzbD0Q7GfEm+ZfiZQgdYiZEP5y0J4xUP4Lf5VgBItCWA3Q 7gG8DDQwC4/LIYWZRm0gg+/94/cn2yUXzMYRJyBBbXitJVCnZjFYf8XsPdtaO53e Tx3Z71Ya9A5uuZYgKRSw41ndGkRBTCrbo/VgNxEm297a0AHJaUXA1s2LxgikEEWU fbAW8tR0CM3bB6e5XlbjxPtRPCev63MQpAA3yi8lUxEPf33jHLuNSkudJORjvzR9 TKQAumjyn2UgrcAqbJ+udnVOxhRhR+Sx0E4tVFPYfUGQr5mEj27P9VUKHU+7SerA XduPaOeT8zBhRhV7RVGaklnqoePg1XVUMJf7w75g2YovqdZlUEBue1AUUXz1Rr/p n+4rmMYcCpXfdYruORzcuJSRDfViGJtKN3Bk2k2Mt3EoVWTDyP5eeGumwOFzn1Cm v5z71J4aGFAOaukxSY2JiJu1i0PnssUKBOKgZEUcX2ctUSHphog= =sBvH -----END PGP SIGNATURE----- --yuhiezjj5qjvvmb7--