From owner-freebsd-stable@freebsd.org Sun Apr 28 06:59:58 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F298015748C5 for ; Sun, 28 Apr 2019 06:59:57 +0000 (UTC) (envelope-from work7@aadhyaitservices.com) Received: from md-in-36.webhostbox.net (md-in-36.webhostbox.net [103.21.58.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0492987227 for ; Sun, 28 Apr 2019 06:59:48 +0000 (UTC) (envelope-from work7@aadhyaitservices.com) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=aadhyaitservices.com; s=default; h=Message-ID:References:In-Reply-To: Subject:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version: Sender:Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+Vm0JWgwU7kHHsdsSPn/dOXihTryNV+nxgrIsrKyGTY=; b=J+Br0cuDGyw0TY7nT7BUlZ8/cO xbsJm3Dy1iQ/Ui7CRv33W+D1vSS7LltDp1Op9N3vIH3PGAMgSHmM2WzSCXu4hgMFC09n+yv4CDjtj Z7XUs1OPNxU5ftltQZeaSfqK6y+EfL3/wCUTxOWiBBJS3pJLnoznUcx2z8DjiRKAKzG/KH5mwsAeF ezz8gctx3kafbybr7Po6Xi3tB/DqKIG3gqmvpdHK/76xdbn117fkK69o0PQCiAyaP6NgVm4pjQ2lO kVUZlJDj0aaXkvHzAEN80EAgcBywlNRHfxyzIySJzMz0rH1TvZsczIHr9KDo26GRx4cRV2DVAx4Yo RwY8gg9Q==; Received: from [127.0.0.1] (port=47587 helo=md-in-36.webhostbox.net) by md-in-36.webhostbox.net with esmtpa (Exim 4.91) (envelope-from ) id 1hKdmX-002d98-FP for freebsd-stable@freebsd.org; Sun, 28 Apr 2019 06:59:25 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Sun, 28 Apr 2019 06:59:25 +0000 From: work7@aadhyaitservices.com To: freebsd-stable@freebsd.org Subject: Post on freebsd.org In-Reply-To: References: <034e68b465d13d33340bca268c77b4b0@aadhyaitservices.com> <6d35a54981ec19b2a882d54cbcd58831@aadhyaitservices.com> <16c0e97c33a22457bca953eca09a1f5c@aadhyaitservices.com> <6e7b10cf7584a416f141aa12d1af24ac@aadhyaitservices.com> <0cd0f91b819ca3f53d430b7f4d3c2e30@aadhyaitservices.com> <970702124e9a447336a5edd41484e319@aadhyaitservices.com> <630362313b8b29a41f5d6114f2654c8b@aadhyaitservices.com> <95e354ca753ee385917066042dc3210e@aadhyaitservices.com> <1d878c567d084b92f4b5d658f7d874b5@aadhyaitservices.com> <8c94356b550fc787b64096a2136327c5@aadhyaitservices.com> <574121e6244b77f50b41e372f7ee45a1@aadhyaitservices.com> <27d9834b35744a719ec398e416be7e5b@aadhyaitservices.com> <42543c34bfc728f4e12c25c825814d68@aadhyaitservices.com> <9c4d539c354abbf99f46c4b1485f1725@aadhyaitservices.com> <21859b3f222f0de84eae67bdf70384aa@aadhyaitservices.com> <50e14419bb22f79412d32bedc18ca5d0@aadhyaitservices.com> <2ab9113acb8b0b4236f5c08a14c099df@aadhyaitservices.com> <037a6a67edb8ce30903d6154d1959770@aadhyaitservices.com> <3a5173c694595e6f7ad943e35a2c601d@aadhyaitservices.com> <00b53f98a1a75c444ebed2b19f9dbc69@aadhyaitservices.com> <581e1ad154a6554b8b069ab330378c6d@aadhyaitservices.com> <5864d78cfd5c385d503a122a6569df10@aadhyaitservices.com> <0a6724243bcc14fa68dbe7e5dd10d661@aadhyaitservices.com> <2548e83d3470a43a632e9fd2dafc1f04@aadhyaitservices.com> <681a14113db8502f1ee4f5a12ff31199@aadhyaitservices.com> <20ee756a6299c4d7dd261e2453ed1bc3@aadhyaitservices.com> <91fa9b0731f7fab239795756f5139c84@aadhyaitservices.com> <44d3566563b28227544054f98b4090a7@aadhyaitservices.com> <5a6ae9cf405194936f6101b7d7895910@aadhyaitservices.com> <3024b7fa46dff61761d923ffea71425c@aadhyaitservices.com> <3b58f90bc7ce9767530015df2feee4c1@aadhyaitservices.com> <45854b226e02c9e3ca1bb24dbe500869@aadhyaitservices.com> <9d1210aaecc75acfeccac410adbeb713@aadhyaitservices.com> <6827960009f80288618147b521e44d6f@aadhyaitservices.com> <04302a589a6143d4b9b28957c9d8de2f@aadhyaitservices.com> <902c243a8b7ebd479ac2d450c6ae0edd@aadhyaitservices.com> <1284cedc3d6c890699130ea4848dcfa2@aadhyaitservices.com> <145278b74a80cb35e3b2fc56e057c037@aadhyaitservices.com> <5a91f724ccdc53fc8fc473cd3aab6d40@aadhyaitservices.com> <5598330597a39c2c2bf16107cf046c90@aadhyaitservices.com> <2dc5fc6d7b9c9c1412d23c9309d4302a@aadhyaitservices.com> <79b250f75e4a4ad05fb966dc30a120e5@aadhyaitservices.com> <7087e3f9bc973df94f1aa13ab5afc901@aadhyaitservices.com> <0aa72f13f232c78102a19a81158e9ff4@aadhyaitservices.com> <9844f1aebfbcfd0f47eb52154834341a@aadhyaitservices.com> <9d7602de237c07314a7fe8962b3a0f80@aadhyaitservices.com> <11b12172c1a28be9ad55edcad1d42148@aadhyaitservices.com> <2efd151e1569b93a1c2e52b400d2c2f0@aadhyaitservices.com> <717f947776521abc3d1801021885e689@aadhyaitservices.com> <6a64a1aa1b14b870cef10dfba1748ce0@aadhyaitservices.com> <71a688f44d87452b4008c481f2330b9d@aadhyaitservices.com> <1c24a33fa24c9e160e03870023e1fd44@aadhyaitservices.com> <2f772f252078928f9eb48f6e6703a24f@aadhyaitservices.com> <229aea9398f039ee1acd16082387ad13@aadhyaitservices.com> <84e7924043b0a48ef34eee54e101ebe1@aadhyaitservices.com> <45a54917e06daa7090994ee418ca66f0@aadhyaitservices.com> <56e3ad8c972c09b74dcee088eeda61ef@aadhyaitservices.com> <3d91080b3ac0c617d1a1596a7db8f4e9@aadhyaitservices.com> <4f32ae15df2380eec62802e87acf420e@aadhyaitservices.com> <7a1c3394dd3ab20739fa828a6b8b4c31@aadhyaitservices.com> <7790d3cf6c09c629f3099df4fd4a7037@aadhyaitservices.com> <10eda9063efbbedac37807d98a49f6d9@aadhyaitservices.com> <2cf0ed8e9d256bd37a2d078b058fc497@aadhyaitservices.com> <9d741f8593fcc828d9d5947bcaaa2489@aadhyaitservices.com> <36ffcedd8210e53fe874ce39590c8cb8@aadhyaitservices.com> <88ef59a3ec4e946868f5fe67a61bd137@aadhyaitservices.com> <463f4d2f85094e1495eeac4b596b6bdd@aadhyaitservices.com> <0947bb990e299de1ac818fce05efbecf@aadhyaitservices.com> <590020e7c4a21de42482903c243dd041@aadhyaitservices.com> <800605760a301c4594ad3fe2ad6d18d8@aadhyaitservices.com> <5c8c8ef9a3eec91ec14f0b0fb54c2a91@aadhyaitservices.com> <971b7e74a3c265e6cfbb8b4311d45c07@aadhyaitservices.com> <8cb4b8543625d4db07988bba03e24acf@aadhyaitservices.com> <42b00812d150e418f755fe7009b3d7cf@aadhyaitservices.com> <08f583c8d06102edaf1fb31bfbc23b69@aadhyaitservices.com> <8ed809afd4fe701a767b8c8531d75654@aadhyaitservices.com> <0f47cca0faf3a7e256ee075a965caf6d@aadhyaitservices.com> <8f178c694b8e312002b1c5f9629e3f37@aadhyaitservices.com> <2f47438e7bfb6d5d6f3665de52a7b135@aadhyaitservices.com> <20fb493bf96b637678ac8b7c1e1de138@aadhyaitservices.com> <876730f78a448c141666203602492b0b@aadhyaitservices.com> <3205c63c5121a8fab9ccbe0e9d7bfcde@aadhyaitservices.com> <3f136a5ac56598f078958f232ee4770b@aadhyaitservices.com> <8a8d848f8256bedfeac5a8a6186b0276@aadhyaitservices.com> <53014f92c554ff7adbcd4b1594f76f9c@aadhyaitservices.com> <227035dc2c947a06a0b740d73c0fcb52@aadhyaitservices.com> <382b998630666d6c581aad149b0dda93@aadhyaitservices.com> <44f30e9f8b2864048cb4ea595888b5fc@aadhyaitservices.com> <3578915c7d130233d56936b882b7acc1@aadhyaitservices.com> <5560ef43ac65e3d17158d2a1f3189573@aadhyaitservices.com> <06d46c61360720bf82abb22cc088d05e@aadhyaitservices.com> <6002f47f199e190c95dbc7c2ea8cf00d@aadhyaitservices.com> <1e8d223f26e2d560d64ad12896f24d5d@aadhyaitservices.com> <6fd27595f17993d3040ea7521baadff6@aadhyaitservices.com> <088740062477536b279c0698649c6e05@aadhyaitservices.com> <8a8c4705328b1bf174ba6f173143d151@aadhyaitservices.com> <7b3f77d9b2df28db0ac59de16f6d1231@aadhyaitservices.com> <1ed9e30aab6f293962aec72e755ef556@aadhyaitservices.com> <9e850f5a7eadee79b47ef1c55fd8e820@aadhyaitservices.com> <039fd24535af61cfb20412fc6cfc01a1@aadhyaitservices.com> <763c0f91e91a2906aededf7062267ae8@aadhyaitservices.com> <0ba136ade390bb4dd83433db629faba1@aadhyaitservices.com> <6f2b9a47b2b070a38c7702d97ff7e3da@aadhyaitservices.com> <1dbddf0896d157b1e469caa9e5a144e3@aadhyaitservices.com> <43e03ae2c34dea9644f8313f96a90e35@aadhyaitservices.com> <1bef2d86aa65357509470af24c852b96@aadhyaitservices.com> <3c04f296f0ecc7e2be17ceeb52e7417b@aadhyaitservices.com> <9e5dd32f81b7bb7341f1d3ee83598322@aadhyaitservices.com> <7e395bc089e0d392b18d32691a2c8532@aadhyaitservices.com> <91856108e719b46f9918e78991d707cf@aadhyaitservices.com> <71e1d9464912fcfa1fbda24ee29d9e77@aadhyaitservices.com> <1acd4c98e9ff96259f3184a04f35f19c@aadhyaitservices.com> <4a55f93f015d49988a6ebdc10d079d46@aadhyaitservices.com> <5877e1601df96736c7e28e828eedba4a@aadhyaitservices.com> <9f8aadc76fd2aea2228e687db81a8d84@aadhyaitservices.com> <9d60d1d3f18021f5c7c6e285c171c498@aadhyaitservices.com> <40b6e618c197a540f4c568d4cb7d9c81@aadhyaitservices.com> <30f30d3c85931a86103c67f3b499a6a1@aadhyaitservices.com> <0b83adf008a0a614b0d8115ad2b27978@aadhyaitservices.com> <3ea6afd5d9b8d2194a611d95bb7bf381@aadhyaitservices.com> <7d7f1bd221af17674269919082b4b69e@aadhyaitservices.com> <41fbaaa6a840178d5b2f7dd5177a8f92@aadhyaitservices.com> <848519bb081674f81e95fb8e165efb9f@aadhyaitservices.com> <4f9f87d94fdaf1dbf6b891ba4aac466b@aadhyaitservices.com> <8d19bbe5f3c2a609e402f3995486ca8f@aadhyaitservices.com> <26d9ab20a442ff2140f34b213a65ec0d@aadhyaitservices.com> <0e636c7f10d8a2a05c829cb426186710@aadhyaitservices.com> <9fe01c81aeaaf4e2d1c26591d7c3053a@aadhyaitservices.com> <70a476d0777bcb0934387070c450d202@aadhyaitservices.com> <66e2bbc910dc5ca1e12ebd87d9ec017c@aadhyaitservices.com> <0e596eb98281bb03bfefd48f6ebc8e2f@aadhyaitservices.com> <99a41858c9947bccc492eb9f957c4916@aadhyaitservices.com> <77e05a0b43cc0c423db365d71c4bbbbb@aadhyaitservices.com> <0c10040f30c66dff70f9292058d2912e@aadhyaitservices.com> <19cf148b7850934559733a1d8160d222@aadhyaitservices.com> <3195f55df102086d1471d6a411e52793@aadhyaitservices.com> <6781947377d9da990ea3860bb2357af9@aadhyaitservices.com> <4aef26b57cc7db2d9751d2ae4d71cc68@aadhyaitservices.com> <5489e12f47e028681189dc2368202f49@aadhyaitservices.com> <8d32d1d637c5c9d60504cf6da5423eb9@aadhyaitservices.com> <6cf588582cccdf13f2f08885979310eb@aadhyaitservices.com> <2503ee32d6692d63fd1c6da45ff1a9f1@aadhyaitservices.com> <075f74a3b93f8d5961c3536afc0f7c05@aadhyaitservices.com> <1d3a9d90d7f33a5fd3a7b976f4a548d7@aadhyaitservices.com> <91d8b12364e3dea4dd084a1ac16d0bcc@aadhyaitservices.com> <9b3ec369b922dc94523ae59f8877194c@aadhyaitservices.com> <071cc4774bac2c02d7d69cac16f8c578@aadhyaitservices.com> <3f9b1b45cb5b5c2e6db5d32556598ddb@aadhyaitservices.com> <15ef5dee19dece5b4a555746b8b61738@aadhyaitservices.com> <11786f2817c10f900ec18bfafb8d25eb@aadhyaitservices.com> <054382bb6503483c00d1e5992a3639c3@aadhyaitservices.com> <41bfc56fcc48749f63da49e4f9668008@aadhyaitservices.com> <93f8ccbc1e4983cf89432007a90278e6@aadhyaitservices.com> <254ad7a4f5411e9aa7758ede74506df3@aadhyaitservices.com> <9c2df2e8586fdc97c169b22c0a64f86c@aadhyaitservices.com> <42981ba8b289c93aaa596b8e9dd5b3bd@aadhyaitservices.com> <743a3c194d6ac05ed317eb22bc18b9bd@aadhyaitservices.com> <1ba35d9d74b0cf8aea40b0df5a0bc90c@aadhyaitservices.com> <1951a6703c523a81ff9ba3e1cc5515e7@aadhyaitservices.com> <6e3b363f78c01280a12464675aa992b1@aadhyaitservices.com> <03db59a2c7387f859a2abf35fff90f4a@aadhyaitservices.com> <53b6e31c1c8f58944bbe5da45c8493c4@aadhyaitservices.com> <52bb33ba064a8987f3a262df3376117c@aadhyaitservices.com> <323ff74d2a4e7b12d3645b4de7611fca@aadhyaitservices.com> <76532f3f9443147c8c5c0fa55c9a5889@aadhyaitservices.com> <58d0fb1e918f30533325b44592860008@aadhyaitservices.com> <5daef82abdfb5ea7ffb9fa3f6765136a@aadhyaitservices.com> <43be0ba345f21d6176618d69ca71d676@aadhyaitservices.com> <9c9da2452aa39233ec6d959f18bd21fd@aadhyaitservices.com> <17069c5bcba00fd229ce1bd3e96362df@aadhyaitservices.com> <94f86bc6e56565a038536fff4b68ffa9@aadhyaitservices.com> <7cad311753f7e27dc1fef562a4f4ac09@aadhyaitservices.com> <1a9a7f85e184a42790ce0946a614053c@aadhyaitservices.com> <22b00727cfdb9419b413e0507d06a409@aadhyaitservices.com> <8622317065059579896becedf347c391@aadhyaitservices.com> <7ea21de9d903d21ee1a6bc1edf07d358@aadhyaitservices.com> <357dc7f52f2e237960e4eff2a1a0d7c3@aadhyaitservices.com> <6163e3759181a53be368e4c3dc8120cd@aadhyaitservices.com> <905d2b242b9bb0d9e6c74d24f371b24a@aadhyaitservices.com> <0de127cd0c73bb280f7b4380b0374fcd@aadhyaitservices.com> <40dd4165339cfb99bba319d1687e9061@aadhyaitservices.com> <32d2009d89670574b0c315444c820fcc@aadhyaitservices.com> <4f7116eed6dbf3f256c42e3e7d6703bb@aadhyaitservices.com> <478a6b3def68d377cfed407318a036d6@aadhyaitservices.com> <5e4d162cb875438bb479c78228f369b6@aadhyaitservices.com> <71da837f87d696d58c4d82721587afba@aadhyaitservices.com> <653cbd3c34ccf8cae71869e513a8d9af@aadhyaitservices.com> <00be2b34dd721e5db02f2f83b39e4db3@aadhyaitservices.com> <93efbbb5805693903a927e8df2ce423c@aadhyaitservices.com> <6c499864d1ab43b2d2d8fc11662278ec@aadhyaitservices.com> <0cbb3848b0cf2c9aed8eadfe2a6095cd@aadhyaitservices.com> <18d02908b033b18c51c62141f32d8de3@aadhyaitservices.com> <580a44ff60bda062c653e4aeee652213@aadhyaitservices.com> <129c8dd8be78c73b8a00f92f82ff1f10@aadhyaitservices.com> <05f5f4a57e6ef153610aeec384a879fb@aadhyaitservices.com> <0347f84baee88c64e2f28bf566eb058b@aadhyaitservices.com> <9a245e2b519f1f751710239c68b73f75@aadhyaitservices.com> <72aa4db0ce8c29f3d929e78825e3b6d8@aadhyaitservices.com> <11d7862a0c99965c03a85323f18c1938@aadhyaitservices.com> <5e5e4f96488972a702fd3dbdc83006a2@aadhyaitservices.com> <123d92ca2e77163e599d5a551029d1e2@aadhyaitservices.com> <0ade95da8418ec017e4228ead08393f8@aadhyaitservices.com> <4e6701dee133e2e96ec94b6e21f04815@aadhyaitservices.com> <28af4bfa3bc7b2eaf5aed30a71d92894@aadhyaitservices.com> <6c9d73002d084b6e5979f418b6975b9c@aadhyaitservices.com> <51295ab97b6bd8a43c0fe0adf615bf4e@aadhyaitservices.com> <1958214438b691447eb4698788dc2824@aadhyaitservices.com> <7d6c070d474a7da4c220b1dc2fd84434@aadhyaitservices.com> <4a19891b7eab3829cffc7d6c34deb21c@aadhyaitservices.com> <0886035af3f781f557a0783d23741f3e@aadhyaitservices.com> <91ffc5451f3d34a3ba4b704568e1d120@aadhyaitservices.com> <674596d8bfd541f4c093297f4f3b66c5@aadhyaitservices.com> <33e3c7baca6d59ab3cbae38a33699254@aadhyaitservices.com> <8ba50696109d16ed865a98759b247fe5@aadhyaitservices.com> <5ec418c92d0aa2a813648546fa239153@aadhyaitservices.com> <038e2dc599f645b658d687f6bd51c4bc@aadhyaitservices.com> <6007381fd4558de44812941070f53f3d@aadhyaitservices.com> <648b7a04a57891d5d848acdd67d803b6@aadhyaitservices.com> <48f08a2038ca905ba3cd1ca4feed9109@aadhyaitservices.com> <003c9bb2330d4d1df20443db8b570e81@aadhyaitservices.com> <35bf39ebaa4198b382b5e0274259672a@aadhyaitservices.com> <83c9c6f49b9915cf6e0ff54687329ffe@aadhyaitservices.com> <99f0cfe2e631db3efae6c43071387f41@aadhyaitservices.com> <71935ba79436aaca16f8f39d8b5a191b@aadhyaitservices.com> <8278f1f2fb41737ffce58c611a70a68c@aadhyaitservices.com> <2df1a6c4228aa2e8125a7cd37d232f49@aadhyaitservices.com> <793e9b6e4cc0c1fe76f801a681186f4e@aadhyaitservices.com> <64a83513b8fb4593946d8cff6d11ac76@aadhyaitservices.com> <938eda170ac57be45b15dacaf76412fa@aadhyaitservices.com> <57cf2d3bcb504d84b207b87b05659e0a@aadhyaitservices.com> <5e135f19493d22ab89e099a80d0d5d32@aadhyaitservices.com> <6ef22e05188a205bb12cc11d9fb877b5@aadhyaitservices.com> <3be12e0f9c1f78d87edeea3d9951ab1a@aadhyaitservices.com> <8de6c5d7580aa601dc815433e501ec9a@aadhyaitservices.com> <6100676c0e82fbf8bf456051b0c0163f@aadhyaitservices.com> <48276634e7bf7350cc3ad1442d2d742b@aadhyaitservices.com> <192a6e71df0ab2560574ac973f770d46@aadhyaitservices.com> <096922e84c3d42cdeb1e2bd9b56b97da@aadhyaitservices.com> <1f6995b148dc26b28263648b76c32e5b@aadhyaitservices.com> <44ca056ee8af2bda1be0cbf9545cff2f@aadhyaitservices.com> <3a3dd9b2407d7a8f55a6f9b43ed48af6@aadhyaitservices.com> <0355154632b16b947789ff395bb100f8@aadhyaitservices.com> <17c946255fc2c34d463de9b8051bac61@aadhyaitservices.com> <9bd616c7ed3fc17aa8824161ad60ad19@aadhyaitservices.com> <4196e7a037f3cc62f99d6f29dce84b99@aadhyaitservices.com> <57f6cd1a89bf242fd3d5008fc619dd48@aadhyaitservices.com> <4efb3aed4959f6467c5d3a988b502137@aadhyaitservices.com> <69d35e677568e64afddb8faf850a000e@aadhyaitservices.com> <85cc139884091852acd1b6c021421bdd@aadhyaitservices.com> <36e2e298934d2a455a233ad0cda2b8c2@aadhyaitservices.com> <22ad8c260cf00b6dce994d1fcc4c5178@aadhyaitservices.com> <43249b04462261861c4522ee96168d98@aadhyaitservices.com> <3dd55542fd5b8aac68fefba8869809e7@aadhyaitservices.com> <18e60800fc39ffef0eae2ac4e821b38b@aadhyaitservices.com> <899f79bd157a07ff6c79b693c8789224@aadhyaitservices.com> <339ba0de9477166c077519a991487941@aadhyaitservices.com> <124882bdc91a051dc46d3cfa35287aad@aadhyaitservices.com> <5d935d6c481b5ad60e9cb9da82bf0e8b@aadhyaitservices.com> <81cc3455f8f25d34661d97f302d0dd66@aadhyaitservices.com> <5532601a442f2f77d78a33cc14c6b6fe@aadhyaitservices.com> <8938f62a7ff35f7a44dba13789d167fd@aadhyaitservices.com> <0a61bc3ea7e4b045d0f3d4d28ceeb87c@aadhyaitservices.com> <929c08334d6b6c46508c7d784b83a69d@aadhyaitservices.com> <457b5b940add0ea6c5f4e691f9603d49@aadhyaitservices.com> <079c848b8dcaecec8439859c17088ea0@aadhyaitservices.com> <1d898f595911b8eab08bf27bd481cf8e@aadhyaitservices.com> <7fb4daeaaa4f78b9768fe3616db2fc87@aadhyaitservices.com> <303a57834030e96cab777f5ca1360e5b@aadhyaitservices.com> <1943555e04992453ac8f6c4f9bdead76@aadhyaitservices.com> <09d0f1d00a25f9d11c0362550c4b90dc@aadhyaitservices.com> <65356baf0a3e626485a52efe358cf625@aadhyaitservices.com> <248c718f38d43cacca1d355fb6d37452@aadhyaitservices.com> <324607052c075d5c0d5090d7ce465a98@aadhyaitservices.com> <86281daada9013ece5fb75e40bb072b5@aadhyaitservices.com> <0f9ed3acb6ad11e2768723f6f8366357@aadhyaitservices.com> <8831dfcde0e71f085bf114636442367d@aadhyaitservices.com> <5adb69b06d2a91102876e41bbd0f8046@aadhyaitservices.com> <450fbef24484d830382e59a4b980478e@aadhyaitservices.com> <1094565bfabd8643e0aaa57e3a5a88aa@aadhyaitservices.com> <16da3ebe0155cd5882bacbd4baf503c2@aadhyaitservices.com> <6310d6c111bce17468843b916d5c3180@aadhyaitservices.com> <7942293ee5846d046e94ba31902090ff@aadhyaitservices.com> <018ab5374e450f2a08e0c694b277b12d@aadhyaitservices.com> <2151373aa85d3efff763cdfd5de95c40@aadhyaitservices.com> <4cabb4a03c784d49edffbbdf0c5e3f3b@aadhyaitservices.com> <81e068f7729dcf63288960639655057e@aadhyaitservices.com> <76565c9b3415fa11d549cfc00306125b@aadhyaitservices.com> <4d91df4563863f1c5ea5a7982cf77950@aadhyaitservices.com> <07ac94557f28a9ef2f8567e33a75dc2c@aadhyaitservices.com> <13cef9134033ac48459126e1370f3dfa@aadhyaitservices.com> <995111ec3bbfbf1dc22a1da1d6393717@aadhyaitservices.com> <40613628d83bfd7af3f2aa563a2e7d28@aadhyaitservices.com> <972e89bb9337e3117e6f2091ef4859b5@aadhyaitservices.com> <5a9877c511d2707871cffa84a2daff0a@aadhyaitservices.com> <7ed9e0c097917540fd2136f1dc6ea769@aadhyaitservices.com> <91eb5bfd851c24d56223ae821d74d433@aadhyaitservices.com> <17819bbdc93d95141c1ea1d7eafa6e98@aadhyaitservices.com> <36665c2e622d2bb7c92857cae5de3a23@aadhyaitservices.com> <812cf9179e955ba1d1254c8c1a19aec0@aadhyaitservices.com> <24849574b7f7bec11f8b153756b1b5f2@aadhyaitservices.com> <2233084b5586b076698a61e55a82a840@aadhyaitservices.com> <29d86ef76f24e1bc1d12e586bf874b21@aadhyaitservices.com> <4ccc92b4d51bdaf604cf12ca8cc917e9@aadhyaitservices.com> <984c1dbba4be03c20902d924aab70f52@aadhyaitservices.com> <071e818eab00cde6ceafc70ff4864fa2@aadhyaitservices.com> <8155a19dbd5847c84d6af40465b6a70c@aadhyaitservices.com> <9481e0b1f3635753b01cac8ecbe75708@aadhyaitservices.com> <4dcc4fd9b40e217596ebc7425c0cff50@aadhyaitservices.com> <7b467b1fac8913869a26a48fa344db63@aadhyaitservices.com> <3f4eb3633066616bfe3c1fc5f0dd7413@aadhyaitservices.com> <9aa8b1b2173bbb126634d782f2309bc7@aadhyaitservices.com> <9b5a1e5634aeb60ead42feb731d6c7b2@aadhyaitservices.com> <844e6e3a5966def26dfc820340986d4b@aadhyaitservices.com> <07c282b1c090e1cf77325e33bca3c9ab@aadhyaitservices.com> <15ebc6abbb5ac2111a8b6f08d302ab0a@aadhyaitservices.com> <044dc165909eb5a598968fd062101447@aadhyaitservices.com> <30d81c769eb09bfb3ed659ac73008578@aadhyaitservices.com> <8e928d05a38609de8ae692ea1cbdd225@aadhyaitservices.com> <77df56b94e392e71dbd069ea5bec0897@aadhyaitservices.com> <1e5d0a5050845bbb6aa7c525b81f7abc@aadhyaitservices.com> <52ebed6b2802dbf4f798fa7d64b8869f@aadhyaitservices.com> <0dc0f0bf0a3419d99f6dc94958498be4@aadhyaitservices.com> <2bbe43d942e72357ad2343d95a8cf3fc@aadhyaitservices.com> <164b70162457d4baab71335b70ec89b0@aadhyaitservices.com> <0a485cc6d7d380327e7faa530a5a273e@aadhyaitservices.com> <9009f1b12c8974eade636e9dfe22e311@aadhyaitservices.com> <2dc45c2bb2cff2548f7d2f55c3612f93@aadhyaitservices.com> <9ad4bb75e75fdbdef11b68a5f4af7d58@aadhyaitservices.com> <6d0a161f23ffd7c628739b7174d7aedc@aadhyaitservices.com> <176b2d9c80032a19d6944570df2ef470@aadhyaitservices.com> <4ccdfb571fcde1200579aed8ab24c70f@aadhyaitservices.com> <0875648dae66f085bb59cc52fece8ce8@aadhyaitservices.com> <4cec1078cc36bd7ff1544c1f6f2ee31f@aadhyaitservices.com> <3d0c3754bb3ede9a014edfa6256917b8@aadhyaitservices.com> <56e56ef6674b00826e38e4c0bfc4acd4@aadhyaitservices.com> <9fb7116256564cd66dab3b2e28477301@aadhyaitservices.com> <68127e27c8e4f9db0a8496af1152a666@aadhyaitservices.com> <37a35639be0ea0b714db091861d5a31c@aadhyaitservices.com> <4a057b39979579b6f62cfe2e801c649f@aadhyaitservices.com> <4cb109818db33353463c2a893a716d39@aadhyaitservices.com> <56ce664a976bbb99131500f4b89a06e6@aadhyaitservices.com> <415bbdfa6c89ef95e6e0136fd1b67efa@aadhyaitservices.com> <4f5a3db0c4fc70268cd472538af29f4d@aadhyaitservices.com> <8f1b4171e8779aec9e0914562e130fe9@aadhyaitservices.com> <208f4e683de1dae8f17af66ed04271a5@aadhyaitservices.com> <1dc7f2913c4b766a623e6bbf7ae2d0a8@aadhyaitservices.com> <15e0dbe8fba7a2ff8011b8c1ddd46102@aadhyaitservices.com> <09b2c94c3d6e5979c566fbea980e42cd@aadhyaitservices.com> <81550df6f5c1ebe15d9a7f0a310ceb0c@aadhyaitservices.com> <59f1e0242ff44d103fb4ed708ca723aa@aadhyaitservices.com> <9f2c39dffc7688b6ae5d7a2d5d7259f8@aadhyaitservices.com> <9b2e96e02b4d3b8792c4db3ebf934215@aadhyaitservices.com> <55ce969f6cae9d887ba2f8633af82a72@aadhyaitservices.com> <5c90d705886ec2bcc315afc42d239b24@aadhyaitservices.com> <8aa2a23040fcf3379248f8aa2d96350a@aadhyaitservices.com> <260023db6faeed0191e53f9b0390a8f9@aadhyaitservices.com> <585712db499187ade7f1bd3d2144e614@aadhyaitservices.com> <7a9193b15ae54a59bea0f6e77893890e@aadhyaitservices.com> <25cb0ed26e875128420ea14fc99363ff@aadhyaitservices.com> <153efb49541c56f449f6e67fe457978f@aadhyaitservices.com> <23530bd32ebab5aefb88d69877bacb9e@aadhyaitservices.com> <9296d03c32bfd1f3470ea042f4134cb9@aadhyaitservices.com> <5476d84229ec2ee6aa6409e30848edef@aadhyaitservices.com> <3ec0ea788f1e25519a99e9653ab8a1de@aadhyaitservices.com> <2d0bf25c537ab46444393771aa0c66df@aadhyaitservices.com> <35fb854f22174ad78342c2c74ba11127@aadhyaitservices.com> <9aeca749bd6c8fc2f9a440f69e32bf9f@aadhyaitservices.com> <1e8156909a35a66fba957d32c8c135ec@aadhyaitservices.com> <7e980b73db12413f7c18b99c2d922cdb@aadhyaitservices.com> <0d21b0b810df61618687d0d1618da8f6@aadhyaitservices.com> <1b0a7a8c6ea93d46de37f1c16adfc6f7@aadhyaitservices.com> <141ad31340c6bebb2be9930bf45d9105@aadhyaitservices.com> <53d3c52db67cb5bccba38e22cfea5fed@aadhyaitservices.com> <08dc09c3ca9fc7df9d447f91d69fc613@aadhyaitservices.com> <289551f19d9382e6177ef37ad6dcce23@aadhyaitservices.com> <1c663b190cad7ad1d7460399c18407c0@aadhyaitservices.com> <2c2e2358271e34ee868d5271f3b4252f@aadhyaitservices.com> <724374ebc77e7f83fdf23ca9dde6cb7a@aadhyaitservices.com> <1cd74b6d808f9f87aaddeb3e466ee13e@aadhyaitservices.com> <1fd123e9578b8edc5105fc647b47a3cc@aadhyaitservices.com> <3c2c697e75aafe565fe8fc67ecc99110@aadhyaitservices.com> <50910d1d5005c09b13e9807886efc8ce@aadhyaitservices.com> <5d838e781ecd49408b8bffff04d47185@aadhyaitservices.com> <6b91055f4dca093ba3683c854ce024b8@aadhyaitservices.com> <7090fba635fe280cd6d85eec1fcdebd1@aadhyaitservices.com> <119ba7acc951176876412edb253f0910@aadhyaitservices.com> <48fa9a43c17079ea22ca88dad131be9b@aadhyaitservices.com> <8aa90af4c9cd25c5da84c717eba7564f@aadhyaitservices.com> <93b1689b3c7bd1f8642c7f2214b752cb@aadhyaitservices.com> <6ccf9befc3ef76479a618b91d3a76823@aadhyaitservices.com> <189bc1ff8a682b3f1fed8f15de24c5ad@aadhyaitservices.com> <0c9be9454cdaba4c1da34f1101008016@aadhyaitservices.com> <2b059dec98632573f8f45848c9fe3047@aadhyaitservices.com> <0fe9ce87f13958c76045e76a3c9adc62@aadhyaitservices.com> <3e4184fe2e0a01cbd349ca1f4d136209@aadhyaitservices.com> <819cc10a8c76e360198e40f3e10073c5@aadhyaitservices.com> <1145c3334000d6cde3ac177c584fcc95@aadhyaitservices.com> <51e18da1eb52a7e558d5ba519e65c8b4@aadhyaitservices.com> <7e65586ae3548bcd5df1f7d343a9cb9e@aadhyaitservices.com> <3dd74d10b9a006eaf27c095e8add3746@aadhyaitservices.com> <8da1a7c5fd6e5c312955265a8b7a1a97@aadhyaitservices.com> <30b4c52c7698885139e1194d09cce1d1@aadhyaitservices.com> <0dae9708e16e1f8255c7b819d0a5c96d@aadhyaitservices.com> <7c0af58dc8599ce5015d7ae8d1730044@aadhyaitservices.com> <357a19221e42dc240ae4e5afb1ff190f@aadhyaitservices.com> <296ddfb7be19880c269c415dba2fc88e@aadhyaitservices.com> <0ed182af372b3d52dc06dec3ad631184@aadhyaitservices.com> <7a425ae8f894c16b01a1fec6dd4cfa10@aadhyaitservices.com> <6971cab782a5a2a1378e9d925f299aaa@aadhyaitservices.com> <238e0b0e045fbb2e73c9a724f2cd3478@aadhyaitservices.com> <2c5f228fc77e43794c9610c85d2ba5fe@aadhyaitservices.com> <9ac9951822f2ea6239602e0148387188@aadhyaitservices.com> <0d274e30587f5af1506b2cdb2c30bd7f@aadhyaitservices.com> <6c8802b76a1eebc2c209d772916584ed@aadhyaitservices.com> <93080092c12138889a692a464a0d0dcf@aadhyaitservices.com> <2a1de6d5a9075d147e7ffb04f9583ec6@aadhyaitservices.com> <5755f7655363cb4e74c7421063a8e1ba@aadhyaitservices.com> <469499d413652898fb099893d4a64871@aadhyaitservices.com> <16fff719e5869c10c45e82a75410addc@aadhyaitservices.com> <992813e906d5276b1a3b8202b99cc9d8@aadhyaitservices.com> <5ffd699f39e5a09a0513faf9a8082134@aadhyaitservices.com> <7e812ea7d7f4e5df4b174adba31c087b@aadhyaitservices.com> <2127e73d86db3ce7b1c534bf6278e3b1@aadhyaitservices.com> <718f08de5fc8f7ae98010a26bf51a2f1@aadhyaitservices.com> <1156a011f5737ec4bb5f619d6ff03069@aadhyaitservices.com> <540fcc672b64c757c8180ed82d0abb54@aadhyaitservices.com> <96a7893f9957a414c17c1c7b240ce83c@aadhyaitservices.com> <0e2160117242c9e3a97ee3b212d950f7@aadhyaitservices.com> <91e10aa5b2a91ee15d67bce17055576b@aadhyaitservices.com> <1513a4f57b1e24cfb339c86add1fb6ee@aadhyaitservices.com> <6f2de0f8c08697cf7d124f5334564ef5@aadhyaitservices.com> <57af4be1b41c8a908614536965552480@aadhyaitservices.com> <2c55a06af3ee5baea7b8e665dedc1700@aadhyaitservices.com> <0f0bd5f3373f79cccf47d31447173a07@aadhyaitservices.com> <4161889e9542fcbaded596a27733aec0@aadhyaitservices.com> <684100c495e00029a870282fcd3ad55b@aadhyaitservices.com> <91852b02741ee5fbe682cdb50b8ef0de@aadhyaitservices.com> <114da817022c4504a696fdad551035ef@aadhyaitservices.com> <42b26e79f9fe8acfe1325cb1b76294ff@aadhyaitservices.com> <6848e8e9c977da19a7fea2ccef2a5504@aadhyaitservices.com> <785f07511e47219c7c3f17149270b93c@aadhyaitservices.com> <86a8e564bb869b3dc31881dadba83d6b@aadhyaitservices.com> <3447d7c576b3dfbb8080ca191eb0b825@aadhyaitservices.com> <808001726a7135a362710a42191df487@aadhyaitservices.com> <2e56519b5cc44f57ff4cdadb398113a2@aadhyaitservices.com> <202f5827f4bada4a4b2d7bd599be92d9@aadhyaitservices.com> <0bb72936671b60df287b7446c8408473@aadhyaitservices.com> <0dca09b739c4aa33e81515509df2749e@aadhyaitservices.com> <6a839b83a7e0c91cc53c91e13c8dc687@aadhyaitservices.com> <879310313eee250211d44b6e6227c395@aadhyaitservices.com> <777a3d7bbc07c7848f26a1891f5b3009@aadhyaitservices.com> <1708ba82737697cce5998732521d260a@aadhyaitservices.com> <0210eb4b585847b00710308f3baa80f7@aadhyaitservices.com> <5e91da258b46ba3f7475ae8d92459b14@aadhyaitservices.com> <8aa00f7818ecf39845a87d5ea1747d4f@aadhyaitservices.com> <6a5e01fe28251804cf3a806bcba8404d@aadhyaitservices.com> <1b7426b6e0f18df5dcafa1c7156d3ca9@aadhyaitservices.com> <8fa1da7446ff912bef36a7f888fbef42@aadhyaitservices.com> <0f6925f3072e55d1e48aa5fb275bf652@aadhyaitservices.com> <81fa96657fd2adc131e2e552c685bbb5@aadhyaitservices.com> <6ec14413ed42328c5e2941060aac6a2f@aadhyaitservices.com> <4a524fbdf12d98fc73c266684eb2059c@aadhyaitservices.com> <9a89229909a6cc1e495f4efb074aa1d0@aadhyaitservices.com> <0512c7931c9b36c98bb08ca6bf270f99@aadhyaitservices.com> <19665700c8be886bc45eece7dcfcd7f1@aadhyaitservices.com> <40b17f457511d7f69c667269b4637cf5@aadhyaitservices.com> <4397ebd69d3485335db41e9d22bb703d@aadhyaitservices.com> <7604570d179e6da9a976cd90ec567a85@aadhyaitservices.com> <49eae105066e97c4ef9dc8672dda891c@aadhyaitservices.com> <7dac3593ad54431c6856925f5ca547cb@aadhyaitservices.com> <1d9b9324568da1ebfd52c5af94d6a742@aadhyaitservices.com> <491f8de0fde95397e80b3795cd9a1b7d@aadhyaitservices.com> <8a9a6401ccc0e3af69fdabaec8aa40e5@aadhyaitservices.com> <063858b46efa75a27114c57a09affa56@aadhyaitservices.com> <18631a30358278892d0568bfbaad7b82@aadhyaitservices.com> <3d5832ac5a52bcc6cd94ba31a0d1bd90@aadhyaitservices.com> <9b87ca0617a02d36c9bfe835fc59e62c@aadhyaitservices.com> <4416f5356ed6da9416976b857c0edb70@aadhyaitservices.com> <4d4b69cd61d4a4e5b89395effe27f663@aadhyaitservices.com> <47ebc2baa9b43305d83e9a06f5024345@aadhyaitservices.com> <696376d23b27b7eee597dfae4894061a@aadhyaitservices.com> <20059ed677277148c54fe5aed6a7a439@aadhyaitservices.com> <979e096c235b0767b752f10e7932b9d5@aadhyaitservices.com> <4e077ff1724b62d9b0f1eabcd1fc8570@aadhyaitservices.com> <08bdbe8040b24b2ede13a7cbd769ae82@aadhyaitservices.com> <5fb6cfd0b5ec2635ce01574353b1ba1a@aadhyaitservices.com> <6a6aa6b94b205d9494699dd95660670f@aadhyaitservices.com> <50e51f3f7a744fb20d7e06a88923c612@aadhyaitservices.com> <5010df73f10bc8d2c2cacf3be98a47e6@aadhyaitservices.com> <4a52b387f485044f56ddc4ff2d76d4f0@aadhyaitservices.com> <5cdb0a470618155099a414b8ffb606bb@aadhyaitservices.com> <8f50e56a721a6a1ff5aa3f2c71b5a60f@aadhyaitservices.com> <3fd8cf46d0001b5123a4fd39de3e6044@aadhyaitservices.com> <90717cb05548123d86c863c989ba3b2f@aadhyaitservices.com> <4b83e428d6c1ca909356e72177087d48@aadhyaitservices.com> <2fdc223c05ab38dfe1324b07c9165927@aadhyaitservices.com> <88f5a564bb9b931dc369a082c8c34800@aadhyaitservices.com> <07aa58bb913ecd09a49b7904b381031e@aadhyaitservices.com> <678e0e1803479b06bbb8add9aab773a5@aadhyaitservices.com> <2c33454befd2f3b76b76a99e86341a8e@aadhyaitservices.com> <9942ba84f2928de39c4ba24137fcb4ef@aadhyaitservices.com> <4419458223374199d2f281e81eefd810@aadhyaitservices.com> <9ff11a59663359b0d2d1fd5ba171ae9c@aadhyaitservices.com> <9302503517dc44f9853b171387a66287@aadhyaitservices.com> <9a86fc979788a7862e76de0804f8a3cc@aadhyaitservices.com> <29b85805990bd6c0a87db5770b418e41@aadhyaitservices.com> <04c7096a614ffbc3e03ff1ebbcf98935@aadhyaitservices.com> <0438e750f81fa25871f82eb88b3a5b49@aadhyaitservices.com> <41bd923f7f9e773a24e7f0fd28145f8f@aadhyaitservices.com> <85dfb21c4626f8b4552a1d9acbe5f214@aadhyaitservices.com> <9b8af41a03d7fc7cbddffad4ef7074e9@aadhyaitservices.com> <8f9f1ae28e0ed682ab3bf6020aa714ed@aadhyaitservices.com> <58dbbb32d41861e5e2a8372b0462fe3e@aadhyaitservices.com> <36e3a201bf73c90c522a6d7781d22173@aadhyaitservices.com> <4851a85f7c46ac084650a3d8bfe97ae7@aadhyaitservices.com> <37b978f9648c387403466b722725e146@aadhyaitservices.com> <7d2511b79d3e46f5f4dc6bdbb7098020@aadhyaitservices.com> <8d430e2f82a0f47380696779b49c2df0@aadhyaitservices.com> <879bc2d8496cefcddb7966bd4f6ce9ef@aadhyaitservices.com> <82cae4c2110650988e9deb0e5eab2176@aadhyaitservices.com> <4dba683e2effc551c796b4f099bb43e2@aadhyaitservices.com> <9ef7f7c18c27ac36c218939f9a315554@aadhyaitservices.com> <76ecf00ee9445f7c33fc8c5a90194f24@aadhyaitservices.com> <43aabe8c8b398494e6b72d7fc0132e11@aadhyaitservices.com> <290e2b8bb2217d04ee72c0a6c629f2d1@aadhyaitservices.com> <6b95f5f1107218c9f806d23dcf9a6249@aadhyaitservices.com> <64fec6ad4802538c237d12ecc8106b69@aadhyaitservices.com> <9613ac459ad25270c89ffa28d667e139@aadhyaitservices.com> Message-ID: X-Sender: work7@aadhyaitservices.com User-Agent: Roundcube Webmail/1.3.7 X-Authenticated_sender: work7@aadhyaitservices.com X-OutGoing-Spam-Status: No, score=-1.0 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - md-in-36.webhostbox.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - aadhyaitservices.com X-Get-Message-Sender-Via: md-in-36.webhostbox.net: authenticated_id: work7@aadhyaitservices.com X-Authenticated-Sender: md-in-36.webhostbox.net: work7@aadhyaitservices.com X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 0492987227 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aadhyaitservices.com header.s=default header.b=J+Br0cuD X-Spamd-Result: default: False [-2.04 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_X_SOURCE(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[aadhyaitservices.com:+]; MX_GOOD(-0.01)[aadhyaitservices.com]; NEURAL_HAM_SHORT(-0.36)[-0.361,0]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.05)[asn: 394695(0.33), country: US(-0.06)]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:394695, ipnet:103.21.58.0/23, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; HAS_X_AS(0.00)[work7@aadhyaitservices.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.88)[-0.882,0]; R_DKIM_ALLOW(-0.20)[aadhyaitservices.com:s=default]; URL_IN_SUBJECT(0.40)[freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.94)[-0.945,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[aadhyaitservices.com]; RCPT_COUNT_ONE(0.00)[1]; FROM_NO_DN(0.00)[]; HAS_X_GMSV(0.00)[work7@aadhyaitservices.com]; R_SPF_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2019 06:59:58 -0000 Hi, ​​ How are you? Current working for a​ ​client who is interested in buying paid advertising articles with links on​ ​good business, news , casino forex and other relevant sites. So, if you are interested in selling article placement on sites so please ​send me your websites URL with price to have an article with my website​ ​links. Looking for a good business relationship. Other than that b​​eing a freelancer The provided services by me are: · Content writing, · Web Design/HTML/Animation · SEO, SMM, · Media Buying, · Facebook, · Facebook likes, · Twitter working, I have 10 years experience in this field. Currently I am searching fo​r some more clients.​ ​Please let me know if you are interested in my services.​ ​Looking forward to hearing from you. Thanks And Regards Shweta Seo, Smo, Content Writing And Web Designing Service Provider Contact: Skpye: Seo1seo1 Contact Number: +919993036692 From owner-freebsd-stable@freebsd.org Sun Apr 28 19:52:26 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9FB981587BC6 for ; Sun, 28 Apr 2019 19:52:26 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-yw1-xc32.google.com (mail-yw1-xc32.google.com [IPv6:2607:f8b0:4864:20::c32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A63BF72DAD for ; Sun, 28 Apr 2019 19:52:24 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-yw1-xc32.google.com with SMTP id a62so3002850ywa.4 for ; Sun, 28 Apr 2019 12:52:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:mime-version:thread-index :content-language; bh=NSTIbpCa7+w7r4ylaTWEYB4Lnu1Q8Sut3YEI09iRwis=; b=U35id6EIDOJ98JNETYQVJnwYNBTY9lipIFKtdZoYj5TnKN/G9mKgqk4O76BXMv3t9a BbeVX+gJZnw2KNm6ydLRKBq+dthp/Xs/Xg2s44XF4ZI1hdbzmpu5d2WEZBzKfgqiqfrJ eYYAEVm+ai7uRdL9vpA3DtIHt1rtrAZSIKErlwipUPIcCLPA8/wPe87TJzGfSlZob/Em hKI9jNTmUzS7tB3WCNReN6pWoSDO2sNhAC2lWnSVOkzhpxDN/OCi69PW5cGSnjjmkBGC JBgK/3Ydt4qVdeu9Hf95f/Me5tTwnInaNORJnO6OqhtPdSWqJQV2aVE1rvoy3l/uuj8a JeBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :thread-index:content-language; bh=NSTIbpCa7+w7r4ylaTWEYB4Lnu1Q8Sut3YEI09iRwis=; b=Hg262nvAhVehaJHMrtz60xcBoB4ZbjFHTuU2282GPHuFF9ed6UTq8THQV/ovJCmz4k mOeecgDNXKM9brRycEwp0lOauP2b3uxcEc40xmyvdZy/GxARddVcpl77jRMjlXcXM4B8 VTqOxBKgmFz665MDVwSIiL17Mamd8KTDsbLBIvqWK0HGJwtQWmCbGOdIbTnBDHpB4McQ 8RaeHfugKBktpcE3DbiDxC5YxwLHnT9X2CIQEWbd8bJcAtIG5lFlkO9l0zHV++CpuitU lmNTILwMPcq/jNn4T53W89qA+Vbuy++AHmDvR4Gp7mO97uZRs6Nm/8asWUap+yqQewH6 r3lg== X-Gm-Message-State: APjAAAWVTmMIpoDPTzrMP+PkahJLYfELASSqyFJ2/mJMui5N/qZ3ygs2 fZpqAhDeNLsxrk77ZzMOtX/aMQ== X-Google-Smtp-Source: APXvYqxWeBYspJ9/XEuoo3/JUvn0hoC2eoW+jJZT5W0y3roTKwp34/qp77pkDEtj4C5/0zz1Iq31jA== X-Received: by 2002:a81:a104:: with SMTP id y4mr25595960ywg.76.1556481143008; Sun, 28 Apr 2019 12:52:23 -0700 (PDT) Received: from KrisWindows (71-136-150-27.lightspeed.knvltn.sbcglobal.net. [71.136.150.27]) by smtp.gmail.com with ESMTPSA id w3sm2969049ywc.45.2019.04.28.12.52.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 28 Apr 2019 12:52:22 -0700 (PDT) From: To: , , , , , Subject: CFT: FreeBSD Package Base Date: Sun, 28 Apr 2019 15:52:21 -0400 Message-ID: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdT9+kYS/QrrZeloRTSTN/rqnNT4jQ== Content-Language: en-us X-Rspamd-Queue-Id: A63BF72DAD X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=U35id6EI; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::c32 as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-6.62 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-2.64)[ip: (-7.69), ipnet: 2607:f8b0::/32(-3.19), asn: 15169(-2.27), country: US(-0.06)]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; FROM_NO_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.3.c.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; NEURAL_HAM_SHORT(-0.97)[-0.966,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2019 19:52:26 -0000 FreeBSD Community, I'm pleased to announce a CFT for builds of FreeBSD 12-stable and 13-current using "TrueOS-inspired" packaged base. These are stock FreeBSD images which will allow users to perform all updating via the 'pkg' command directly. Rather than trying to answer all questions in this announcement, we've created a FAQ page with more details. Please refer to this page, and let us know if you have additional questions that we can include on that page going forward. Additionally, I will be hosting a Package Base working group at BSDCan 2019, and welcome user and developer attendance to discuss this and other ongoing package work: https://wiki.freebsd.org/DevSummit/201905/PackageBase FAQ ------------------------------------------------------------- https://trueos.github.io/pkgbase-docs/ Download Links ------------------------------------------------------------- FreeBSD 12-STABLE: https://pkg.trueos.org/iso/freebsd12-pkgbase/ FreeBSD 13-CURRENT: https://pkg.trueos.org/iso/freebsd-pkgbase/ -- Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source From owner-freebsd-stable@freebsd.org Sun Apr 28 22:02:07 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A52E158B797 for ; Sun, 28 Apr 2019 22:02:07 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4653777A95 for ; Sun, 28 Apr 2019 22:02:06 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 0F05221109F; Sun, 28 Apr 2019 18:01:34 -0400 (EDT) Received: from [192.168.10.28] (D18.Denninger.Net [192.168.10.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 589BBC248; Sun, 28 Apr 2019 17:01:34 -0500 (CDT) Subject: Re: Concern: ZFS Mirror issues (12.STABLE and firmware 19 .v. 20) [[UPDATE w/more tests]] To: freebsd-stable@freebsd.org References: <9a96b1b5-9337-fcae-1a2a-69d7bb24a5b3@denninger.net> <1866e238-e2a1-ef4e-bee5-5a2f14e35b22@denninger.net> <3d2ad225-b223-e9db-cce8-8250571b92c9@FreeBSD.org> <2bc8a172-6168-5ba9-056c-80455eabc82b@denninger.net> <2c23c0de-1802-37be-323e-d390037c6a84@denninger.net> <864062ab-f68b-7e63-c3da-539d1e9714f9@denninger.net> <6dc1bad1-05b8-2c65-99d3-61c547007dfe@denninger.net> <758d5611-c3cf-82dd-220f-a775a57bdd0b@multiplay.co.uk> <3f53389a-0cb5-d106-1f64-bbc2123e975c@denninger.net> <8108da18-2cdd-fa29-983c-3ae7be6be412@multiplay.co.uk> Cc: Steven Hartland From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <5ff09821-98e2-8520-1a46-e6541c9c8891@denninger.net> Date: Sun, 28 Apr 2019 17:01:34 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <8108da18-2cdd-fa29-983c-3ae7be6be412@multiplay.co.uk> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms010203060509060908070707" X-Rspamd-Queue-Id: 4653777A95 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-5.91 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(-2.51)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.08), asn: 14061(1.49), country: US(-0.06)]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; DMARC_NA(0.00)[denninger.net]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: px.denninger.net]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.19)[-0.194,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2019 22:02:07 -0000 This is a cryptographically signed message in MIME format. --------------ms010203060509060908070707 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 4/20/2019 15:56, Steven Hartland wrote: > Thanks for extra info, the next question would be have you eliminated > that corruption exists before the disk is removed? > > Would be interesting to add a zpool scrub to confirm this isn't the > case before the disk removal is attempted. > > =C2=A0=C2=A0=C2=A0 Regards > =C2=A0=C2=A0=C2=A0 Steve > > On 20/04/2019 18:35, Karl Denninger wrote: >> >> On 4/20/2019 10:50, Steven Hartland wrote: >>> Have you eliminated geli as possible source? >> No; I could conceivably do so by re-creating another backup volume >> set without geli-encrypting the drives, but I do not have an extra >> set of drives of the capacity required laying around to do that.=C2=A0= I >> would have to do it with lower-capacity disks, which I can attempt if >> you think it would help.=C2=A0 I *do* have open slots in the drive >> backplane to set up a second "test" unit of this sort.=C2=A0 For reaso= ns >> below it will take at least a couple of weeks to get good data on >> whether the problem exists without geli, however. >> Ok, following up on this with more data.... First step taken was to create a *second* backup pool (I have the backplane slots open, fortunately) with three different disks but *no encryption.* I ran both side-by-side for several days, with the *unencrypted* one operating with one disk detached and offline (pulled physically) just as I do normally.=C2=A0 Then I swapped the two using the same paradigm. The difference was *dramatic* -- the resilver did *not* scan the entire disk; it only copied the changed blocks and was finished FAST.=C2=A0 A subsequent scrub came up 100% clean. Next I put THOSE disks in the vault (so as to make sure I didn't get hosed if something went wrong) and re-initialized the pool in question, leaving only the "geli" alone (in other words I zpool destroy'd the pool and then re-created it with all three disks connected and geli-attached.)=C2=A0 The purpose for doing this was to eliminate the possibility of old corruption somewhere, or some sort of problem with multiple, spanning years, in-place "zpool upgrade" commands.=C2=A0 Then I= ran a base backup to initialize all three volumes, detached one and yanked it out of the backplane, as would be the usual, leaving the other two online and operating. I ran backups as usual for most of last week after doing this, with the 61.eli and 62-1.eli volumes online, and 62-2 physically out of the backplane. Today I swapped them again as I usually do (e.g. offline 62.1, geli detach, camcontrol standby and then yank it -- then insert the 62-2 volume, geli attach and zpool online) and this is happening: [\u@NewFS /home/karl]# zpool status backup =C2=A0 pool: backup =C2=A0state: DEGRADED status: One or more devices is currently being resilvered.=C2=A0 The pool= will =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 continue to function, possibly= in a degraded state. action: Wait for the resilver to complete. =C2=A0 scan: resilver in progress since Sun Apr 28 12:57:47 2019 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2.48T scanned at 202M/s, 1.89T= issued at 154M/s, 3.27T total =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1.89T resilvered, 57.70% done,= 0 days 02:37:14 to go config: =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 NAME=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 STATE=C2=A0=C2=A0=C2=A0=C2=A0 READ WRITE CKSUM =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 backup=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 DEGRADED=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0= 0=C2=A0=C2=A0=C2=A0=C2=A0 0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 mirror-0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 DEGRADED=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2= =A0=C2=A0=C2=A0 0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 gpt/ba= ckup61.eli=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 112953= 90187305954877=C2=A0 OFFLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2= =A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0 was /dev/gpt/backup62-1.eli =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 gpt/ba= ckup62-2.eli=C2=A0=C2=A0=C2=A0 ONLINE=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= 0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0 0 errors: No known data errors The "3.27T" number is accurate (by "zpool list") for the space in use. There is not a snowball's chance in Hades that anywhere near 1.89T of that data (thus far, and it ain't done as you can see!) was modified between when all three disks were online and when the 62-2.eli volume was swapped back in for 62.1.eli.=C2=A0 No possible way.=C2=A0 Maybe some= 100-200Gb of data has been touched across the backed-up filesystems in the last three-ish days but there's just flat-out no way it's more than that; this would imply an entropy of well over 50% of the writeable data on this box in less than a week!=C2=A0 That's NOT possible.=C2=A0 Further= it's not 100%; it shows 2.48T scanned but 1.89T actually written to the other driv= e. So something is definitely foooged here and it does appear that geli is involved in it.=C2=A0 Whatever is foooging zfs the resilver process think= s it has to recopy MOST (but not all!) of the blocks in use, it appears, from the 61.eli volume to the 62-2.eli volume. The question is what would lead ZFS to think it has to do that -- it clearly DOES NOT as a *much* smaller percentage of the total TXG set on 61.eli was modified while 62-2.eli was offline and 62.1.eli was online. Again I note that on 11.1 and previous this resilver was a rapid operation; whatever was actually changed got copied but the system never copied *nearly everything* on a resilver, including data that had not been changed at all, on a mirrored set. Obviously on a Raidz volume you have to go through the entire data structure because parity has to be recomputed and blocks regenerated but on a mirror only the changed TXGs need to be looked at and copied.=C2=A0 = TXGs that were on both disks at the time second one was taken offline do not need to be touched *at all* since they're already on the target! What's going on here? --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms010203060509060908070707 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNDI4MjIwMTM0 WjBPBgkqhkiG9w0BCQQxQgRA8nlIqlOwwJzfgg5WD++s5jX8D4IWTIUB2ukTKR0bFpH/4VMp 4Wc8XGz18G223uE/e8ZLCzoMyy5CN0JDww1rSzBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgBKfj/vfiGIpdLjxtar29IxjaPfneLQB2T7JoBLL9NkBhjNSLxkbdVCHiUg5oF8E6ZH VTkVlYskBI6xlPvVF1twm+THd2FF2fYAA0P5pA3pU5po3e/khKU6+8ZuxLwV9in2rxVTcixX XQ5UhqQbok2PfB0heoIlt3LqH87WCjKh5QdKfFNKR4tmpR8khXgjHYDW7nyQl0NywKmYAhC9 dQfruHEkrXXsV5gRtpS2+X6JbcJr8UgHZ+e1p+t8fVNhuhWNEiNgwevUW5MZSF/065E7AXyX nKmaDnOfJTzODjl3qK4PWziUykSWMa9EjPQctDkCWtPLEJJTTfLcTYzqDKICa5Z9hoFScVFO CUJDVdH9PsisiX80ha2sOP+qBNsv4H+l9xTi1bj8pUJR67ugOvl2UXBVMMjMGBhxXNiRD0gZ uU4UOWOtIxgJpYQBH/swpVPlTZhR05NglPRRmD/qNFdIG3FJ1Ne/0lZ0E9RDtk3dUvhoKKzB QZLiFPRkqnHRuWuTzSLZNaNkpu+dH2ry018Vxeme6pkCnGZ195IB1QRIq/fIREivMfIaRRph NuWLj1PfSeJCXxJDgpHIPlwCCGzPZXKH4Hqyha2R7RyQZhcBX7wBtTIn7wN+LrxD0btm34ni KVfRxnOmE7FyAv6CaV7fZ+s0EUXlR4jz8LS2k7Bx9AAAAAAAAA== --------------ms010203060509060908070707-- From owner-freebsd-stable@freebsd.org Sun Apr 28 22:21:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C5EE0158BE22 for ; Sun, 28 Apr 2019 22:21:39 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gromit.dlib.vt.edu", Issuer "Chumby Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B5B2C800BC for ; Sun, 28 Apr 2019 22:21:38 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from mather.gromit23.net (c-98-244-101-97.hsd1.va.comcast.net [98.244.101.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id BFB681A8; Sun, 28 Apr 2019 18:21:31 -0400 (EDT) Content-Type: text/plain; charset=us-ascii; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: CFT: FreeBSD Package Base From: Paul Mather In-Reply-To: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> Date: Sun, 28 Apr 2019 18:21:31 -0400 Cc: FreeBSD Stable Content-Transfer-Encoding: 7bit Message-Id: <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> To: kris@ixsystems.com X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: B5B2C800BC X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dmarc=fail reason="" header.from=vt.edu (policy=none) X-Spamd-Result: default: False [-3.73 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; IP_SCORE(-1.26)[ip: (-3.25), ipnet: 128.173.0.0/16(-1.62), asn: 1312(-1.36), country: US(-0.06)]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[vt.edu : No valid SPF, No valid DKIM,none]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[chumby.dlib.vt.edu,gromit.dlib.vt.edu]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.96)[-0.962,0]; R_SPF_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[97.101.244.98.zen.spamhaus.org : 127.0.0.10]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1312, ipnet:128.173.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2019 22:21:39 -0000 On Apr 28, 2019, at 3:52 PM, wrote: > FreeBSD Community, > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > 13-current > using "TrueOS-inspired" packaged base. These are stock FreeBSD images which > will allow users to perform all updating via the 'pkg' command directly. > Rather than trying to answer all questions in this announcement, we've > created a FAQ page with more details. Please refer to this page, and let us > know if you have additional questions that we can include on that page > going > forward. I currently keep my FreeBSD/arm and FreeBSD/arm64 systems up to date via PkgBase in FreeBSD 12. It works well for me (crossbuilding and hosting the PkgBase repository on a FreeBSD/amd64 system). What is the difference between the above CFT-created PkgBase and one created via "make packages" using the native build system (https://wiki.freebsd.org/PkgBase)? Looking at the FAQ you linked (https://trueos.github.io/pkgbase-docs/), it seems the above CFT system is less granular than the one currently produced via the in-tree "make packages" (which could be a good thing from a simplicity standpoint). Is there anything else? Is the above CFT-produced packages the system that will ultimately become the way packaged base is produced in FreeBSD 13.0-RELEASE, or is it just an alternative you want people to try out and evaluate? I guess I'm not clear what "TrueOS-inspired" packaged base means. :-) Cheers, Paul. From owner-freebsd-stable@freebsd.org Sun Apr 28 23:40:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A0D0158D5DA for ; Sun, 28 Apr 2019 23:40:43 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-yw1-xc31.google.com (mail-yw1-xc31.google.com [IPv6:2607:f8b0:4864:20::c31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F1C0682475 for ; Sun, 28 Apr 2019 23:40:41 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-yw1-xc31.google.com with SMTP id n188so510960ywe.2 for ; Sun, 28 Apr 2019 16:40:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-transfer-encoding:thread-index :content-language; bh=hQdPOA1LK6kkDcdPzBTVBaTCvAnedEsS7BMGx82t5kw=; b=G9BP/AThguVgKQHhTfLhzVutF9XEdZOeqkQGIQq/0BebIHH/vb+tSTjRBLGd4YJa42 l+ZqJVXtsSnqgkYFbCgR2Q7u45XSXvtvNuGGsqd20QT55iBcIwUHgr9lCSf67xWyA5er a5cePXWL6nMBAirBwjZVnR3GQtwdxG5dHh8dMtxeMOHCu8LcGfUMv8UfG3FrjHs0/DCU xMpUhKE2+qAlh4+oFsBGjXSYiWu2Qi9Ul0Tk6QEf7H6PX6iV5InQv9nOzx5y2ZVc5qW0 6/kA0y/REqsXuTXEOo44hIU4+Qk+p+BzloqBmi5ZdnyyjFwhFsvuHZjTWYHLafXyEjxO 8YDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:thread-index :content-language; bh=hQdPOA1LK6kkDcdPzBTVBaTCvAnedEsS7BMGx82t5kw=; b=ZqCU5yoKEhb6MR8YWhXHbDfGL/ogyJvApmyEGJ7+ZMdwi1zf0X8Jienj+68kayVgJN cG2dQPqndyQveR8o0l8Stf7cBkRNc5HY/kQdxVqhFIhx2fj8Tjc9L+QWcOOGZm+GeByX T3yoMZASTV4q324CJQonHqC35xDCl8f2mT18jp+6ikO4K8jiuJEJc6WTZGVGWHo4NQwE jKenWrMszUaJACTGDKQra27kkz4Ho43k0eST3cFqmmbIXcbf+X9ST6flp37OXOownCc4 yHKUvzb/Zz5m9Dv3DDJUyP7vI3YoSTJAJEwVqvH+RsJEpSQS/TpPq1TtZmHPUq0m9csx tkXQ== X-Gm-Message-State: APjAAAUhT2ebESgvpWzxqUqCuiikkvQp4Fv4qA1iZjoEm/TZmkPIS/gs yLpU6Osi0E7ZHoze0ANFOqNXwc1LeUYhBJm3SpQ7Sv4C3xtsj9XmYMv3bPtgLWeadYvmiU9nC1p oFL1tXoTltNjL1amRh3UfhC2gv4twuyQ/zWueisDFVg2sfWCwtIFHFHGS6MNooTrIkZuCUb6j X-Google-Smtp-Source: APXvYqwmCpKU7qR9z5OW49uvNCnKegQFLCejIDcTweYjYrPxCyRG515ivB9ID7SVN/F6qlLA1jBGAw== X-Received: by 2002:a81:1b52:: with SMTP id b79mr50105394ywb.285.1556494840880; Sun, 28 Apr 2019 16:40:40 -0700 (PDT) Received: from KrisWindows (71-136-150-27.lightspeed.knvltn.sbcglobal.net. [71.136.150.27]) by smtp.gmail.com with ESMTPSA id o2sm5371748ywj.0.2019.04.28.16.40.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 28 Apr 2019 16:40:40 -0700 (PDT) From: To: "'Paul Mather'" Cc: "'FreeBSD Stable'" References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> In-Reply-To: <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> Subject: RE: CFT: FreeBSD Package Base Date: Sun, 28 Apr 2019 19:40:39 -0400 Message-ID: <00ae01d4fe1b$c9a93a30$5cfbae90$@ixsystems.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQFBRupbev+o6AHqgfoJLZwOgBKxNQKcVF00p2TEBFA= Content-Language: en-us X-Rspamd-Queue-Id: F1C0682475 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=G9BP/ATh; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::c31 as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-6.95 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-2.95)[ip: (-9.21), ipnet: 2607:f8b0::/32(-3.19), asn: 15169(-2.27), country: US(-0.06)]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; RCVD_IN_DNSWL_NONE(0.00)[1.3.c.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_NO_DN(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-0.998,0]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Apr 2019 23:40:43 -0000 Paul, We also started with the current in-base package system, but ultimately = abandonded it for a variety of reasons that made unsuitable for us to = use in FreeNAS. Some of those were the chaos with having so many = hundreds of packages, and the issues when you started combining them = with various WITH_*/WITHOUT_* flags. This system doesn't touch the = FreeBSD build and instead allows building source / packages directly = from ports and poudriere. (Keeping base and ports in sync far easier) "TrueOS-inspired" was just reference to the fact we've been incubating = this in TrueOS for about 6 months now to shake out some issues and test = our designs. The hope is to envigorate the FreeBSD project to have a = discussion about getting real base packages in a release sooner rather = than later, especially since current efforts seem somewhat stalled. = We're hoping the design we're using here is compelling enough that it = can be adopted in FreeBSD. Some cool asides, being able to 'pkg install src' and have /usr/src be = kept in sync with the current packages is super handy. =F0=9F=98=89 --=20 Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source -----Original Message----- From: Paul Mather =20 Sent: Sunday, April 28, 2019 6:22 PM To: kris@ixsystems.com Cc: FreeBSD Stable Subject: Re: CFT: FreeBSD Package Base On Apr 28, 2019, at 3:52 PM, = wrote: > FreeBSD Community, > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and=20 > 13-current using "TrueOS-inspired" packaged base. These are stock=20 > FreeBSD images which will allow users to perform all updating via the=20 > 'pkg' command directly. > Rather than trying to answer all questions in this announcement, we've = > created a FAQ page with more details. Please refer to this page, and=20 > let us know if you have additional questions that we can include on=20 > that page going forward. I currently keep my FreeBSD/arm and FreeBSD/arm64 systems up to date via = PkgBase in FreeBSD 12. It works well for me (crossbuilding and hosting = the PkgBase repository on a FreeBSD/amd64 system). What is the difference between the above CFT-created PkgBase and one = created via "make packages" using the native build system = (https://wiki.freebsd.org/PkgBase)? Looking at the FAQ you linked = (https://trueos.github.io/pkgbase-docs/), it seems the above CFT system = is less granular than the one currently produced via the in-tree "make = packages" (which could be a good thing from a simplicity standpoint). = Is there anything else? Is the above CFT-produced packages the system that will ultimately = become the way packaged base is produced in FreeBSD 13.0-RELEASE, or is = it just an alternative you want people to try out and evaluate? I guess = I'm not clear what "TrueOS-inspired" packaged base means. :-) Cheers, Paul. From owner-freebsd-stable@freebsd.org Mon Apr 29 01:29:56 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DE62158FC1C for ; Mon, 29 Apr 2019 01:29:56 +0000 (UTC) (envelope-from spork@bway.net) Received: from smtp2.bway.net (smtp2.v6.bway.net [IPv6:2607:d300:1::28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D7BDF8588E for ; Mon, 29 Apr 2019 01:29:54 +0000 (UTC) (envelope-from spork@bway.net) Received: from [192.168.1.7] (unknown [68.200.119.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: spork@bway.net) by smtp2.bway.net (Postfix) with ESMTPSA id CF3269587F; Sun, 28 Apr 2019 21:29:39 -0400 (EDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: CFT: FreeBSD Package Base From: Charles Sprickman In-Reply-To: <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> Date: Sun, 28 Apr 2019 21:29:37 -0400 Cc: kris@ixsystems.com, FreeBSD Stable Content-Transfer-Encoding: quoted-printable Message-Id: <34C3A77A-7B18-487E-9BC4-5B1087FEEBEA@bway.net> References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> To: Paul Mather X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: D7BDF8588E X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.94 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:d300:1::28/128]; MV_CASE(0.50)[]; DKIM_TRACE(0.00)[bway.net:+]; DMARC_POLICY_ALLOW(-0.50)[bway.net,quarantine]; MX_GOOD(-0.01)[mx2.bway.net,mx1.bway.net]; NEURAL_HAM_SHORT(-0.42)[-0.424,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-0.01)[country: US(-0.06)]; ASN(0.00)[asn:8059, ipnet:2607:d300::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; R_DKIM_ALLOW(-0.20)[bway.net:s=mail]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_LOW(-1.00)[bway.net.dwl.dnswl.org : 127.0.5.1]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 01:29:56 -0000 > On Apr 28, 2019, at 6:21 PM, Paul Mather = wrote: >=20 > On Apr 28, 2019, at 3:52 PM, = wrote: >=20 >> FreeBSD Community, >>=20 >>=20 >>=20 >> I'm pleased to announce a CFT for builds of FreeBSD 12-stable and = 13-current >> using "TrueOS-inspired" packaged base. These are stock FreeBSD images = which >> will allow users to perform all updating via the 'pkg' command = directly. >> Rather than trying to answer all questions in this announcement, = we've >> created a FAQ page with more details. Please refer to this page, and = let us >> know if you have additional questions that we can include on that = page going >> forward. >=20 >=20 > I currently keep my FreeBSD/arm and FreeBSD/arm64 systems up to date = via PkgBase in FreeBSD 12. It works well for me (crossbuilding and = hosting the PkgBase repository on a FreeBSD/amd64 system). >=20 > What is the difference between the above CFT-created PkgBase and one = created via "make packages" using the native build system = (https://wiki.freebsd.org/PkgBase)? Looking at the FAQ you linked = (https://trueos.github.io/pkgbase-docs/), it seems the above CFT system = is less granular than the one currently produced via the in-tree "make = packages" (which could be a good thing from a simplicity standpoint). = Is there anything else? >=20 > Is the above CFT-produced packages the system that will ultimately = become the way packaged base is produced in FreeBSD 13.0-RELEASE, or is = it just an alternative you want people to try out and evaluate? I guess = I'm not clear what "TrueOS-inspired" packaged base means. :-) What are the plans to get rid of the hellscape known as = =E2=80=9Cmergemaster=E2=80=9D? Is there anything exciting and new there = either in base or any of the ixSystems projects? Thanks, Charles >=20 > Cheers, >=20 > Paul. >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Apr 29 01:57:52 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2520A1590431 for ; Mon, 29 Apr 2019 01:57:52 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-yw1-xc2f.google.com (mail-yw1-xc2f.google.com [IPv6:2607:f8b0:4864:20::c2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DD2286478 for ; Mon, 29 Apr 2019 01:57:50 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-yw1-xc2f.google.com with SMTP id x204so3174744ywg.6 for ; Sun, 28 Apr 2019 18:57:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-transfer-encoding:thread-index :content-language; bh=iDg2WC1WHDUaMLc4WIsFE/WkKPn+Rf5VDzzmLPeinng=; b=m9dVHS6vgaIEEM3aOHnGXW7Z0Numu5jMhmEkrtwpZMKoPL/6TstzV1uPq9lRVT26x1 PGGfjhL4YZhxOpEbGQ5h24nBiVrC3CIFkMqh0QZY259tIHdHh9D5OIwBIwO/xT0BHdGv VyX8rKtCGAEakBFbmr4MWf875FNoWybzT5fIS2YFswl1b/jDOqvqSt9apTMTqu1Gxyas 6Uf0ZY9seBzk3ipJZx6+GyOlQSz9oeTvxMa5XKQ7G9mZ7LgG8lN2pEJb5N7bEuKbrgsU js9512cfywTGVqP2ij/MhdeN1KEfGGtJwktkLyhVY+VoWXagrWAmHCwl8IrkJJRO4NGw rrkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:thread-index :content-language; bh=iDg2WC1WHDUaMLc4WIsFE/WkKPn+Rf5VDzzmLPeinng=; b=bWoBoAr2LZDb4cr+hGt6eX6Cay23IeYCm5Gp1U5GI4W6o0rAOI6AnchQTLRERaM0o3 BXmayVAarCULdgfHQzgqomTMRfhUUrdZJsLknjATOAJeB3MtaenjIo9VkYtd7IdlTstB R9aQoEkEyEFY42YYnIHlTBaKRXwKcwVvRtGEdwyT13kZh5JY4ZNNz3WLMOIiqbwDem3K bU7/3RTenz6tJhQt1f6V91DnOzTusZhZKInY2kFCdgSrHuI2c3mBdJXBLbz9VE3RfgMc DwiQbva3nA0SiNPuJJSQ5zV0cxbHuhhQCvryq2RzrO4iGIrNnDzaeqzvwfLQA5PbPVeA 0v6w== X-Gm-Message-State: APjAAAX3o8gzkVh2uy7P0I1K4z9OLxdEpXfNhwXjMnjpI/etfFS/94zF omxw83kWsJuXaBiZhqB5d5pHXdWZcfRPARE+Wu02/6zSabt9fiSf/Y3AksADnE7ULhsim7oSexn Maj0zlDcN1MJVpXnYNLGwsAB15XZFr0peUgcnxYRuco/icDl6C94qlYRxd84+p1ae+wKij4j5 X-Google-Smtp-Source: APXvYqyA+JPi1gOgZ/Mw9zWoZgSEClJ+KfvZSxziJJj3qF+rzZLJmZcA4OWtnscI4fFj/iFqEKJqkw== X-Received: by 2002:a81:67c6:: with SMTP id b189mr39777430ywc.24.1556503069560; Sun, 28 Apr 2019 18:57:49 -0700 (PDT) Received: from KrisWindows (71-136-150-27.lightspeed.knvltn.sbcglobal.net. [71.136.150.27]) by smtp.gmail.com with ESMTPSA id n124sm10090675ywf.95.2019.04.28.18.57.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 28 Apr 2019 18:57:48 -0700 (PDT) From: To: "'Charles Sprickman'" , "'Paul Mather'" Cc: "'FreeBSD Stable'" References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> <34C3A77A-7B18-487E-9BC4-5B1087FEEBEA@bway.net> In-Reply-To: <34C3A77A-7B18-487E-9BC4-5B1087FEEBEA@bway.net> Subject: RE: CFT: FreeBSD Package Base Date: Sun, 28 Apr 2019 21:57:47 -0400 Message-ID: <012501d4fe2e$f21cc730$d6565590$@ixsystems.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQFBRupbev+o6AHqgfoJLZwOgBKxNQKcVF00Ap3I+oanT/7cUA== Content-Language: en-us X-Rspamd-Queue-Id: 9DD2286478 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=m9dVHS6v; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::c2f as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-7.00 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; FROM_NO_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[f.2.c.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; IP_SCORE(-3.00)[ip: (-9.46), ipnet: 2607:f8b0::/32(-3.19), asn: 15169(-2.27), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 01:57:52 -0000 Its on the agenda for next months Working Group. That and tooling to = help migrating to pkg base and keeping /etc files intact.=20 --=20 Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source -----Original Message----- From: Charles Sprickman =20 Sent: Sunday, April 28, 2019 9:30 PM To: Paul Mather Cc: kris@ixsystems.com; FreeBSD Stable Subject: Re: CFT: FreeBSD Package Base > On Apr 28, 2019, at 6:21 PM, Paul Mather = wrote: >=20 > On Apr 28, 2019, at 3:52 PM, = wrote: >=20 >> FreeBSD Community, >>=20 >>=20 >>=20 >> I'm pleased to announce a CFT for builds of FreeBSD 12-stable and=20 >> 13-current using "TrueOS-inspired" packaged base. These are stock=20 >> FreeBSD images which will allow users to perform all updating via the = 'pkg' command directly. >> Rather than trying to answer all questions in this announcement,=20 >> we've created a FAQ page with more details. Please refer to this=20 >> page, and let us know if you have additional questions that we can=20 >> include on that page going forward. >=20 >=20 > I currently keep my FreeBSD/arm and FreeBSD/arm64 systems up to date = via PkgBase in FreeBSD 12. It works well for me (crossbuilding and = hosting the PkgBase repository on a FreeBSD/amd64 system). >=20 > What is the difference between the above CFT-created PkgBase and one = created via "make packages" using the native build system = (https://wiki.freebsd.org/PkgBase)? Looking at the FAQ you linked = (https://trueos.github.io/pkgbase-docs/), it seems the above CFT system = is less granular than the one currently produced via the in-tree "make = packages" (which could be a good thing from a simplicity standpoint). = Is there anything else? >=20 > Is the above CFT-produced packages the system that will ultimately=20 > become the way packaged base is produced in FreeBSD 13.0-RELEASE, or=20 > is it just an alternative you want people to try out and evaluate? I=20 > guess I'm not clear what "TrueOS-inspired" packaged base means. :-) What are the plans to get rid of the hellscape known as = =E2=80=9Cmergemaster=E2=80=9D? Is there anything exciting and new there = either in base or any of the ixSystems projects? Thanks, Charles >=20 > Cheers, >=20 > Paul. >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list=20 > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Apr 29 10:01:08 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A58DF1586720 for ; Mon, 29 Apr 2019 10:01:08 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from mail.0x20.net (mail.0x20.net [46.251.251.56]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 168406DC66 for ; Mon, 29 Apr 2019 10:01:08 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from e.0x20.net (mail.0x20.net [46.251.251.56]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (3096 bits) server-digest SHA256) (No client certificate requested) by mail.0x20.net (Postfix) with ESMTPS id 689FEAD91A; Mon, 29 Apr 2019 12:01:06 +0200 (CEST) Received: (from lars@localhost) by e.0x20.net (8.15.2/8.15.2/Submit) id x3TA14w7077892; Mon, 29 Apr 2019 12:01:04 +0200 (CEST) (envelope-from lars) Date: Mon, 29 Apr 2019 12:01:04 +0200 From: Lars Engels To: Charles Sprickman Cc: Paul Mather , kris@ixsystems.com, FreeBSD Stable Subject: Re: CFT: FreeBSD Package Base Message-ID: <20190429100104.GX62338@e.0x20.net> References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> <34C3A77A-7B18-487E-9BC4-5B1087FEEBEA@bway.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <34C3A77A-7B18-487E-9BC4-5B1087FEEBEA@bway.net> X-Editor: VIM - Vi IMproved 8.0 User-Agent: Mutt/1.11.2 (2019-01-07) X-Rspamd-Queue-Id: 168406DC66 X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [1.19 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.69)[-0.689,0]; MX_INVALID(0.50)[cached]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.02)[0.021,0]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[0x20.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.34)[0.343,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[lars.engels@0x20.net,lars@e.0x20.net]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:31400, ipnet:46.251.251.0/24, country:DE]; FROM_NEQ_ENVFROM(0.00)[lars.engels@0x20.net,lars@e.0x20.net]; IP_SCORE(-0.18)[ip: (-1.26), ipnet: 46.251.251.0/24(-0.63), asn: 31400(0.97), country: DE(-0.01)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 10:01:08 -0000 On Sun, Apr 28, 2019 at 09:29:37PM -0400, Charles Sprickman via freebsd-stable wrote: > > > What are the plans to get rid of the hellscape known as “mergemaster”? > Is there anything exciting and new there either in base or any of the > ixSystems projects? There's /usr/sbin/etcupdate in base. From owner-freebsd-stable@freebsd.org Mon Apr 29 10:52:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6F641587F56 for ; Mon, 29 Apr 2019 10:52:53 +0000 (UTC) (envelope-from cmt@burggraben.net) Received: from smtp.burggraben.net (smtp.burggraben.net [IPv6:2a01:4f8:140:50a2::3:1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "ns.exwg.net", Issuer "Christoph Moench-Tegeder" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BA4766FB4B for ; Mon, 29 Apr 2019 10:52:52 +0000 (UTC) (envelope-from cmt@burggraben.net) Received: from localhost (localhost [127.0.0.1]) by smtp.burggraben.net (Postfix) with ESMTP id BF97D6002F8; Mon, 29 Apr 2019 12:52:50 +0200 (CEST) X-Spam-Scanned: by amavisd-new at exwg.net Received: from smtp.burggraben.net ([127.0.0.1]) by localhost (ns.burggraben.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HrVXlloLaCJm; Mon, 29 Apr 2019 12:52:41 +0200 (CEST) Received: from elch.exwg.net (elch.exwg.net [IPv6:2001:470:7120:1:127b:44ff:fe4f:148d]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "elch.exwg.net", Issuer "Christoph Moench-Tegeder" (verified OK)) by smtp.burggraben.net (Postfix) with ESMTPS; Mon, 29 Apr 2019 12:52:41 +0200 (CEST) Received: by elch.exwg.net (Postfix, from userid 1000) id CEDED1399E4; Mon, 29 Apr 2019 12:52:40 +0200 (CEST) Date: Mon, 29 Apr 2019 12:52:40 +0200 From: Christoph Moench-Tegeder To: Lars Engels Cc: Charles Sprickman , kris@ixsystems.com, FreeBSD Stable Subject: Re: CFT: FreeBSD Package Base Message-ID: <20190429105240.GB2396@elch.exwg.net> Mail-Followup-To: Lars Engels , Charles Sprickman , kris@ixsystems.com, FreeBSD Stable References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> <34C3A77A-7B18-487E-9BC4-5B1087FEEBEA@bway.net> <20190429100104.GX62338@e.0x20.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20190429100104.GX62338@e.0x20.net> User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: BA4766FB4B X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.75 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; RCVD_COUNT_FIVE(0.00)[5]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[burggraben.net]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.burggraben.net]; NEURAL_HAM_SHORT(-0.88)[-0.877,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-0.77)[ipnet: 2a01:4f8::/29(-2.12), asn: 24940(-1.73), country: DE(-0.01)] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 10:52:54 -0000 ## Lars Engels (lars.engels@0x20.net): > > What are the plans to get rid of the hellscape known as “mergemaster”? > > Is there anything exciting and new there either in base or any of the > > ixSystems projects? > > There's /usr/sbin/etcupdate in base. Unfortunately, the official documentation on upgrades does not mention etcupdate at all. The release notes point to freebsd-update for binary upgrades, and to /usr/src/UPDATING for source-based upgrades. UPDATING only documents the mergemaster procedure; etcupdate is only mentioned in a few entries without any detail. The Handbook (chapter 25.3) only documents mergemaster. Even after speed-reading etcupdate(8) I'm not sure how the two-step invocation of mergemaster has to be mapped to etcupdate - I feel I'd be inventing my own procedure here (and, of course, if it breaks I get to keep all the pieces). This situation should be improved. Given that etcupdate is in all supported releases, we can even update UPDATING and the Handbook. So, does anyone have a pointer to the official procedure? Regards, Christoph -- Spare Space From owner-freebsd-stable@freebsd.org Mon Apr 29 10:56:35 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFF9F1588070 for ; Mon, 29 Apr 2019 10:56:34 +0000 (UTC) (envelope-from cmt@burggraben.net) Received: from smtp.burggraben.net (smtp.burggraben.net [IPv6:2a01:4f8:140:50a2::3:1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "ns.exwg.net", Issuer "Christoph Moench-Tegeder" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CA9B6FCA8 for ; Mon, 29 Apr 2019 10:56:34 +0000 (UTC) (envelope-from cmt@burggraben.net) Received: from localhost (localhost [127.0.0.1]) by smtp.burggraben.net (Postfix) with ESMTP id 8DF856002F8; Mon, 29 Apr 2019 12:56:33 +0200 (CEST) X-Spam-Scanned: by amavisd-new at exwg.net Received: from smtp.burggraben.net ([127.0.0.1]) by localhost (ns.burggraben.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1Y3+Lu7qHDQg; Mon, 29 Apr 2019 12:56:22 +0200 (CEST) Received: from elch.exwg.net (elch.exwg.net [IPv6:2001:470:7120:1:127b:44ff:fe4f:148d]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "elch.exwg.net", Issuer "Christoph Moench-Tegeder" (verified OK)) by smtp.burggraben.net (Postfix) with ESMTPS; Mon, 29 Apr 2019 12:56:22 +0200 (CEST) Received: by elch.exwg.net (Postfix, from userid 1000) id 1BDE51399E4; Mon, 29 Apr 2019 12:56:22 +0200 (CEST) Date: Mon, 29 Apr 2019 12:56:22 +0200 From: Christoph Moench-Tegeder To: freebsd-stable@freebsd.org Cc: Lars Engels , Charles Sprickman , kris@ixsystems.com Subject: Re: CFT: FreeBSD Package Base Message-ID: <20190429105621.GC2396@elch.exwg.net> Mail-Followup-To: freebsd-stable@freebsd.org, Lars Engels , Charles Sprickman , kris@ixsystems.com References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> <34C3A77A-7B18-487E-9BC4-5B1087FEEBEA@bway.net> <20190429100104.GX62338@e.0x20.net> <20190429105240.GB2396@elch.exwg.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20190429105240.GB2396@elch.exwg.net> User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: 5CA9B6FCA8 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.80 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; RCVD_COUNT_FIVE(0.00)[5]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[burggraben.net]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: smtp.burggraben.net]; NEURAL_HAM_SHORT(-0.93)[-0.927,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-0.77)[ipnet: 2a01:4f8::/29(-2.12), asn: 24940(-1.73), country: DE(-0.01)] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 10:56:35 -0000 ## Christoph Moench-Tegeder (cmt@burggraben.net): > in a few entries without any detail. The Handbook (chapter 25.3) only ^ 23.5, of course. Regards, Christoph -- Spare Space From owner-freebsd-stable@freebsd.org Mon Apr 29 11:18:47 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E33021588B33 for ; Mon, 29 Apr 2019 11:18:46 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: from mail-yb1-f195.google.com (mail-yb1-f195.google.com [209.85.219.195]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49ADE70DFF for ; Mon, 29 Apr 2019 11:18:46 +0000 (UTC) (envelope-from lwhsu.freebsd@gmail.com) Received: by mail-yb1-f195.google.com with SMTP id 68so3614511ybn.9 for ; Mon, 29 Apr 2019 04:18:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=MrW4JYSFgufpBuiWSMwxDZK8SmCXydf4iTG7IhTvw6I=; b=Cs/rGoI3OGgHbjH3swXLk/mMJ5VronCyRMHguFixMdZBS3J8EEVHADsCow2CPVQnH1 d8F1jB+FYpfVW9PQCgnnVRA8y4xgISR60t9/PUD2EABcTmsWF0Tp4eTQe9Z0VYD9db6S 81eR5P2snISnt8+kWDfdg1+37eXF29J/dvdZ3tlYN10UMMi4rvGWrZ7rnwaQw2TA9J4b SQbtKLRcs3RkLYIksGC47xdhMTowxsN5ygOMWtZB03g2PDiMqrlqcGx3IEbknvp5PyTb HTrgvCJMuTURtdUxBU7a7e0D8ViuxQg8HAaaMNLQoNcDhrRXtJFRqyaBwQkiPxbiMu/2 rzaw== X-Gm-Message-State: APjAAAUovNDOHjy2iZcglcLYQdWXcC40OKiWqdkNxqXekmXSx7UplkQm WCBFaSwxDPhU2JCVx/EuPa3EU03vD/0p+sGJ6LQ= X-Google-Smtp-Source: APXvYqxIHCvSVIqHxlK6T2NhKctjRkis4BEEHmg1Ot0OT5fPIJII0T3OUsMgaxdeM0eBakrkBP6ltyuwAweXTGANVnA= X-Received: by 2002:a25:d93:: with SMTP id 141mr23672239ybn.489.1556536284218; Mon, 29 Apr 2019 04:11:24 -0700 (PDT) MIME-Version: 1.0 From: Li-Wen Hsu Date: Mon, 29 Apr 2019 20:11:13 +0900 Message-ID: Subject: FreeBSD Continuous Weekly Report 2019-04-28 To: freebsd-testing@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 49ADE70DFF X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of lwhsufreebsd@gmail.com designates 209.85.219.195 as permitted sender) smtp.mailfrom=lwhsufreebsd@gmail.com X-Spamd-Result: default: False [-4.09 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FORGED_RECIPIENTS(0.00)[freebsd-testing@freebsd.org,freebsd-stable@freebsd.org]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.84)[-0.843,0]; DMARC_NA(0.00)[freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[195.219.85.209.list.dnswl.org : 127.0.5.0]; IP_SCORE(-1.24)[ipnet: 209.85.128.0/17(-3.87), asn: 15169(-2.26), country: US(-0.06)]; FORGED_SENDER(0.30)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; TAGGED_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[lwhsu@freebsd.org,lwhsufreebsd@gmail.com]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Mailman-Approved-At: Mon, 29 Apr 2019 11:39:49 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 11:18:47 -0000 (bcc -current and -stable for more audience) FreeBSD CI Weekly Report 2019-04-28 =================================== Here is a summary of the FreeBSD Continuous Integration results for the period from 2019-04-22 to 2019-04-28. During this period, we have: * 2358 builds (96.9% passed, 3.1% failed) were executed on aarch64, amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, sparc64 architectures for head, stable/12, stable/11 branches. * 436 test runs (34.9% passed, 59.4% unstable, 5.7% exception) were executed on amd64, i386, riscv64 architectures for head, stable/12, stable/11 branches. * 9 doc buils (100% passed) (The statistics from experimental jobs are omitted) If any of the issues found by CI are in your area of interest or expertise please investigate the PRs listed below. The latest web version of this report is available at https://hackmd.io/s/ByfvuSs54 and archive is available at http://hackfoldr.org/freebsd-ci-report/, any help is welcome. ## Fixed tests * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/ * sys.netpfil.pf.forward.v4 * sys.netpfil.pf.forward.v6 * sys.netpfil.pf.fragmentation.v6 * sys.netpfil.pf.icmp.cve_2019_5598 * sys.netpfil.pf.set_tos.v4 https://bugs.freebsd.org/237305 * https://ci.freebsd.org/job/FreeBSD-head-i386-test/ * sys.kern.coredump_phnum_test.coredump_phnum https://svnweb.freebsd.org/changeset/base/346542 * https://ci.freebsd.org/job/FreeBSD-stable-12-i386-test/ * sys.netmap.ctrl-api-test.main * https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/ * common.ip.t_dtrace_contrib.tst_ipv4localsctp_ksh * common.ip.t_dtrace_contrib.tst_localsctpstate_ksh https://svnweb.freebsd.org/changeset/base/346854 ## Failing Tests * https://ci.freebsd.org/job/FreeBSD-head-i386-test/ * sys.opencrypto.runtests.main * sys.netpfil.pf.forward.v6 * sys.netpfil.pf.forward.v4 * sys.netpfil.pf.set_tos.v4 * https://ci.freebsd.org/job/FreeBSD-stable-12-i386-test/ * sys.opencrypto.runtests.main * lib.libc.regex.exhaust_test.regcomp_too_big * lib.libregex.exhaust_test.regcomp_too_big * sys.kern.coredump_phnum_test.coredump_phnum MFC pending: https://svnweb.freebsd.org/changeset/base/346542 * sys.netpfil.pf.forward.v6 * usr.bin.procstat.procstat_test.command_line_arguments * lib.libc.sys.sendfile_test.fd_positive_shm_v4 * lib.libc.sys.sendfile_test.hdtr_negative_bad_pointers_v4 * sys.netpfil.pf.forward.v4 * sys.netpfil.pf.set_tos.v4 * https://ci.freebsd.org/job/FreeBSD-stable-11-amd64-test/ * (flaky) usr.bin.procstat.procstat_test.environment * https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/ * usr.bin.procstat.procstat_test.kernel_stacks * local.kyua.* (31 cases) * local.lutok.* (3 cases) * lib.libc.sys.sendfile_test.fd_positive_shm_v4 * lib.libc.sys.sendfile_test.hdtr_negative_bad_pointers_v4 ## Failing Tests (from experimental jobs) * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/ There are ~60 failing cases, including flakey ones, see https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/ for more details ## Disabled Tests * lib.libc.sys.mmap_test.mmap_truncate_signal https://bugs.freebsd.org/211924 * sys.fs.tmpfs.mount_test.large https://bugs.freebsd.org/212862 * sys.fs.tmpfs.link_test.kqueue https://bugs.freebsd.org/213662 * sys.kqueue.libkqueue.kqueue_test.main https://bugs.freebsd.org/233586 * usr.bin.procstat.procstat_test.command_line_arguments https://bugs.freebsd.org/233587 * usr.bin.procstat.procstat_test.environment https://bugs.freebsd.org/233588 ## Closed Issues * https://bugs.freebsd.org/237305 Multiple sys.netpfil.pf.* tests failing on ^/head and ^/stable/12 because of TypeError with scapy library reading interfaces from bpf ## Oepn Issues * https://bugs.freebsd.org/237077 possible race in build: /usr/src/sys/amd64/linux/linux_support.s:38:2: error: expected relocatable expression * https://bugs.freebsd.org/237403 Tests in sys/opencrypto should be converted to Python3 ### Cause build fails * [233735: Possible build race: genoffset.o /usr/src/sys/sys/types.h: error: machine/endian.h: No such file or directory](https://bugs.freebsd.org/233735) * [233769: Possible build race: ld: error: unable to find library -lgcc_s](https://bugs.freebsd.org/233769) ### Others [Tickets related to testing@](https://preview.tinyurl.com/y9maauwg) From owner-freebsd-stable@freebsd.org Mon Apr 29 11:56:02 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 693F41589BB3 for ; Mon, 29 Apr 2019 11:56:02 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F640737AA for ; Mon, 29 Apr 2019 11:56:02 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [81.2.117.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: matthew/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id BE961D484 for ; Mon, 29 Apr 2019 11:56:01 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from leaf.local (unknown [88.212.184.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id 2E1F8AB74 for ; Mon, 29 Apr 2019 11:55:59 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk/2E1F8AB74; dkim=none; dkim-atps=neutral Subject: Re: CFT: FreeBSD Package Base To: freebsd-stable@freebsd.org References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <15583041-1D56-44BE-8E88-ECC11C935A09@gromit.dlib.vt.edu> <34C3A77A-7B18-487E-9BC4-5B1087FEEBEA@bway.net> <20190429100104.GX62338@e.0x20.net> <20190429105240.GB2396@elch.exwg.net> From: Matthew Seaman Message-ID: <701337aa-9236-a089-db11-85405642ead7@FreeBSD.org> Date: Mon, 29 Apr 2019 12:55:44 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <20190429105240.GB2396@elch.exwg.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 0F640737AA X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; ASN(0.00)[asn:11403, ipnet:96.47.64.0/20, country:US] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 11:56:02 -0000 On 29/04/2019 11:52, Christoph Moench-Tegeder wrote: > This situation should be improved. Given that etcupdate is in all > supported releases, we can even update UPDATING and the Handbook. > So, does anyone have a pointer to the official procedure? Basically run: # etcupdate towards the end of your standard update-from-src process. Pretty much all the available arguments to etcupdate have sane defaults, so it's quite likely you can just run the bare command. (Obv. check the man page for details though) It will automatically do a 3-way merge between the latest sources, the sources from the previous time you ran etcupdate and your deployed configuration in /etc Most of the time, that's all you need to do. Occasionally however etcupdate will run into a merge conflict. When that happens etcupdate will display a SVN-like status report showing the conflicting files and refuse to do anything until the conflicts have been resolved. Which you do by: # etcupdate resolve That's an interactive process which runs through all of the conflicts and allows you to examine the differences and select either your own or the upstream version, or to just edit the files and merge the conflicts manually. Should be pretty familiar if you've ever used git or svn... Once everything has been marked as resolved, etcupdate reverts back to the usual behaviour and will automatically merge in any new updates as far as possible. Most of the time using etcupdate just boils down to a single, non-interactive command to keep everything up to date during the update process. Much less faff than using mergemaster. Occasionally you may need to run: # etcupdate -p before you 'make buildworld' -- that's in exactly the same circumstances where you'ld previously have run 'mergemaster -p' and it does the same job: apply any necessary changes to allow your build to succeed. The times you need to do this should be flagged in UPDATING. Cheers, Matthew From owner-freebsd-stable@freebsd.org Mon Apr 29 12:12:49 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B705F158AD69; Mon, 29 Apr 2019 12:12:49 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.blih.net", Issuer "mail.blih.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E2BC74999; Mon, 29 Apr 2019 12:12:47 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) by mail.blih.net (OpenSMTPD) with ESMTP id 87105727; Mon, 29 Apr 2019 14:12:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=bidouilliste.com; h=date :from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=mail; bh=Ms5XXVorARtlOGUgvbKshdkOgo8=; b=V+JgettZt96XaRWjvJhsyyRzWvlD 7skvprZYj3ZfcbeGVWktzytAYg8KBd7jgnruNjnWkee85fZ1yd0YlDwuEHtiSHxv 4OeeMbBHEl1kCivSHZgY73Jl+KaY6NF5AenaxAuDXCWTGHci9rDkgOlUCp3b5c0v fVd/bs0WPMFG3xo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=bidouilliste.com; h=date :from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; q=dns; s= mail; b=fLgukhsrzDainCJvVhz22KowORFQow/wQuiVQ11LCCN7X0T2vuVwDRQc iUZgVYCFdZKKUx4jcjx2rwtoznm7DkhA9IoVMVoIEnT7QIIpNLWer4Mb+XXU34o2 4iuQsnJIA9Dol0Uksx7Va8zdaBu4NJ10mDBd4vktUSCjyx5GWaU= Received: from knuckles.blih.net (ip-9.net-89-3-105.rev.numericable.fr [89.3.105.9]) by mail.blih.net (OpenSMTPD) with ESMTPSA id 024e27e4 TLS version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO; Mon, 29 Apr 2019 14:12:40 +0200 (CEST) Date: Mon, 29 Apr 2019 14:12:39 +0200 From: Emmanuel Vadot To: Cc: , , , , , Subject: Re: CFT: FreeBSD Package Base Message-Id: <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> In-Reply-To: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; amd64-portbld-freebsd13.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 7E2BC74999 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bidouilliste.com header.s=mail header.b=V+JgettZ; spf=pass (mx1.freebsd.org: domain of manu@bidouilliste.com designates 212.83.177.182 as permitted sender) smtp.mailfrom=manu@bidouilliste.com X-Spamd-Result: default: False [0.41 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[bidouilliste.com:s=mail]; NEURAL_HAM_MEDIUM(-0.56)[-0.561,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.83.177.182/32]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[bidouilliste.com]; NEURAL_HAM_LONG(-0.05)[-0.054,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bidouilliste.com:+]; MX_GOOD(-0.01)[mx-backup.blih.net,mail.blih.net]; RCPT_COUNT_SEVEN(0.00)[7]; NEURAL_HAM_SHORT(-0.95)[-0.947,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.38)[ip: (-0.89), ipnet: 212.83.160.0/19(2.97), asn: 12876(-0.17), country: FR(-0.01)]; ASN(0.00)[asn:12876, ipnet:212.83.160.0/19, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 12:12:50 -0000 Hi Kris, On Sun, 28 Apr 2019 15:52:21 -0400 wrote: > FreeBSD Community, > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and 13-current > using "TrueOS-inspired" packaged base. These are stock FreeBSD images which > will allow users to perform all updating via the 'pkg' command directly. > Rather than trying to answer all questions in this announcement, we've > created a FAQ page with more details. Please refer to this page, and let us > know if you have additional questions that we can include on that page going > forward. > While I appreciate the effort I have some doubt about your "re-implementation" of pkgbase. I don't see any improvement compared to what is in base currently, I even see downside of your implementation. - How do you plan with the need of updating kernel first, reboot and updating the rest of the userland after ? (Needed for major and minor upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and -HEAD branch). This is still a problem with the base pkgbase. - This is even worse because you are using the same repository for base and pkg so if a user pkg update and both kernel and pkg(8) needs to be updated and pkg use a new syscall or capsicum thing it will be updated first and couldn't proceed with the rest of the update (this is a supposition, I haven't personally tested). - It seems that multiple kernels isn't supported in your implementation, this is already supported in pkgbase but still need some love. This is an important point as it will allow user to choose easily the kernel that they want to use and will also allow us developper to push kernels with new features to help testing. - Since you reduced the granularity on the userland bits it would mean that if we use your implementation for -p updates we would download the whole userland packages instead of just updating the package that was patched. For example with pkgbase, updating from 12.0 to 12.0p1 will only update the FreeBSD-runtime package. Yes this package is still big to download when you compare to what have changed but until pkg(8) have delta pkg supports (and if it will have support, I don't know if this is a wish or not) this is the best way to go. - I see that you are sorting the plist for kernel and userland based on the line length [1], why is that ? I think that the only advantage that your solution offers is that if we remove a componant of base (rcmds for example in 12-CURRENT) those files would be removed as they are in the userland-base package while for pkgbase the FreeBSD-rcmd package will be deleted in the repo and will not be deleted in the user computer. > > Additionally, I will be hosting a Package Base working group at BSDCan 2019, > and welcome user and developer attendance to discuss this and other ongoing > package work: > > > > https://wiki.freebsd.org/DevSummit/201905/PackageBase > I will be present and looking forward to work with you on this. Cheers, P.S. : FYI I'm working on pkgbase currently and I will have some patches to commit soon (bsdinstall support, memstick creation that install a pkgbase aware installaton etc ...). [1] : https://github.com/trueos/trueos-ports/blob/trueos-master/os/userland-base/Makefile#L35 -- Emmanuel Vadot From owner-freebsd-stable@freebsd.org Mon Apr 29 13:25:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 184C0158CDBB for ; Mon, 29 Apr 2019 13:25:16 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-it1-x12b.google.com (mail-it1-x12b.google.com [IPv6:2607:f8b0:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F9C47785E for ; Mon, 29 Apr 2019 13:25:14 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-it1-x12b.google.com with SMTP id v143so5641403itc.1 for ; Mon, 29 Apr 2019 06:25:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=K5ww+uvGAcRKntVX3CuKClDjGquFkWxKL+ayep4zX10=; b=18l2jegDtSE6M/3b/cI37vftQhp1g1wk9v2IzVOXTKgaosGWmiuSFAbvGdThL6KVIA EKIv560wjXu/OgSXqXmtR53wM3jehutRugi/uhQVZUazZQ18FUVNCNmeroJw0LNyopZC tDS6mVrPnaZDxWHsltKW7CDfGcSRyxcd14goOGSsXsqyfVg1/+oARd6BjdpyedNgA2b+ 7V8kBfiwsaHCw5x8JZf1xCg2M39zu1hMabIfBMYquNQwMhFMC3/CX3rFTZyh1xBGvGUQ pwU/efzGr4jFpPwjwMLuUE2q3aVerLLxxGXU/YLYDrRYr+Rzzu5BtpKvVl6YEic8jGYL DMXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=K5ww+uvGAcRKntVX3CuKClDjGquFkWxKL+ayep4zX10=; b=ItJ7sm9xouOcKHmsPMAwMuGrN8zpAuKn+HUf21QAxP2Y2BeF3qp4uepYsV1YSAuqW6 dPQ+rmP9ee6D8yzTziNkrj3gERHdOqy3mrQxQOlZpAuzPlDv4sCejlqw9kyOTQ3dipU7 p/FfvTt006Tf4U9+oTRaKGTV4A0gsGcRMPAURBcXgFAnXUMTUlzkgcE+yEzHlWoEaiuO HAiwqfMIz8PZXKkKlOviApdk50N6+Lboafn+PXB0NtCH6PVNvYtr02IbMhC/ez1QylNw lsyuZV9vSfPbuVhH9nrKZ6H9TsW3r1wlvuJe8gaenXA+pPuza4wAjt8VP+CYoKdONf2C UOzg== X-Gm-Message-State: APjAAAXfCX0plAjvQpPEBGXlZqCnhTY/Yusvwwu7AdO8KpK1eRBngFBk NfCtisdWKDgut+i6RN8MZ+mI5KHJq1brEkzeNf/dYA== X-Google-Smtp-Source: APXvYqwm14h2MgICKuqcdrdQJ/8RW3/UZp/daTTGMbYrmHnfG1Ujj5fda1MZPlsSOZzj6zfkimSVCHWyeAgGN3FbvDM= X-Received: by 2002:a24:4682:: with SMTP id j124mr18995293itb.90.1556544313611; Mon, 29 Apr 2019 06:25:13 -0700 (PDT) MIME-Version: 1.0 References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> In-Reply-To: <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> From: Kris Moore Date: Mon, 29 Apr 2019 09:25:05 -0400 Message-ID: Subject: Re: CFT: FreeBSD Package Base To: Emmanuel Vadot Cc: FreeBSD Stable , FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org X-Rspamd-Queue-Id: 9F9C47785E X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=18l2jegD; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::12b as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-5.11 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; RCPT_COUNT_SEVEN(0.00)[7]; RCVD_IN_DNSWL_NONE(0.00)[b.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; IP_SCORE(-2.67)[ip: (-7.85), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.94)[-0.938,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; SUSPICIOUS_RECIPS(1.50)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 13:25:16 -0000 On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot wrote: > > Hi Kris, > > On Sun, 28 Apr 2019 15:52:21 -0400 > wrote: > > > FreeBSD Community, > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > 13-current > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > which > > will allow users to perform all updating via the 'pkg' command directly. > > Rather than trying to answer all questions in this announcement, we've > > created a FAQ page with more details. Please refer to this page, and let > us > > know if you have additional questions that we can include on that page > going > > forward. > > > > While I appreciate the effort I have some doubt about your > "re-implementation" of pkgbase. I don't see any improvement compared to > what is in base currently, I even see downside of your implementation. > > - How do you plan with the need of updating kernel first, reboot and > updating the rest of the userland after ? (Needed for major and minor > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > -HEAD branch). This is still a problem with the base pkgbase. > We've written our own tool "sysutils/sysup" in GO which handles this. It performs updates using Boot-Environments to ensure that kernel/world are updated at same time. > - This is even worse because you are using the same repository for > base and pkg so if a user pkg update and both kernel and pkg(8) needs > to be updated and pkg use a new syscall or capsicum thing it will be > updated first and couldn't proceed with the rest of the update (this is > a supposition, I haven't personally tested). > See above. > - It seems that multiple kernels isn't supported in your > implementation, this is already supported in pkgbase but still need > some love. This is an important point as it will allow user to choose > easily the kernel that they want to use and will also allow us > developper to push kernels with new features to help testing. > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll get the Witness-enabled kernel installed alongside non-debugging one. > - Since you reduced the granularity on the userland bits it would mean > that if we use your implementation for -p updates we would download the > whole userland packages instead of just updating the package that was > patched. For example with pkgbase, updating from 12.0 to 12.0p1 will > only update the FreeBSD-runtime package. Yes this package is still big > to download when you compare to what have changed but until pkg(8) have > delta pkg supports (and if it will have support, I don't know if > this is a wish or not) this is the best way to go. > Correct, this is by design. We used the in-tree pkg base for nearly a year, and found that the granularity didn't really offer any savings from a download or time perspective. Updating 100+ packages took far longer than a single one, due to all the meta operations. Additionally in real-world usage, we found that base packages tended to all get updated at the same time, which took far longer via pkg, since it had to go and perform 100+ fetch operations just to download the base system bits. > - I see that you are sorting the plist for kernel and userland based > on the line length [1], why is that ? Whoops! I'll fix :) > > I think that the only advantage that your solution offers is that if > we remove a componant of base (rcmds for example in 12-CURRENT) those > files would be removed as they are in the userland-base package while > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > will not be deleted in the user computer. > Correct, this is one of the things which prompted us to go this direction. Being able to handle crazy mixed WITH/WITHOUT flags was important to us, current pkg base did not handle that so gracefully. Additionally we've added some additional features, such as being able to 'pkg install os/src' to get system sources used in exact build, as well as being able to rebuild your local world / kernel packages using ports "make config" framework is super handy. > > > > > Additionally, I will be hosting a Package Base working group at BSDCan > 2019, > > and welcome user and developer attendance to discuss this and other > ongoing > > package work: > > > > > > > > https://wiki.freebsd.org/DevSummit/201905/PackageBase > > > > I will be present and looking forward to work with you on this. > > Cheers, > > P.S. : FYI I'm working on pkgbase currently and I will have some > patches to commit soon (bsdinstall support, memstick creation that > install a pkgbase aware installaton etc ...). > Great! Looking forward to discussion then! > > [1] : > > https://github.com/trueos/trueos-ports/blob/trueos-master/os/userland-base/Makefile#L35 > > -- > Emmanuel Vadot > From owner-freebsd-stable@freebsd.org Mon Apr 29 13:43:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08546158DC84; Mon, 29 Apr 2019 13:43:39 +0000 (UTC) (envelope-from meka@tilda.center) Received: from mail.tilda.center (srv02.tilda.center [199.247.21.11]) by mx1.freebsd.org (Postfix) with ESMTP id D416280907; Mon, 29 Apr 2019 13:43:37 +0000 (UTC) (envelope-from meka@tilda.center) Received: from hal9000.home.meka.rs (79-101-248-108.dynamic.isp.telekom.rs [79.101.248.108]) by mail.tilda.center (Postfix) with ESMTPSA id BA3D318F2E; Mon, 29 Apr 2019 15:43:29 +0200 (CEST) Date: Mon, 29 Apr 2019 15:43:29 +0200 From: Goran =?utf-8?B?TWVracSH?= To: Kris Moore Cc: Emmanuel Vadot , FreeBSD Stable , FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org Subject: Re: CFT: FreeBSD Package Base Message-ID: <20190429134329.g7t77muqfb6dh2fj@hal9000.home.meka.rs> References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="t5aclbayfnyvbvsj" Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20180716 X-Rspamd-Queue-Id: D416280907 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of meka@tilda.center designates 199.247.21.11 as permitted sender) smtp.mailfrom=meka@tilda.center X-Spamd-Result: default: False [-6.27 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; IP_SCORE(-2.84)[ip: (-9.65), ipnet: 199.247.16.0/21(-4.74), asn: 20473(0.24), country: US(-0.06)]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[tilda.center]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mail.tilda.center]; NEURAL_HAM_SHORT(-0.61)[-0.615,0]; RCPT_COUNT_SEVEN(0.00)[8]; SIGNED_PGP(-2.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:20473, ipnet:199.247.16.0/21, country:US]; RCVD_COUNT_TWO(0.00)[2]; SUSPICIOUS_RECIPS(1.50)[]; RECEIVED_SPAMHAUS_PBL(0.00)[108.248.101.79.zen.spamhaus.org : 127.0.0.11] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 13:43:39 -0000 --t5aclbayfnyvbvsj Content-Type: text/plain; charset=utf-8 Content-Disposition: inline On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > We've written our own tool "sysutils/sysup" in GO which handles this. It > performs updates using Boot-Environments to ensure that kernel/world are > updated at same time. If I'm right, UFS doesn't support boot environments, so how would it work for UFS based installs? I personally feel GO is a bit ackward choice of language for something that practically should be part of base. At least I would expect OS update/upgrade not to require any external package. Regards, meka --t5aclbayfnyvbvsj Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE1WIFkXy2ZeMKjjKEWj1TknovrLYFAlzG/30ACgkQWj1Tknov rLbwqQ//VC6wp5BQt7sP2Jr8NoCIo3pwn7oJOk+JPeFHFGuKqb0G/zyLj31LeYSH mtj+QN6lCIALHNuSis25OAUjBLp6rHXj7VM7oExuKfhtxJ4DJGXBxz4O8TnHwFuk CBGqFCwpv9Suedylc0evdHQls737zS46ExeM96DaDqm7Z86Ca2ODm7zm0RmeO19i m+FAdMDERGGgneQdTs8pHS5If420/vWkqldJzEKQoibXJQyom0hxtWZM/UoIvK8q h07d3yCa0s4A+2qzhWwuAJlTZs+SGZGC+Q/Kt4eMANRmp68obca3grFg7ribm1yq xTcS92ehnGBhLoNILWs3V0Nptcr7jy/aenRiX5Ume4P2ciH5tJEgw7n9iDqwSp49 Q4KVDsBOF5wfNIhrcHM9LGovxdUYPcnJSXOcbgcw1OQ2lOm8UC2rkMsRNx7Wci0P Q4Qi6w7KxQwaVhzzItAM/kdjAAFuXTbVpLmTo5yn81W26lR0t0d9mOF8DQ4Nw8B/ wPp/dOlGRT/tKETMUXTCQcdxFWSklvH6a1QKTVFwgLg4S5zi8fqma5y7B7TLDw1f E/zc4vz6SsT+1q6TSFmRwsjjI3yALWXQGHNb5PcH20j7Ws3+PXkCTFbxe1tHuDNJ nam2pOQyfyWhI9kaveYkVjmEEmuOR9HQDtGM+FmrfKTCpAc1kXs= =h7gm -----END PGP SIGNATURE----- --t5aclbayfnyvbvsj-- From owner-freebsd-stable@freebsd.org Mon Apr 29 13:53:52 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9DC17158E65C for ; Mon, 29 Apr 2019 13:53:52 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-yw1-xc35.google.com (mail-yw1-xc35.google.com [IPv6:2607:f8b0:4864:20::c35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7A3D6816D1 for ; Mon, 29 Apr 2019 13:53:51 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-yw1-xc35.google.com with SMTP id t79so3706485ywc.7 for ; Mon, 29 Apr 2019 06:53:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-transfer-encoding:thread-index :content-language; bh=EXuzmmllqeTQTipAzB39ooVpKvhDFQhxUdVgc/PWqBg=; b=Z3yUmcvdoyCbGASfDGr0mO3yqAfP31IZjfRtPSy2XRN80DRo/Mpi0Z4VHt5mXzkjqg oNb/2UPd5EeCVhIaiouhpthLsl1yOAHJYYbtnOU7PrfgiARyh/auwQpf+oJgYGQmqwpR 3KYwajbHeS8GKN7G/1LTjeuOwXv++5ENLmt+5YOl2jgLyGmnRpseNo0Y7n/hzk1Ds7/1 ytZHvp4UK3mBFfbqrIhH5J6DbOQVfC6B/aiAzUEULTD5BrMEh60EGgXIA9pPiRQhQXvf VghnSCSpcSpxbw1b0F0jY/1U9ClVc6t+xKdvfIuPf/Bo/nvpzUduWAmLIt/oCtKoiJW+ YiKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:thread-index :content-language; bh=EXuzmmllqeTQTipAzB39ooVpKvhDFQhxUdVgc/PWqBg=; b=sf6Jf7u5Nk49SkPL/XndLqLAU+ZlAfNyezc2HCTILnITkzPZPYDNswhsgq6QXtsdDW jWpcsI9W8+Qa8JKC9p1zAxmKhUQZ3yapH6spdLPqyiMtqN1HdeVsI3FPaS6wFhLUncXa NEi4z0a20dThG53Tp7r8zJ4nHDBOO0eV9qLC9/Lf4iIXcIJ/hyT1AKzEW7+l5RuEy/p9 7+J2TimZvN1w3aCBDxZvyomIdvY/MGZaFIXX3De5hk/VY1y40SdlcKEVvIWS6/9idw4R l2CGXxbqlkBpGB4a6TXWAjqWiB7aw09HK2DXC5oKtzsFcc6sHHYOdKVkvHJay9EpRuLQ wphA== X-Gm-Message-State: APjAAAUL7OIT5NS/ykmsXVOLYbPLmK7GBzhTAjK/zLjp/31fdHTyl7W0 pjBRgJFIMENVVcHLyr9390K9nA== X-Google-Smtp-Source: APXvYqyzdAVysTSezWAz8diDN7s0WdwoHEdPyaAG0WEMHsI/iD3D4N2/ztpzGHFUUXKS+CPfADDWHw== X-Received: by 2002:a81:3cd3:: with SMTP id j202mr51629632ywa.149.1556546030447; Mon, 29 Apr 2019 06:53:50 -0700 (PDT) Received: from KrisYogaC930 ([12.189.233.130]) by smtp.gmail.com with ESMTPSA id i13sm1383267ywl.22.2019.04.29.06.53.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Apr 2019 06:53:49 -0700 (PDT) From: To: =?UTF-8?Q?'Goran_Meki=C4=87'?= Cc: "'Emmanuel Vadot'" , "'FreeBSD Stable'" , "'FreeBSD Current'" , , , , References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> <20190429134329.g7t77muqfb6dh2fj@hal9000.home.meka.rs> In-Reply-To: <20190429134329.g7t77muqfb6dh2fj@hal9000.home.meka.rs> Subject: RE: CFT: FreeBSD Package Base Date: Mon, 29 Apr 2019 09:53:48 -0400 Message-ID: <03b001d4fe92$f908c990$eb1a5cb0$@ixsystems.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQFBRupbev+o6AHqgfoJLZwOgBKxNQFhdrr5Ad4+GxoBeMI9n6dU1Bqg Content-Language: en-us X-Rspamd-Queue-Id: 7A3D6816D1 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=Z3yUmcvd; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::c35 as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-5.33 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-2.84)[ip: (-8.73), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; FROM_NO_DN(0.00)[]; RCPT_COUNT_SEVEN(0.00)[8]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; RCVD_IN_DNSWL_NONE(0.00)[5.3.c.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.97)[-0.975,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 13:53:52 -0000 Correct, this is ZFS only. And it's something we're using specific to = FreeNAS / TrueOS, which is why I didn't originally mention it as apart = of our CFT.=20 For UFS, there will need to be additional care taken when doing updates. = --=20 Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source -----Original Message----- From: Goran Meki=C4=87 =20 Sent: Monday, April 29, 2019 9:43 AM To: Kris Moore Cc: Emmanuel Vadot ; FreeBSD Stable = ; FreeBSD Current = ; freebsd-pkgbase@freebsd.org; = freebsd-pkg@freebsd.org; freebsd-hackers@freebsd.org; = freebsd-ports@freebsd.org Subject: Re: CFT: FreeBSD Package Base On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > We've written our own tool "sysutils/sysup" in GO which handles this.=20 > It performs updates using Boot-Environments to ensure that=20 > kernel/world are updated at same time. If I'm right, UFS doesn't support boot environments, so how would it = work for UFS based installs? I personally feel GO is a bit ackward choice of language for something = that practically should be part of base. At least I would expect OS = update/upgrade not to require any external package. Regards, meka From owner-freebsd-stable@freebsd.org Mon Apr 29 13:55:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 23902158E871; Mon, 29 Apr 2019 13:55:28 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.blih.net", Issuer "mail.blih.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CB398197D; Mon, 29 Apr 2019 13:55:25 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) by mail.blih.net (OpenSMTPD) with ESMTP id 76f0e8c2; Mon, 29 Apr 2019 15:55:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=bidouilliste.com; h=date :from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=mail; bh=MLAcrXcF53XA7rh7ywbaMUgcCS4=; b=FrWKqTnTfxlianTyg0eI+7KrLjVb L6GgT4tt8gE90MEQfz8nNlz+3cmzpC6TpXee+8Z20fE6vpTnxZRAh30M+5rI9WB6 Ej2gMptD/Jy7RVJmr4P2BKQ35tfRVBs9Ha/FHvp2VIIVbo9RzFW65ukQ1+Jqh/00 VkeUL2ZjqlWaT8I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=bidouilliste.com; h=date :from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; q=dns; s= mail; b=DjwipvzFHHdC3LvK7OlJ5FYooPxmQJnYK1IAsoHOChxJo+ZFX+/TobIs z7WpJtQA91Ij5B+cym6YB3/6RKi0GtuybjkqPlwcd1lpZDgbyrN3FBC1VojKmy32 sl+1ajou9LaTXqDqdt8nojcrvdf+abEGIeVQ+/a2vRfI/WRV2wo= Received: from skull.home.blih.net (ip-9.net-89-3-105.rev.numericable.fr [89.3.105.9]) by mail.blih.net (OpenSMTPD) with ESMTPSA id 20095b58 TLS version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO; Mon, 29 Apr 2019 15:55:23 +0200 (CEST) Date: Mon, 29 Apr 2019 15:55:22 +0200 From: Emmanuel Vadot To: Kris Moore Cc: FreeBSD Stable , freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-ports@freebsd.org Subject: Re: CFT: FreeBSD Package Base Message-Id: <20190429155522.81dcc291a6b252884219de1e@bidouilliste.com> In-Reply-To: References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; amd64-portbld-freebsd13.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 2CB398197D X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bidouilliste.com header.s=mail header.b=FrWKqTnT; spf=pass (mx1.freebsd.org: domain of manu@bidouilliste.com designates 212.83.177.182 as permitted sender) smtp.mailfrom=manu@bidouilliste.com X-Spamd-Result: default: False [2.37 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[bidouilliste.com:s=mail]; NEURAL_HAM_MEDIUM(-0.45)[-0.454,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.83.177.182/32]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[bidouilliste.com]; NEURAL_SPAM_SHORT(0.66)[0.664,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mx-backup.blih.net]; DKIM_TRACE(0.00)[bidouilliste.com:+]; RCPT_COUNT_SEVEN(0.00)[7]; NEURAL_SPAM_LONG(0.18)[0.183,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.38)[ip: (-0.87), ipnet: 212.83.160.0/19(2.97), asn: 12876(-0.17), country: FR(-0.01)]; ASN(0.00)[asn:12876, ipnet:212.83.160.0/19, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 13:55:28 -0000 On Mon, 29 Apr 2019 09:25:05 -0400 Kris Moore wrote: > On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot > wrote: > > > > > Hi Kris, > > > > On Sun, 28 Apr 2019 15:52:21 -0400 > > wrote: > > > > > FreeBSD Community, > > > > > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > > 13-current > > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > > which > > > will allow users to perform all updating via the 'pkg' command directly. > > > Rather than trying to answer all questions in this announcement, we've > > > created a FAQ page with more details. Please refer to this page, and let > > us > > > know if you have additional questions that we can include on that page > > going > > > forward. > > > > > > > While I appreciate the effort I have some doubt about your > > "re-implementation" of pkgbase. I don't see any improvement compared to > > what is in base currently, I even see downside of your implementation. > > > > - How do you plan with the need of updating kernel first, reboot and > > updating the rest of the userland after ? (Needed for major and minor > > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > > -HEAD branch). This is still a problem with the base pkgbase. > > > > We've written our own tool "sysutils/sysup" in GO which handles this. It > performs updates using Boot-Environments to ensure that kernel/world are > updated at same time. > Which could never be imported into FreeBSD. > > > > - This is even worse because you are using the same repository for > > base and pkg so if a user pkg update and both kernel and pkg(8) needs > > to be updated and pkg use a new syscall or capsicum thing it will be > > updated first and couldn't proceed with the rest of the update (this is > > a supposition, I haven't personally tested). > > > > See above. > > > > - It seems that multiple kernels isn't supported in your > > implementation, this is already supported in pkgbase but still need > > some love. This is an important point as it will allow user to choose > > easily the kernel that they want to use and will also allow us > > developper to push kernels with new features to help testing. > > > > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll get > the Witness-enabled kernel installed alongside non-debugging one. Mhm no, the kernel-debug packages only add the debug file in /usr/lib/debug/boot/ I'm talking about installing multiple kernels in // (i.e. /boot/kernel.GENERIC /boot/kernel.MYFEATUREIWANTTOTEST) like describe here : https://wiki.freebsd.org/PkgBase#Project_goals_and_additional_unresolved_issues in the "How to handle /boot/kernel and /boot/kernel.$KERNCONF" point. > > > - Since you reduced the granularity on the userland bits it would mean > > that if we use your implementation for -p updates we would download the > > whole userland packages instead of just updating the package that was > > patched. For example with pkgbase, updating from 12.0 to 12.0p1 will > > only update the FreeBSD-runtime package. Yes this package is still big > > to download when you compare to what have changed but until pkg(8) have > > delta pkg supports (and if it will have support, I don't know if > > this is a wish or not) this is the best way to go. > > > > Correct, this is by design. We used the in-tree pkg base for nearly a year, > and found that the granularity didn't really offer any savings from a > download or time perspective. Updating 100+ packages took far longer than a > single one, due to all the meta operations. Additionally in real-world > usage, we found that base packages tended to all get updated at the same > time, which took far longer via pkg, since it had to go and perform 100+ > fetch operations just to download the base system bits. > But you never need to update 100+ packages on a proper pkgbase setup for -p updates. Again on a 12.0 to 12.0-p1 update only one package will be updated. > > > - I see that you are sorting the plist for kernel and userland based > > on the line length [1], why is that ? > > > Whoops! I'll fix :) > > > > > > I think that the only advantage that your solution offers is that if > > we remove a componant of base (rcmds for example in 12-CURRENT) those > > files would be removed as they are in the userland-base package while > > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > > will not be deleted in the user computer. > > > > > Correct, this is one of the things which prompted us to go this direction. > Being able to handle crazy mixed WITH/WITHOUT flags was important to us, > current pkg base did not handle that so gracefully. Can you give me more info on this ? What where the WITH/WITHOUT flags that causes problems ? > Additionally we've > added some additional features, such as being able to 'pkg install os/src' > to get system sources used in exact build, as well as being able to rebuild > your local world / kernel packages using ports "make config" framework is > super handy. > That would not be hard to add to pkgbase, I'll add it to my todo list. > > > > > > > > Additionally, I will be hosting a Package Base working group at BSDCan > > 2019, > > > and welcome user and developer attendance to discuss this and other > > ongoing > > > package work: > > > > > > > > > > > > https://wiki.freebsd.org/DevSummit/201905/PackageBase > > > > > > > I will be present and looking forward to work with you on this. > > > > Cheers, > > > > P.S. : FYI I'm working on pkgbase currently and I will have some > > patches to commit soon (bsdinstall support, memstick creation that > > install a pkgbase aware installaton etc ...). > > > > Great! Looking forward to discussion then! > > > > > > [1] : > > > > https://github.com/trueos/trueos-ports/blob/trueos-master/os/userland-base/Makefile#L35 > > > > -- > > Emmanuel Vadot > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Emmanuel Vadot From owner-freebsd-stable@freebsd.org Mon Apr 29 14:06:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA70C158F496 for ; Mon, 29 Apr 2019 14:06:24 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-it1-x134.google.com (mail-it1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A20F9829AF for ; Mon, 29 Apr 2019 14:06:23 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-it1-x134.google.com with SMTP id a190so16317549ite.4 for ; Mon, 29 Apr 2019 07:06:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rkBXTnF4X5xo6I+15kt9hGs8sq+74DaaTGWyV1SenI4=; b=HNtcg/lv1HI8HPYZ1UR0iJ/cW3hCd/0PUXcXcPyjx56PWwbonrT24svfaV104XgS2M XyOCt4Y/3m4yigbe3f6vsDVKzufaoi3OOIIWQN302Vagm+9p7ANA6uRJG8bmjVXR9iGf RqEHUUu60vcMIhzY1jITqcCBc5FQF+In/GF1jMj8P3fs/UboF35nzEDu/3g3Qv62etch +U3L0D2v5JOTEWQmS5rnu1IUtXS+DkqZIRnmFPANEuUswWoLS01T8Ad2ThTfPwdhs+Qs /HecQh3HtK48CNUkqTtDIrCVQam0YDVmkmnwGJmywK2YP0AgvCOqmbB1OyGCtzZplgge Iwig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rkBXTnF4X5xo6I+15kt9hGs8sq+74DaaTGWyV1SenI4=; b=FCwi8HW1OpbaDnSulAJtqCgChCzH3OI5d0PIRtrJI+G/a6asam5ow1YfeChgcZK7iA OG4tgHkB8Y+aecRJ3SSsZv4kEQ8lzBo0OYvFjaI/Gguv15T04JIubMwhey5r4kQ/RQ+Z X4rw/O2oS3rbvGdEbCYVcQxFJHnkvO9bfPjd4+e6zfywP38pexmF+BuafmPN2GZ6aemN gz9DPw0ssUiYh36FfAY3XyCICB/2qlW7kTNKm+gV4xR9Mg4oeUlLd4BQ8AaadJwrXrQg nYoOfqFhhJUKf289arQ1uZJMRxiFbP+aX1P6o/ThgVhe6gJvRfxx4djMRINwrLy6E/W9 aBRw== X-Gm-Message-State: APjAAAWYossQA+Ni4GlrLdP+2hp3TITK+5OuUxTTJ1wqOvLQvl5B69zM THjcH38w/RNXY/E1RiBGfKfhhkQ2VrvvVETM0dvzPA== X-Google-Smtp-Source: APXvYqz+8g7dXIkrTysL5y4+vtcwYgi5jxDCKrjcP2POhFwsT/KjZyv8dXTDR8y6bks7xSMPkHppOYMkPM0FJ2Jt3vE= X-Received: by 2002:a24:f68a:: with SMTP id u132mr19235507ith.45.1556546770513; Mon, 29 Apr 2019 07:06:10 -0700 (PDT) MIME-Version: 1.0 References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> <20190429155522.81dcc291a6b252884219de1e@bidouilliste.com> In-Reply-To: <20190429155522.81dcc291a6b252884219de1e@bidouilliste.com> From: Kris Moore Date: Mon, 29 Apr 2019 10:05:59 -0400 Message-ID: Subject: Re: CFT: FreeBSD Package Base To: Emmanuel Vadot Cc: FreeBSD Stable , freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-ports@freebsd.org X-Rspamd-Queue-Id: A20F9829AF X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=HNtcg/lv; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::134 as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-5.41 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; RCPT_COUNT_SEVEN(0.00)[7]; RCVD_IN_DNSWL_NONE(0.00)[4.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; IP_SCORE(-2.91)[ip: (-9.08), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; SUSPICIOUS_RECIPS(1.50)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:06:25 -0000 On Mon, Apr 29, 2019 at 9:55 AM Emmanuel Vadot wrote: > On Mon, 29 Apr 2019 09:25:05 -0400 > Kris Moore wrote: > > > On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot > > wrote: > > > > > > > > Hi Kris, > > > > > > On Sun, 28 Apr 2019 15:52:21 -0400 > > > wrote: > > > > > > > FreeBSD Community, > > > > > > > > > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > > > 13-current > > > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > > > which > > > > will allow users to perform all updating via the 'pkg' command > directly. > > > > Rather than trying to answer all questions in this announcement, > we've > > > > created a FAQ page with more details. Please refer to this page, and > let > > > us > > > > know if you have additional questions that we can include on that > page > > > going > > > > forward. > > > > > > > > > > While I appreciate the effort I have some doubt about your > > > "re-implementation" of pkgbase. I don't see any improvement compared to > > > what is in base currently, I even see downside of your implementation. > > > > > > - How do you plan with the need of updating kernel first, reboot and > > > updating the rest of the userland after ? (Needed for major and minor > > > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > > > -HEAD branch). This is still a problem with the base pkgbase. > > > > > > > We've written our own tool "sysutils/sysup" in GO which handles this. It > > performs updates using Boot-Environments to ensure that kernel/world are > > updated at same time. > > > > Which could never be imported into FreeBSD. > Not suggesting it should be. Just information on how we solved that problem in our own appliance / platforms. For FreeBSD it would need some tooling still to handle this style of updating, regardless of which pkg base is used. And for what it's worth, FreeBSD is all the poorer for not being able to bring modern language based tools into the base. Personally I'm hoping the shift to base-packages makes this a moot point since the idea of 'what is base' can be diluted to just a manifest of what gets installed out of box. Just my 2C on the matter though :) > > > > > > > > - This is even worse because you are using the same repository for > > > base and pkg so if a user pkg update and both kernel and pkg(8) needs > > > to be updated and pkg use a new syscall or capsicum thing it will be > > > updated first and couldn't proceed with the rest of the update (this is > > > a supposition, I haven't personally tested). > > > > > > > See above. > You can selectively update os/kernel and reboot before doing rest. > > > > > > > - It seems that multiple kernels isn't supported in your > > > implementation, this is already supported in pkgbase but still need > > > some love. This is an important point as it will allow user to choose > > > easily the kernel that they want to use and will also allow us > > > developper to push kernels with new features to help testing. > > > > > > > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll > get > > the Witness-enabled kernel installed alongside non-debugging one. > > Mhm no, the kernel-debug packages only add the debug file > in /usr/lib/debug/boot/ > I'm talking about installing multiple kernels in // > (i.e. /boot/kernel.GENERIC /boot/kernel.MYFEATUREIWANTTOTEST) like > describe here : > > https://wiki.freebsd.org/PkgBase#Project_goals_and_additional_unresolved_issues > in the "How to handle /boot/kernel and /boot/kernel.$KERNCONF" point. > > Incorrect, os/kernel-debug installs /boot/kernel-debug which is (on 13-CURRENT) the Witness enabled kernel. os/kernel-debug-symbols are the /usr/lib/debug bits. > > > > > > > I think that the only advantage that your solution offers is that if > > > we remove a componant of base (rcmds for example in 12-CURRENT) those > > > files would be removed as they are in the userland-base package while > > > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > > > will not be deleted in the user computer. > > > > > > > > > Correct, this is one of the things which prompted us to go this > direction. > > Being able to handle crazy mixed WITH/WITHOUT flags was important to us, > > current pkg base did not handle that so gracefully. > > Can you give me more info on this ? What where the WITH/WITHOUT flags > that causes problems ? > I may have to pick Miwi's brain on this, but I believe some of the issues we saw were when introducing flags such as WITHOUT_RADIUS. Additionally there is a runtime problem to solve. I.E. if you change flags mid-stream, and user updates, there was no clean way on pkg-side to remove those already installed granular packages. Not without external tooling anyway. > > -- > Emmanuel Vadot > From owner-freebsd-stable@freebsd.org Mon Apr 29 14:09:32 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43F7D158F869; Mon, 29 Apr 2019 14:09:32 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D9DC82F63; Mon, 29 Apr 2019 14:09:30 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x3TE9OGH072532; Mon, 29 Apr 2019 07:09:24 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x3TE9OFj072531; Mon, 29 Apr 2019 07:09:24 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201904291409.x3TE9OFj072531@gndrsh.dnsmgr.net> Subject: Re: CFT: FreeBSD Package Base In-Reply-To: <03b001d4fe92$f908c990$eb1a5cb0$@ixsystems.com> To: kris@ixsystems.com Date: Mon, 29 Apr 2019 07:09:24 -0700 (PDT) CC: "=?UTF-8?Q?'Goran_Meki=C4=87'?=" , "'Emmanuel Vadot'" , "'FreeBSD Stable'" , "'FreeBSD Current'" , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 1D9DC82F63 X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [3.54 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.29)[-0.286,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.86)[0.858,0]; IP_SCORE(0.04)[ip: (0.15), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.05), country: US(-0.06)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: gndrsh.dnsmgr.net]; NEURAL_SPAM_LONG(0.54)[0.537,0]; RCPT_COUNT_SEVEN(0.00)[9]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:09:32 -0000 > > Correct, this is ZFS only. And it's something we're using specific to FreeNAS / TrueOS, which is why I didn't originally mention it as apart of our CFT. Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", calling this FreeBSD pkg base when it is not was wrong, and miss leading. > For UFS, there will need to be additional care taken when doing updates. > > -- > Kris Moore > Vice President of Engineering > iXsystems, Inc > Ph: (408) 943-4100 > Ph: (408) 943-4101 > The Groundbreaking TrueNAS M-Series - > Enterprise Storage & Servers Driven By Open Source > > -----Original Message----- > From: Goran Meki? > Sent: Monday, April 29, 2019 9:43 AM > To: Kris Moore > Cc: Emmanuel Vadot ; FreeBSD Stable ; FreeBSD Current ; freebsd-pkgbase@freebsd.org; freebsd-pkg@freebsd.org; freebsd-hackers@freebsd.org; freebsd-ports@freebsd.org > Subject: Re: CFT: FreeBSD Package Base > > On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > > We've written our own tool "sysutils/sysup" in GO which handles this. > > It performs updates using Boot-Environments to ensure that > > kernel/world are updated at same time. > > If I'm right, UFS doesn't support boot environments, so how would it work for UFS based installs? > > I personally feel GO is a bit ackward choice of language for something that practically should be part of base. At least I would expect OS update/upgrade not to require any external package. > > Regards, > meka > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-stable@freebsd.org Mon Apr 29 14:12:00 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB298158FD1E for ; Mon, 29 Apr 2019 14:11:59 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EA7A183442 for ; Mon, 29 Apr 2019 14:11:58 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-io1-xd2f.google.com with SMTP id r18so9087367ioh.2 for ; Mon, 29 Apr 2019 07:11:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=40AQJesMSDreckm3W3V4ATFBI5KF5MeS/44F3ZnXX3c=; b=nYkpusi1yW4pdelw+f5HrIzcu5pHmJ6AtFAUPXdWxhEIzs2O+9mrVst5cpe+9IiTBj buKkqkoH03+f9OlGwIQIwdHPqD+UScMHbimGSo3iV3cGRgp8VY98wSjOBqXByUWBnud2 oxPAEavbyZO9JJUxTfmA5qMUlzAKGxH6VWU/FqaYoD4lqKzsTawG++ufv+n0mqRA7cDA 2Mj0AxeI1IJ+3k0xAoeJlETShu1v7mHO9efo3yRJ4sirKLAj7cBnP39wcn2xLM0/ZSzc 2b81tXEznDwNaTweWwNqLD7WeH2cL9qRzoBVrRzX7/E6YvJ11WYmft729e3Bg1ODOyZ3 lZsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=40AQJesMSDreckm3W3V4ATFBI5KF5MeS/44F3ZnXX3c=; b=JSbeKRemkiIzW1ftuW4YsG9uDagByTPAfuEjvD89toJ6zMZ/uvsKWuSyWpSHMYC6QX YiR01VaxZ+1GpZqKV9nEKUzL96lau3wUclftO4thJkxRNc7+r+y/SSBBvJcPPfVrD3KF ah2x/vaPQZJk1d/0qjYVsUvFe+Jy4AbEEj6O76G7xSgpk9zGt1Nr/m1QSoVz9OhUmUVB NDYSItgex6kdiwyckP7EshXBnpGE2Rfko2cqi5DL64XbR+PVUftAcKbVTCgBXUCMd9+I pLpm2hvJt5WWmdmtRb7ZBG0cAt6/x5qiTnerZVidyWWyDNPSA/Xj33mw8pxGrjnZR9yv wMRw== X-Gm-Message-State: APjAAAXENH4r/lqTKd3nUG8YmXAlCuxZ4W56mPqJnqRlGBlKMxZqJ5yD vWh8tMGSMIISF+sMfua4U5PAN8vzWq5PXKENftMAtw== X-Google-Smtp-Source: APXvYqyZczPSIfSA6FzUHIJ7nQ3lD+luMktNDaF/bc0ZvyXUkQSwBd9pNokOJ/XQddSd0tbCF/391BGIPpYk4cyTzcY= X-Received: by 2002:a6b:f809:: with SMTP id o9mr529367ioh.232.1556547118014; Mon, 29 Apr 2019 07:11:58 -0700 (PDT) MIME-Version: 1.0 References: <03b001d4fe92$f908c990$eb1a5cb0$@ixsystems.com> <201904291409.x3TE9OFj072531@gndrsh.dnsmgr.net> In-Reply-To: <201904291409.x3TE9OFj072531@gndrsh.dnsmgr.net> From: Kris Moore Date: Mon, 29 Apr 2019 10:11:47 -0400 Message-ID: Subject: Re: CFT: FreeBSD Package Base To: "Rodney W. Grimes" Cc: =?UTF-8?B?R29yYW4gTWVracSH?= , Emmanuel Vadot , FreeBSD Stable , FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org X-Rspamd-Queue-Id: EA7A183442 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=nYkpusi1; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::d2f as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-5.34 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; RCPT_COUNT_SEVEN(0.00)[9]; RCVD_IN_DNSWL_NONE(0.00)[f.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; IP_SCORE(-2.84)[ip: (-8.74), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; SUSPICIOUS_RECIPS(1.50)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:12:00 -0000 On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > > > Correct, this is ZFS only. And it's something we're using specific to > FreeNAS / TrueOS, which is why I didn't originally mention it as apart of > our CFT. > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", > calling this FreeBSD pkg base when it is not was wrong, > and miss leading. > Sorry, I disagree. This pkg base is independent of the ZFS tool we're using to wrangle boot-environments. Hence why it wasn't mentioned in the CFT. These base packages work the same as existing in-tree pkg base on UFS, no difference. If anything are probably safer due to being able to update all of userland in single extract operation, so you don't have out of order extraction of libc or some such. > > > For UFS, there will need to be additional care taken when doing updates. > > > > -- > > Kris Moore > > Vice President of Engineering > > iXsystems, Inc > > Ph: (408) 943-4100 > > Ph: (408) 943-4101 > > The Groundbreaking TrueNAS M-Series - > > Enterprise Storage & Servers Driven By Open Source > > > > -----Original Message----- > > From: Goran Meki? > > Sent: Monday, April 29, 2019 9:43 AM > > To: Kris Moore > > Cc: Emmanuel Vadot ; FreeBSD Stable < > freebsd-stable@freebsd.org>; FreeBSD Current ; > freebsd-pkgbase@freebsd.org; freebsd-pkg@freebsd.org; > freebsd-hackers@freebsd.org; freebsd-ports@freebsd.org > > Subject: Re: CFT: FreeBSD Package Base > > > > On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > > > We've written our own tool "sysutils/sysup" in GO which handles this. > > > It performs updates using Boot-Environments to ensure that > > > kernel/world are updated at same time. > > > > If I'm right, UFS doesn't support boot environments, so how would it > work for UFS based installs? > > > > I personally feel GO is a bit ackward choice of language for something > that practically should be part of base. At least I would expect OS > update/upgrade not to require any external package. > > > > Regards, > > meka > > > > _______________________________________________ > > freebsd-current@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > To unsubscribe, send any mail to " > freebsd-current-unsubscribe@freebsd.org" > > > > > > -- > Rod Grimes > rgrimes@freebsd.org > From owner-freebsd-stable@freebsd.org Mon Apr 29 14:26:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00A851590DAF for ; Mon, 29 Apr 2019 14:26:43 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x829.google.com (mail-qt1-x829.google.com [IPv6:2607:f8b0:4864:20::829]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2BFD884932 for ; Mon, 29 Apr 2019 14:26:42 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x829.google.com with SMTP id i31so3341229qti.13 for ; Mon, 29 Apr 2019 07:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=XpmADt+7fKLz9rWWj/FSMZ2PqEOB9SXz1yz4WIk59Lw=; b=2THWgFrHuqAZBmw3kXuHXIrvovLC0T9kpr+apiqn+Qn+UZKs7IiJfUlYhB899B3STz ix8JECw3zwTSiyOJlNQUp+A/Rfax3t0v4aCpJ+W7RzueoKQKAT1MONk7id/xjQEixhtq BVvRm41/Mct93ko5IT8rn5bMiCHTFagjUV1u9SmTw9G0/wbMKe0y9HV3ehFlh271BE+Z yPc/en3glM+L0Oo479CZKQlGf7pXopTnkdmYJ1xk0vrN03C/VDKZCsdZwqivSnttt0v8 OMZn181pcegDmzBv5Br452GgLf7aGsUL9zx0qBg3Ms/HSC38NwboQNMk8qdt4/8IlOf1 Wfuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=XpmADt+7fKLz9rWWj/FSMZ2PqEOB9SXz1yz4WIk59Lw=; b=AJWPuBqFOymay3N2Yqt09Guzjqhb6s0pxkDfADqXVd7cc/BoXgjsycBl/d7ZDmJ/du FprUnBHLhTBltNxUWLY7yE3q7Xhmkdhs3aK13ZVhsuHU0mvFsUwgvZ5qa8HUZX6t/PJr adSM8WgmsVgvR7c/GF/5qy1qulnJI5/fKEhZ5ALnPIQtcz8X2e1pjn1VotUZ2B5ZPDFa z0LCxTU5WJxThKEV2NiHY2Q/uib0NXT3inerk83whUfm2diY8SkyQ11lPeh3KbeUnPxP ip0qoFOCNckg8RKC3Ai4mjdcZgshMtfAKmQn6pxBB3IDYi7cHBNSBSZTKYiUWiYLcP4h Q+NQ== X-Gm-Message-State: APjAAAUB9ix56Wx/8La7Vri1BErdUXF+4/0tQa5OlorOTrBhhrBJxdU9 M25upHId0nhqPalzL//F5OfvbPSBtLRz6jjMEbURUwgV X-Google-Smtp-Source: APXvYqxwd3uJCuN9fVgY+22EcmEYaK+ULCg77YHFL4FaXtYDO+YDgQL6J5bnD6x5S8bUa/3df1I98FvUFPajujo6J3g= X-Received: by 2002:aed:3e93:: with SMTP id n19mr26411080qtf.345.1556548001506; Mon, 29 Apr 2019 07:26:41 -0700 (PDT) MIME-Version: 1.0 References: <9a96b1b5-9337-fcae-1a2a-69d7bb24a5b3@denninger.net> <1866e238-e2a1-ef4e-bee5-5a2f14e35b22@denninger.net> <3d2ad225-b223-e9db-cce8-8250571b92c9@FreeBSD.org> <2bc8a172-6168-5ba9-056c-80455eabc82b@denninger.net> <2c23c0de-1802-37be-323e-d390037c6a84@denninger.net> <864062ab-f68b-7e63-c3da-539d1e9714f9@denninger.net> <6dc1bad1-05b8-2c65-99d3-61c547007dfe@denninger.net> <758d5611-c3cf-82dd-220f-a775a57bdd0b@multiplay.co.uk> <3f53389a-0cb5-d106-1f64-bbc2123e975c@denninger.net> <8108da18-2cdd-fa29-983c-3ae7be6be412@multiplay.co.uk> <5ff09821-98e2-8520-1a46-e6541c9c8891@denninger.net> In-Reply-To: <5ff09821-98e2-8520-1a46-e6541c9c8891@denninger.net> From: Warner Losh Date: Mon, 29 Apr 2019 08:26:29 -0600 Message-ID: Subject: Re: Concern: ZFS Mirror issues (12.STABLE and firmware 19 .v. 20) [[UPDATE w/more tests]] To: Karl Denninger Cc: FreeBSD-STABLE Mailing List , Steven Hartland X-Rspamd-Queue-Id: 2BFD884932 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=2THWgFrH X-Spamd-Result: default: False [-5.91 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT1.aspmx.l.google.com]; RCVD_IN_DNSWL_NONE(0.00)[9.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.96)[-0.964,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; IP_SCORE(-2.93)[ip: (-9.19), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:26:43 -0000 On Sun, Apr 28, 2019 at 4:03 PM Karl Denninger wrote: > On 4/20/2019 15:56, Steven Hartland wrote: > > Thanks for extra info, the next question would be have you eliminated > > that corruption exists before the disk is removed? > > > > Would be interesting to add a zpool scrub to confirm this isn't the > > case before the disk removal is attempted. > > > > Regards > > Steve > > > > On 20/04/2019 18:35, Karl Denninger wrote: > >> > >> On 4/20/2019 10:50, Steven Hartland wrote: > >>> Have you eliminated geli as possible source? > >> No; I could conceivably do so by re-creating another backup volume > >> set without geli-encrypting the drives, but I do not have an extra > >> set of drives of the capacity required laying around to do that. I > >> would have to do it with lower-capacity disks, which I can attempt if > >> you think it would help. I *do* have open slots in the drive > >> backplane to set up a second "test" unit of this sort. For reasons > >> below it will take at least a couple of weeks to get good data on > >> whether the problem exists without geli, however. > >> > Ok, following up on this with more data.... > > First step taken was to create a *second* backup pool (I have the > backplane slots open, fortunately) with three different disks but *no > encryption.* > > I ran both side-by-side for several days, with the *unencrypted* one > operating with one disk detached and offline (pulled physically) just as > I do normally. Then I swapped the two using the same paradigm. > > The difference was *dramatic* -- the resilver did *not* scan the entire > disk; it only copied the changed blocks and was finished FAST. A > subsequent scrub came up 100% clean. > > Next I put THOSE disks in the vault (so as to make sure I didn't get > hosed if something went wrong) and re-initialized the pool in question, > leaving only the "geli" alone (in other words I zpool destroy'd the pool > and then re-created it with all three disks connected and > geli-attached.) The purpose for doing this was to eliminate the > possibility of old corruption somewhere, or some sort of problem with > multiple, spanning years, in-place "zpool upgrade" commands. Then I ran > a base backup to initialize all three volumes, detached one and yanked > it out of the backplane, as would be the usual, leaving the other two > online and operating. > > I ran backups as usual for most of last week after doing this, with the > 61.eli and 62-1.eli volumes online, and 62-2 physically out of the > backplane. > > Today I swapped them again as I usually do (e.g. offline 62.1, geli > detach, camcontrol standby and then yank it -- then insert the 62-2 > volume, geli attach and zpool online) and this is happening: > > [\u@NewFS /home/karl]# zpool status backup > pool: backup > state: DEGRADED > status: One or more devices is currently being resilvered. The pool will > continue to function, possibly in a degraded state. > action: Wait for the resilver to complete. > scan: resilver in progress since Sun Apr 28 12:57:47 2019 > 2.48T scanned at 202M/s, 1.89T issued at 154M/s, 3.27T total > 1.89T resilvered, 57.70% done, 0 days 02:37:14 to go > config: > > NAME STATE READ WRITE CKSUM > backup DEGRADED 0 0 0 > mirror-0 DEGRADED 0 0 0 > gpt/backup61.eli ONLINE 0 0 0 > 11295390187305954877 OFFLINE 0 0 0 was > /dev/gpt/backup62-1.eli > gpt/backup62-2.eli ONLINE 0 0 0 > > errors: No known data errors > > The "3.27T" number is accurate (by "zpool list") for the space in use. > > There is not a snowball's chance in Hades that anywhere near 1.89T of > that data (thus far, and it ain't done as you can see!) was modified > between when all three disks were online and when the 62-2.eli volume > was swapped back in for 62.1.eli. No possible way. Maybe some > 100-200Gb of data has been touched across the backed-up filesystems in > the last three-ish days but there's just flat-out no way it's more than > that; this would imply an entropy of well over 50% of the writeable data > on this box in less than a week! That's NOT possible. Further it's not > 100%; it shows 2.48T scanned but 1.89T actually written to the other drive. > > So something is definitely foooged here and it does appear that geli is > involved in it. Whatever is foooging zfs the resilver process thinks it > has to recopy MOST (but not all!) of the blocks in use, it appears, from > the 61.eli volume to the 62-2.eli volume. > > The question is what would lead ZFS to think it has to do that -- it > clearly DOES NOT as a *much* smaller percentage of the total TXG set on > 61.eli was modified while 62-2.eli was offline and 62.1.eli was online. > > Again I note that on 11.1 and previous this resilver was a rapid > operation; whatever was actually changed got copied but the system never > copied *nearly everything* on a resilver, including data that had not > been changed at all, on a mirrored set. > > Obviously on a Raidz volume you have to go through the entire data > structure because parity has to be recomputed and blocks regenerated but > on a mirror only the changed TXGs need to be looked at and copied. TXGs > that were on both disks at the time second one was taken offline do not > need to be touched *at all* since they're already on the target! > > What's going on here? > Silly question, and maybe I missed you doing this, but have you tried dd'ing zeros each of the eli volumes that you are putitng the zpool on? Does that make the problem go away? Maybe there's something weird going on with un-written sectors and/or reusing sectors with weird content that is accentuated with geli, or that causes an error path to trigger that doesn't with !geli. Otherwise, this testing sounds quite complete. Warner From owner-freebsd-stable@freebsd.org Mon Apr 29 14:34:26 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C15031591384 for ; Mon, 29 Apr 2019 14:34:26 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-yb1-xb33.google.com (mail-yb1-xb33.google.com [IPv6:2607:f8b0:4864:20::b33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C14985046 for ; Mon, 29 Apr 2019 14:34:25 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-yb1-xb33.google.com with SMTP id m186so1341721ybf.13 for ; Mon, 29 Apr 2019 07:34:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-transfer-encoding:thread-index :content-language; bh=ZqHr2f+E82t1/GGqjLx8j4ilpHoQ1L6+AoKuzJ+v1Dk=; b=ak/QEgA22OuHs4wv4+LEPwHMQpTLFZqFhZbOI8nMtNUm2QxNlGEugIOrd6CXsk6XeT LWvrj9tENq/KaDWdPmzaiRI8H5V1ib0nASeeg0NVccu9cEkX2SEI5fsodU+1jaZtemq1 SyjQvKgDom+cToIZMK7bthm3Kk/GUR2L7zfwhy4UZ1HOIXZvfAdIMg/Hoy51uKHkcR3E YDQJdw4aLDNlw2IxocQtg+kVLKVyoZn3xMKT6MoPuZ42IOMu2rMOoA6LTyxfglfN7i+0 EgQg7QobkB25BkFKeHMu+bID4HK6SHigV20E1DY7C3G07YTMkZa5L6St5Rqq7Q20PXQA v4zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:thread-index :content-language; bh=ZqHr2f+E82t1/GGqjLx8j4ilpHoQ1L6+AoKuzJ+v1Dk=; b=dy4FgBVuZmE6K9Mbkch815iCySu+LwBzIs8k4yUJ5MQv649XtCMahUB9fIMte0bAJV imk0pgsbJF0jtRUeiQtXYZ0dintPnr4mAU6y4uADTrTNZ31LAzL/fhRto4FQsLYmygMF H6P3vgByWbuZDbAFyy8vEy5YbtTFgt1/xPV93/c0MJkZ2w9bLuB33CO3Xqc1ezFMnb06 eZ0Gpq0PeUFzf8IZI/ZBDKnJUIFwt2HDnUz4QFuMZpnKyiUv+GhZ5DCYPsh4esn2VoGN kimh5eLYYYoDLXCRYHzMa+GeVSJWQpp54KpgXD7JhIEZF7ava7PJIOdD/Bwusl/v5qUR EvLw== X-Gm-Message-State: APjAAAVG8zu8XqjkW9ZU4LUuo6awMmSXzg0Y0Gd6lcqs7hKw0axpgEmr 59mPDORLTDwT9PYsdNvY9m6t0Q== X-Google-Smtp-Source: APXvYqyj8aKJSTL3doWXofSGGuSBRIl0mlpM6RBNJhl/PQmjrymmMikrHQfk/PWWrpef+cmYu/maUw== X-Received: by 2002:a25:dcc9:: with SMTP id y192mr5235384ybe.10.1556548464445; Mon, 29 Apr 2019 07:34:24 -0700 (PDT) Received: from KrisYogaC930 ([12.189.233.130]) by smtp.gmail.com with ESMTPSA id l202sm1283609ywc.30.2019.04.29.07.34.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Apr 2019 07:34:23 -0700 (PDT) From: To: "'Emmanuel Vadot'" Cc: "'FreeBSD Stable'" , , , "'FreeBSD Current'" , , References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> <20190429155522.81dcc291a6b252884219de1e@bidouilliste.com> <20190429162847.a32262b6073f5523bdbe0ed4@bidouilliste.com> In-Reply-To: <20190429162847.a32262b6073f5523bdbe0ed4@bidouilliste.com> Subject: RE: CFT: FreeBSD Package Base Date: Mon, 29 Apr 2019 10:34:22 -0400 Message-ID: <047601d4fe98$a3c77930$eb566b90$@ixsystems.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQFBRupbev+o6AHqgfoJLZwOgBKxNQFhdrr5Ad4+GxoCGFDnowGvv4BvAf5pkjSnMnEXQA== Content-Language: en-us X-Rspamd-Queue-Id: 5C14985046 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=ak/QEgA2; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::b33 as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-0.92 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(0.00)[+ip6:2607:f8b0:4000::/36]; URIBL_RED(3.50)[trueos.org.multi.uribl.com]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; RCPT_COUNT_SEVEN(0.00)[7]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; DMARC_POLICY_ALLOW(0.00)[ixsystems.com,none]; HAS_ANON_DOMAIN(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-3.02)[ip: (-9.61), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(0.00)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NO_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[3.3.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:34:27 -0000 > > > > > Incorrect, os/kernel-debug installs /boot/kernel-debug which is (on > > 13-CURRENT) the Witness enabled kernel. os/kernel-debug-symbols are > > the /usr/lib/debug bits. > > I only see kernel-20190420203550_1.txz and kernel-debug- > 20190420203550.txz in https://pkg.trueos.org/pkg/freebsd- > pkgbase/FreeBSD%3A13%3Aamd64/latest/All/ > and kernel-debug only contain the debug files. > If I'm not looking in the right directory please correct me. > > > -- > Emmanuel Vadot Ahh, you are correct. I checked and those packages haven't pushed to the mirrors yet, Jenkins is still chewing on a build of them here. I was using the 12-stable packages yesterday which has these changes. They should be synced up to the mirrors in the next 24-48 hours. Sorry about the confusion. -- Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source From owner-freebsd-stable@freebsd.org Mon Apr 29 14:35:30 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A517F15914ED; Mon, 29 Apr 2019 14:35:30 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.blih.net", Issuer "mail.blih.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3703A8524C; Mon, 29 Apr 2019 14:35:29 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mail.blih.net (mail.blih.net [212.83.177.182]) by mail.blih.net (OpenSMTPD) with ESMTP id 11eeb002; Mon, 29 Apr 2019 16:28:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=bidouilliste.com; h=date :from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; s=mail; bh=pd7/fUDtndS60t3fDY8gFbr+Two=; b=AqXJs+17NHV0FINgAotTesH59c+B 109keWOxWL1+TMOSF+NDVQxB4H+HXTzoufryQIkVE+TtguwWQ9iHIhyNKMxw6DMp RScGGZ0hI8U8OtkWinOEOQxUm/tzAs8uQptSM4NCVj/OZ0h6MVS6SKnAwxTN4a1H ENep/QNErV197Gk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=bidouilliste.com; h=date :from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-type:content-transfer-encoding; q=dns; s= mail; b=n3JmpHsLPTGSrGwCTV7BGFwAwOPpILae22RBID1ZZiMRhksxrNHRWKqz pWHc2ldbxvBXtXPy51rxP3LbvvXIpZOYQcSKynauU421jQ9Ebow1gQW5B3cJjAYY 2OyQpeurrNKXkP5OnQLesEspie7y6WNyOknQ6b0jg2MyAELY2ag= Received: from skull.home.blih.net (ip-9.net-89-3-105.rev.numericable.fr [89.3.105.9]) by mail.blih.net (OpenSMTPD) with ESMTPSA id d0d7e356 TLS version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO; Mon, 29 Apr 2019 16:28:47 +0200 (CEST) Date: Mon, 29 Apr 2019 16:28:47 +0200 From: Emmanuel Vadot To: Kris Moore Cc: FreeBSD Stable , freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-ports@freebsd.org Subject: Re: CFT: FreeBSD Package Base Message-Id: <20190429162847.a32262b6073f5523bdbe0ed4@bidouilliste.com> In-Reply-To: References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> <20190429155522.81dcc291a6b252884219de1e@bidouilliste.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; amd64-portbld-freebsd13.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 3703A8524C X-Spamd-Bar: ++++++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bidouilliste.com header.s=mail header.b=AqXJs+17; spf=pass (mx1.freebsd.org: domain of manu@bidouilliste.com designates 212.83.177.182 as permitted sender) smtp.mailfrom=manu@bidouilliste.com X-Spamd-Result: default: False [8.90 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:212.83.177.182/32]; MV_CASE(0.50)[]; URIBL_RED(3.50)[trueos.org.multi.uribl.com]; DKIM_TRACE(0.00)[bidouilliste.com:+]; MX_GOOD(-0.01)[cached: mx-backup.blih.net]; RCPT_COUNT_SEVEN(0.00)[7]; HAS_ANON_DOMAIN(0.10)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(0.39)[ip: (-0.85), ipnet: 212.83.160.0/19(2.96), asn: 12876(-0.17), country: FR(-0.01)]; ASN(0.00)[asn:12876, ipnet:212.83.160.0/19, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(0.00)[bidouilliste.com:s=mail]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.92)[0.924,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[bidouilliste.com]; NEURAL_SPAM_MEDIUM(0.90)[0.897,0]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(1.00)[0.998,0]; RCVD_COUNT_TWO(0.00)[2]; GREYLIST(0.00)[pass,body]; SUSPICIOUS_RECIPS(1.50)[] X-Spam: Yes X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:35:30 -0000 On Mon, 29 Apr 2019 10:05:59 -0400 Kris Moore wrote: > On Mon, Apr 29, 2019 at 9:55 AM Emmanuel Vadot > wrote: > > > On Mon, 29 Apr 2019 09:25:05 -0400 > > Kris Moore wrote: > > > > > On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot > > > wrote: > > > > > > > > > > > Hi Kris, > > > > > > > > On Sun, 28 Apr 2019 15:52:21 -0400 > > > > wrote: > > > > > > > > > FreeBSD Community, > > > > > > > > > > > > > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > > > > 13-current > > > > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > > > > which > > > > > will allow users to perform all updating via the 'pkg' command > > directly. > > > > > Rather than trying to answer all questions in this announcement, > > we've > > > > > created a FAQ page with more details. Please refer to this page, and > > let > > > > us > > > > > know if you have additional questions that we can include on that > > page > > > > going > > > > > forward. > > > > > > > > > > > > > While I appreciate the effort I have some doubt about your > > > > "re-implementation" of pkgbase. I don't see any improvement compared to > > > > what is in base currently, I even see downside of your implementation. > > > > > > > > - How do you plan with the need of updating kernel first, reboot and > > > > updating the rest of the userland after ? (Needed for major and minor > > > > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > > > > -HEAD branch). This is still a problem with the base pkgbase. > > > > > > > > > > We've written our own tool "sysutils/sysup" in GO which handles this. It > > > performs updates using Boot-Environments to ensure that kernel/world are > > > updated at same time. > > > > > > > Which could never be imported into FreeBSD. > > > > Not suggesting it should be. Just information on how we solved that problem > in our own appliance / platforms. For FreeBSD it would need some tooling > still to handle this style of updating, regardless of which pkg base is > used. > > And for what it's worth, FreeBSD is all the poorer for not being able to > bring modern language based tools into the base. Personally I'm hoping the > shift to base-packages makes this a moot point since the idea of 'what is > base' can be diluted to just a manifest of what gets installed out of box. > Just my 2C on the matter though :) > > > > > > > > > > > > > > - This is even worse because you are using the same repository for > > > > base and pkg so if a user pkg update and both kernel and pkg(8) needs > > > > to be updated and pkg use a new syscall or capsicum thing it will be > > > > updated first and couldn't proceed with the rest of the update (this is > > > > a supposition, I haven't personally tested). > > > > > > > > > > See above. > > > > You can selectively update os/kernel and reboot before doing rest. > > > > > > > > > > > > - It seems that multiple kernels isn't supported in your > > > > implementation, this is already supported in pkgbase but still need > > > > some love. This is an important point as it will allow user to choose > > > > easily the kernel that they want to use and will also allow us > > > > developper to push kernels with new features to help testing. > > > > > > > > > > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll > > get > > > the Witness-enabled kernel installed alongside non-debugging one. > > > > Mhm no, the kernel-debug packages only add the debug file > > in /usr/lib/debug/boot/ > > I'm talking about installing multiple kernels in // > > (i.e. /boot/kernel.GENERIC /boot/kernel.MYFEATUREIWANTTOTEST) like > > describe here : > > > > https://wiki.freebsd.org/PkgBase#Project_goals_and_additional_unresolved_issues > > in the "How to handle /boot/kernel and /boot/kernel.$KERNCONF" point. > > > > > Incorrect, os/kernel-debug installs /boot/kernel-debug which is (on > 13-CURRENT) the Witness enabled kernel. os/kernel-debug-symbols are the > /usr/lib/debug bits. I only see kernel-20190420203550_1.txz and kernel-debug-20190420203550.txz in https://pkg.trueos.org/pkg/freebsd-pkgbase/FreeBSD%3A13%3Aamd64/latest/All/ and kernel-debug only contain the debug files. If I'm not looking in the right directory please correct me. > > > > > > > > > > > I think that the only advantage that your solution offers is that if > > > > we remove a componant of base (rcmds for example in 12-CURRENT) those > > > > files would be removed as they are in the userland-base package while > > > > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > > > > will not be deleted in the user computer. > > > > > > > > > > > > > Correct, this is one of the things which prompted us to go this > > direction. > > > Being able to handle crazy mixed WITH/WITHOUT flags was important to us, > > > current pkg base did not handle that so gracefully. > > > > Can you give me more info on this ? What where the WITH/WITHOUT flags > > that causes problems ? > > > > > I may have to pick Miwi's brain on this, but I believe some of the issues > we saw were when introducing flags such as WITHOUT_RADIUS. Additionally > there is a runtime problem to solve. I.E. if you change flags mid-stream, > and user updates, there was no clean way on pkg-side to remove those > already installed granular packages. Not without external tooling anyway. > > > > > > > -- > > Emmanuel Vadot > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Emmanuel Vadot From owner-freebsd-stable@freebsd.org Mon Apr 29 14:50:37 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46D751592682; Mon, 29 Apr 2019 14:50:37 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from ms-10.1blu.de (ms-10.1blu.de [178.254.4.101]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4225D8646F; Mon, 29 Apr 2019 14:50:35 +0000 (UTC) (envelope-from guru@unixarea.de) Received: from [188.174.53.230] (helo=localhost.unixarea.de) by ms-10.1blu.de with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.86_2) (envelope-from ) id 1hL7bz-0002Y9-RN; Mon, 29 Apr 2019 16:50:31 +0200 Received: from localhost.my.domain (localhost [127.0.0.1]) by localhost.unixarea.de (8.15.2/8.14.9) with ESMTPS id x3TEoUWV003159 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 29 Apr 2019 16:50:30 +0200 (CEST) (envelope-from guru@unixarea.de) Received: (from guru@localhost) by localhost.my.domain (8.15.2/8.14.9/Submit) id x3TEoTvi003158; Mon, 29 Apr 2019 16:50:29 +0200 (CEST) (envelope-from guru@unixarea.de) X-Authentication-Warning: localhost.my.domain: guru set sender to guru@unixarea.de using -f Date: Mon, 29 Apr 2019 16:50:28 +0200 From: Matthias Apitz To: Emmanuel Vadot Cc: Kris Moore , FreeBSD Stable , freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-ports@freebsd.org Subject: Re: CFT: FreeBSD Package Base Message-ID: <20190429145028.GA3124@c720-r342378> Reply-To: Matthias Apitz Mail-Followup-To: Emmanuel Vadot , Kris Moore , FreeBSD Stable , freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-ports@freebsd.org References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> <20190429155522.81dcc291a6b252884219de1e@bidouilliste.com> <20190429162847.a32262b6073f5523bdbe0ed4@bidouilliste.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="SLDf9lqlvOQaIe6s" Content-Disposition: inline In-Reply-To: <20190429162847.a32262b6073f5523bdbe0ed4@bidouilliste.com> X-Operating-System: FreeBSD 13.0-CURRENT r342378 (amd64) X-message-flag: Mails containing HTML will not be read! Please send only plain text. User-Agent: Mutt/1.11.1 (2018-12-01) X-Con-Id: 51246 X-Con-U: 0-guru X-Originating-IP: 188.174.53.230 X-Rspamd-Queue-Id: 4225D8646F X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-5.27 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[guru@unixarea.de]; HAS_XOIP(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(-3.06)[ip: (-8.83), ipnet: 178.254.0.0/19(-3.59), asn: 42730(-2.85), country: DE(-0.01)]; HAS_XAW(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; MX_GOOD(-0.01)[cached: mail.unixarea.de]; NEURAL_HAM_SHORT(-0.91)[-0.906,0]; RCPT_COUNT_SEVEN(0.00)[8]; SIGNED_PGP(-2.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[230.53.174.188.zen.spamhaus.org : 127.0.0.10]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:42730, ipnet:178.254.0.0/19, country:DE]; RCVD_IN_DNSWL_LOW(-0.10)[101.4.254.178.list.dnswl.org : 127.0.5.1]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[unixarea.de]; AUTH_NA(1.00)[]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_NA(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:50:37 -0000 --SLDf9lqlvOQaIe6s Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Why this thread has to go to all these lists? I receive any mail 5 times! matthias --=20 Matthias Apitz, =E2=9C=89 guru@unixarea.de, http://www.unixarea.de/ +49-176= -38902045 Public GnuPG key: http://www.unixarea.de/key.pub N =E2=82=AC I N zur EU! "Gegen das EU-Europa der Banken, Konzerne und Kriegstreiber. F=C3=BCr ein soziales und friedliches Europa der V=C3=B6lker." DKP --SLDf9lqlvOQaIe6s Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEXmn7rBYYViyzy/vBR8z35Hb+nREFAlzHDzEACgkQR8z35Hb+ nREGJw//d4Me8jC0GcEDanOfjeW5fmHAtwKrbve2YkvvA0Iw3Wzmdn9nyh7s2K/H /lzTcGqx4BaKKM4uGhCtD62aVD2xEi9Hv6EPLz9W0MM3pNBMmrgzQYYdYZcA2N7Y pELE84TpCfrPY9n2kIe9mJVHxAcaWcQzltbCvGg1WFNQ1UfxUvyrKFl5ZnwJidHG mo5EyBIX4x/IYMnVRJggapH2agOhSinIlnEFxLnHZ0+/NAIlb4SgbqW04klZZR+Z x0wTwRYN1y2fRXauMuB4mvnv9PJOyYuK8NzSW/Dp6RSi9QhGhvvFbNxnJqs8I/dS WfD4i4MciK7p1wBXUnHcqvsCk7J+z05CXbTzplYoAD+t3ToI1EHlHZSxX0tO2N3z ymKc+bdLhOYVCargYh9Ow9otwRf5+isK0fQvuRAR9MH98EJMAvpNi969VKK1Nj/a teHR0Rf6hpLwFn3/mcwvLCRqoG49DfDU9CpEeISG8fguTnR7ZoE3wtyfSkw9+DoL i4RPA0Yi52iQ2QiVXWY8L8hYxzyoK2nBFOsyBDxqCxCeZKWyc0qgRcmGrIZSVJVT gXdXOLAZYW2pVlhciKtFAzusOTJ/cCZAksO/mwcNaXMaE8DIgOxYO2g/+kDhnIpj Ton1X+UbjzEM+61iLZrQKwWlwXTIdCGcEPFHkje/Wz7Mt3lr+0I= =JlvZ -----END PGP SIGNATURE----- --SLDf9lqlvOQaIe6s-- From owner-freebsd-stable@freebsd.org Mon Apr 29 14:50:56 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6CC7159273C for ; Mon, 29 Apr 2019 14:50:55 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-yw1-xc33.google.com (mail-yw1-xc33.google.com [IPv6:2607:f8b0:4864:20::c33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 85B6286518 for ; Mon, 29 Apr 2019 14:50:54 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-yw1-xc33.google.com with SMTP id v15so3767497ywe.13 for ; Mon, 29 Apr 2019 07:50:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-transfer-encoding:thread-index :content-language; bh=yDWcEIUqpI/8RHhni+7NV8CWfzwUCh3D7x8chiwsbA4=; b=GVbnM8jWMiaofl9yIT2dur5Gd4zyt/k/MWSNopdWF4vt/eSwtQOKucH8/6kKox73y3 V8Ji5goVdsrFYyMfXRPjOk0TiDqxLQozUlg9tCMZaXKEbl0Noo2Nfq/ANLhZ02hQrVsu cYO0fw0uRnKVNVyPS3XqHtkNq7u7s83v0Lv4woTqhBO+yrxv0A04k6yh6PmzralbkOdr WI8fdk9FmrOTp/iRImFUiEPH9mcK71sNXcDsMqyGJ0szb1GE43X8U3b2sFuMGPv8qPGs Sc3rK9OiYNThtG6xQUMr3/u43wO8XHOQ6wsmakxPJ/vSsc5w/BGQBAOV+LKDoKLRYTGF yp8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:thread-index :content-language; bh=yDWcEIUqpI/8RHhni+7NV8CWfzwUCh3D7x8chiwsbA4=; b=SV2+wvORG7dNAm/ujqMfgPe6jo4mg6pu8G8X6I88rfoosTasRQ7YY2dhaaIp/xQyfL oucYLasRrHFos46dX8BdocufykPDRr1N5+enBQaHNKHyWsAwhIHIRr+Ejb8VqIwkiaeJ nsVsTzHzem3j+j/hhyFBprCbD/+k29lFglGFB8D2DKvyFU7IV1n54+DA8W/fDlR70doM So0hVD0nv7q+Br12/lPtpTpG2t7MSAdAVxFWOSliQmSEu6xhXSqtVGURr3wq3dKKBs5+ PE055R/KvwLwxtpWICbgpksXy+gaNnVc3o9od4ezrKZfLCJYSGIRLcPCc52pJkuJKRUo XKWQ== X-Gm-Message-State: APjAAAVFP81EtNkFE75IbHZGviblD/UFgjCjTc7RxA2TT0uMZmzlKgR1 IUY6gnH7N56ugbuWK58IsD+YsA== X-Google-Smtp-Source: APXvYqxOxZ2n/jEWWoIVaMIBGxQcXd/lwsQcs8+tKZyzj3ZcesHaemSZTZHfpTMMdj0Oh/OWIfulTw== X-Received: by 2002:a0d:f441:: with SMTP id d62mr38198373ywf.350.1556549453101; Mon, 29 Apr 2019 07:50:53 -0700 (PDT) Received: from KrisYogaC930 ([12.189.233.130]) by smtp.gmail.com with ESMTPSA id c184sm2254330ywe.75.2019.04.29.07.50.52 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Apr 2019 07:50:52 -0700 (PDT) From: To: "'Rodney W. Grimes'" Cc: =?iso-8859-2?Q?'Goran_Meki=E6'?= , "'Emmanuel Vadot'" , "'FreeBSD Stable'" , "'FreeBSD Current'" , , , , References: <201904291441.x3TEfMid072751@gndrsh.dnsmgr.net> In-Reply-To: <201904291441.x3TEfMid072751@gndrsh.dnsmgr.net> Subject: RE: CFT: FreeBSD Package Base Date: Mon, 29 Apr 2019 10:50:51 -0400 Message-ID: <053101d4fe9a$f1152320$d33f6960$@ixsystems.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQM63uti6cm6tb0qpYZSIKL3MVPss6OHdbsQ Content-Language: en-us X-Rspamd-Queue-Id: 85B6286518 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=GVbnM8jW; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::c33 as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-5.32 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-2.83)[ip: (-8.66), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; FROM_NO_DN(0.00)[]; RCPT_COUNT_SEVEN(0.00)[9]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.3.c.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:50:56 -0000 > -----Original Message----- > From: Rodney W. Grimes > Sent: Monday, April 29, 2019 10:41 AM > To: Kris Moore > Cc: Rodney W. Grimes ; Goran Meki=E6 > ; Emmanuel Vadot ; FreeBSD > Stable ; FreeBSD Current current@freebsd.org>; freebsd-pkgbase@freebsd.org; freebsd- > pkg@freebsd.org; freebsd-hackers@freebsd.org; = freebsd-ports@freebsd.org > Subject: Re: CFT: FreeBSD Package Base >=20 > > On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < > > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > > > > > > > > > Correct, this is ZFS only. And it's something we're using = specific > > > > to > > > FreeNAS / TrueOS, which is why I didn't originally mention it as > > > apart of our CFT. > > > > > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", = calling > > > this FreeBSD pkg base when it is not was wrong, and miss leading. > > > > > > > Sorry, I disagree. > Which is fine. >=20 > > This pkg base is independent of the ZFS tool we're using > > to wrangle boot-environments. Hence why it wasn't mentioned in the = CFT. > > These base packages work the same as existing in-tree pkg base on = UFS, no > > difference. If anything are probably safer due to being able to = update all > > of userland in single extract operation, so you don't have out of = order > > extraction of libc or some such. >=20 > You missed the major string change and focused on the edge, > No comment on calling iXsystems :stuff: FreeBSD instead of > FreeNAS/TrueOS? >=20 > That was the major point of my statement, your miss leading the user > community, you yourself said this would never be imported into FreeBSD > base, so I see no reason that it should be called "FreeBSD package = Base", > as it is not, that is a different project. >=20 > -- > Rod Grimes rgrimes@freebsd.org I think somehow you've missed the entire point here. This is being = brought forth as a FreeBSD CFT in the hopes of upstream adoption. No misleading = here whatsoever. The only thing that I wouldn't expect to be imported into = base was this external tool we use on FreeNAS/TrueOS to handle our specific use-case of ZFS only. Total strawman here. Seriously, suggest you bother looking at it and reading further to get = the full context. If anything this is far less invasive since it doesn't = require lots of hacking on base, and can even be used to package old versions of FreeBSD if desired. The only thing I changed to make these images was a patch to bsdinstall to replace dist-file extraction with 'pkg install userland kernel pkg ...'. From owner-freebsd-stable@freebsd.org Mon Apr 29 14:51:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 532411592850 for ; Mon, 29 Apr 2019 14:51:39 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-yb1-xb30.google.com (mail-yb1-xb30.google.com [IPv6:2607:f8b0:4864:20::b30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DB2686773 for ; Mon, 29 Apr 2019 14:51:38 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-yb1-xb30.google.com with SMTP id a13so3863859ybl.8 for ; Mon, 29 Apr 2019 07:51:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-transfer-encoding:thread-index :content-language; bh=AQIQkMP6AQck7in5qJFQ2vtmaTwARnI28db71iO9Vnk=; b=GadSAgMRs6W+apHAGxU+e48wXWBzpjkHXvhMgeLTu3DiJSpdGoQDnhlxNAGxuX4rIF ynv+DUFbSXj+tKaYAGX0+54YpINfKvZSbVzJ3GEL4pETjHjf1G80wGlNleziLdh/i1de ANkGRCBk3VYVHW21SG4QFTfTf8OOrfvnNsT7kvcQF6ee+NEnMhefQUC4DTS8ro86qxWw OMuhIOe7CQafhSHnHNz9l+GSANUFcyvsF1fxXGNID5yV5We2+Hvw+gI9dK1pbMaoKGBf MEk1ZZ1X1t3Tw9au74gaZLcDPY+DE2OpZc3o9mDoCSMbkaw0AHrARBfyLNIZhfga+R1R 6DrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:thread-index :content-language; bh=AQIQkMP6AQck7in5qJFQ2vtmaTwARnI28db71iO9Vnk=; b=l86Ucf3t+9j0asulDpFmEXOy9b2evALu7l13/YCtAjiH4002M7lZWY1MkHL7b9F0ju sf7FCmP678IeF5O3CW9/kTi/9mpUbwyedEtEGNQdWFyEUem4bC3m6shLtdVQd1fM/mTa zc6koALvB1HWuLOD+YrGjxKgrrjZW/BEmREWPCnORFjOwBiKTsURFKaVW1za7VcVoHnL 5G6B1XdwCK72KsT2oMtfDMjNbotV6JeR6ILGsWOd76lWrZrPGuSgtGihNygxjeHdOXNb h07+/aT1dZsT8ysamHbFn/8Q4QxxXJ07dQKuPDWOZOsP8d1bC+O/VHbaMJmddkHpaxIn wYJQ== X-Gm-Message-State: APjAAAXJPIbwDMYyhP8MwiqNlWHXl3rMemitimUiRCxRZOX2v+r1oy2S l9o9m2ai42/W7P1UK2esWeG5Ew== X-Google-Smtp-Source: APXvYqw7K5Maxh/pnX0zwUikrxHIgY8XjV7caaqSdvIv/UjHhOEBRT/kgetlQI+L9p4OMfLFAJNYMQ== X-Received: by 2002:a5b:903:: with SMTP id a3mr16031873ybq.487.1556549497527; Mon, 29 Apr 2019 07:51:37 -0700 (PDT) Received: from KrisYogaC930 ([12.189.233.130]) by smtp.gmail.com with ESMTPSA id r11sm8806468ywb.81.2019.04.29.07.51.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Apr 2019 07:51:36 -0700 (PDT) From: To: "'Matthias Apitz'" , "'Emmanuel Vadot'" Cc: "'FreeBSD Stable'" , , , "'FreeBSD Current'" , , References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> <20190429155522.81dcc291a6b252884219de1e@bidouilliste.com> <20190429162847.a32262b6073f5523bdbe0ed4@bidouilliste.com> <20190429145028.GA3124@c720-r342378> In-Reply-To: <20190429145028.GA3124@c720-r342378> Subject: RE: CFT: FreeBSD Package Base Date: Mon, 29 Apr 2019 10:51:35 -0400 Message-ID: <053501d4fe9b$0b8c0a20$22a41e60$@ixsystems.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook 16.0 Thread-Index: AQFBRupbev+o6AHqgfoJLZwOgBKxNQFhdrr5Ad4+GxoCGFDnowGvv4BvAf5pkjQA0yLFdKcr3YuA Content-Language: en-us X-Rspamd-Queue-Id: 4DB2686773 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=GadSAgMR; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::b30 as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-5.33 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-2.82)[ip: (-8.65), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; FROM_NO_DN(0.00)[]; RCPT_COUNT_SEVEN(0.00)[8]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; RCVD_IN_DNSWL_NONE(0.00)[0.3.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:51:39 -0000 > -----Original Message----- > From: Matthias Apitz > Sent: Monday, April 29, 2019 10:50 AM > To: Emmanuel Vadot > Cc: Kris Moore ; FreeBSD Stable stable@freebsd.org>; freebsd-pkg@freebsd.org; freebsd- > hackers@freebsd.org; FreeBSD Current ; > freebsd-pkgbase@freebsd.org; freebsd-ports@freebsd.org > Subject: Re: CFT: FreeBSD Package Base >=20 >=20 > Why this thread has to go to all these lists? I receive any mail 5 = times! >=20 > Matthias Fair point. I'll restrict my replies to the -pkgbase list from here on = out, suggest others do the same. Sorry about the noise =F0=9F=98=8A > -- > Matthias Apitz, =E2=9C=89 guru@unixarea.de, http://www.unixarea.de/ = +49-176- > 38902045 Public GnuPG key: http://www.unixarea.de/key.pub N =E2=82=AC = I N zur EU! > "Gegen das EU-Europa der Banken, Konzerne und Kriegstreiber. > F=C3=BCr ein soziales und friedliches Europa der V=C3=B6lker." DKP From owner-freebsd-stable@freebsd.org Mon Apr 29 16:19:32 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 92E291595619 for ; Mon, 29 Apr 2019 16:19:32 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id A0EE68AB3B for ; Mon, 29 Apr 2019 16:19:28 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from isux.com (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQQ005JVB7HZY40@hades.sorbs.net> for freebsd-stable@freebsd.org; Mon, 29 Apr 2019 08:33:18 -0700 (PDT) To: freebsd-stable From: Michelle Sullivan Subject: ZFS... Message-id: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> Date: Tue, 30 Apr 2019 01:19:20 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 X-Rspamd-Queue-Id: A0EE68AB3B X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.58 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sorbs.net]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-1.41)[ip: (-3.98), ipnet: 72.12.192.0/19(-1.71), asn: 11114(-1.28), country: US(-0.06)]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[battlestar.sorbs.net,anaconda.sorbs.net,ninja.sorbs.net,catapilla.sorbs.net,scorpion.sorbs.net,desperado.sorbs.net]; NEURAL_HAM_SHORT(-0.91)[-0.913,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 16:19:32 -0000 I know I'm not going to be popular for this, but I'll just drop it here anyhow. http://www.michellesullivan.org/blog/1726 Perhaps one should reconsider either: 1. Looking at tools that may be able to recover corrupt ZFS metadata, or 2. Defaulting to non ZFS filesystems on install. -- Michelle Sullivan http://www.mhix.org/ From owner-freebsd-stable@freebsd.org Mon Apr 29 14:41:31 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3F3F1591C40; Mon, 29 Apr 2019 14:41:30 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1872C85C1E; Mon, 29 Apr 2019 14:41:29 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x3TEfNiC072752; Mon, 29 Apr 2019 07:41:23 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x3TEfMid072751; Mon, 29 Apr 2019 07:41:22 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201904291441.x3TEfMid072751@gndrsh.dnsmgr.net> Subject: Re: CFT: FreeBSD Package Base In-Reply-To: To: Kris Moore Date: Mon, 29 Apr 2019 07:41:22 -0700 (PDT) CC: "Rodney W. Grimes" , =?UTF-8?Q?Goran_Meki=C4=87?= , Emmanuel Vadot , FreeBSD Stable , FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 1872C85C1E X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [2.27 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.78)[-0.776,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.43)[0.430,0]; IP_SCORE(0.04)[ip: (0.15), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.05), country: US(-0.06)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: gndrsh.dnsmgr.net]; NEURAL_SPAM_LONG(0.19)[0.186,0]; RCPT_COUNT_SEVEN(0.00)[10]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-Mailman-Approved-At: Mon, 29 Apr 2019 16:20:18 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 14:41:31 -0000 > On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > > > > > > Correct, this is ZFS only. And it's something we're using specific to > > FreeNAS / TrueOS, which is why I didn't originally mention it as apart of > > our CFT. > > > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", > > calling this FreeBSD pkg base when it is not was wrong, > > and miss leading. > > > > Sorry, I disagree. Which is fine. > This pkg base is independent of the ZFS tool we're using > to wrangle boot-environments. Hence why it wasn't mentioned in the CFT. > These base packages work the same as existing in-tree pkg base on UFS, no > difference. If anything are probably safer due to being able to update all > of userland in single extract operation, so you don't have out of order > extraction of libc or some such. You missed the major string change and focused on the edge, No comment on calling iXsystems :stuff: FreeBSD instead of FreeNAS/TrueOS? That was the major point of my statement, your miss leading the user community, you yourself said this would never be imported into FreeBSD base, so I see no reason that it should be called "FreeBSD package Base", as it is not, that is a different project. > > > For UFS, there will need to be additional care taken when doing updates. > > > > > > -- > > > Kris Moore > > > Vice President of Engineering > > > iXsystems, Inc > > > Ph: (408) 943-4100 > > > Ph: (408) 943-4101 > > > The Groundbreaking TrueNAS M-Series - > > > Enterprise Storage & Servers Driven By Open Source > > > > > > -----Original Message----- > > > From: Goran Meki? > > > Sent: Monday, April 29, 2019 9:43 AM > > > To: Kris Moore > > > Cc: Emmanuel Vadot ; FreeBSD Stable < > > freebsd-stable@freebsd.org>; FreeBSD Current ; > > freebsd-pkgbase@freebsd.org; freebsd-pkg@freebsd.org; > > freebsd-hackers@freebsd.org; freebsd-ports@freebsd.org > > > Subject: Re: CFT: FreeBSD Package Base > > > > > > On Mon, Apr 29, 2019 at 09:25:05AM -0400, Kris Moore wrote: > > > > We've written our own tool "sysutils/sysup" in GO which handles this. > > > > It performs updates using Boot-Environments to ensure that > > > > kernel/world are updated at same time. > > > > > > If I'm right, UFS doesn't support boot environments, so how would it > > work for UFS based installs? > > > > > > I personally feel GO is a bit ackward choice of language for something > > that practically should be part of base. At least I would expect OS > > update/upgrade not to require any external package. > > > > > > Regards, > > > meka > > > > > > _______________________________________________ > > > freebsd-current@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-current > > > To unsubscribe, send any mail to " > > freebsd-current-unsubscribe@freebsd.org" > > > > > > > > > > -- > > Rod Grimes > > rgrimes@freebsd.org > > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-stable@freebsd.org Mon Apr 29 17:06:35 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EB3D1596BC8 for ; Mon, 29 Apr 2019 17:06:35 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf1-f52.google.com (mail-lf1-f52.google.com [209.85.167.52]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 431FA8CAF3 for ; Mon, 29 Apr 2019 17:06:34 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf1-f52.google.com with SMTP id j20so8615934lfh.2 for ; Mon, 29 Apr 2019 10:06:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fX4ShCtfUBTozeat+Iyqg9LM7KC9Fh5VIaKj4Ynb20s=; b=BBTs/5pG5jBGkW3Tt8wEst5BsmzPRJSmF4mV4Tk7MH0MwNb2DFBQbl7ZcVQi8VfWHE eSCMtGe3NCHcBwcotTFqo5G5oq8wgwOw4x3RrQMpxxvxhGw8AK5axwLiQTcT4aJHaF9O jhN/SVu1FeLoXJYFRrmXd51MZVdcvnubxW73rqDrE+vWo8OtOSZLCBK9mHzmSTcaNXcR SymfSN9NQialLxWCXmUEE0jSBAa/J7dTSN//6QYW42TtMF+pfEjTFyASzS/8BAHz+AG6 1fzggMVPzbQKODjQpuiFXz71XoGxKO3LeBDo3RD51Srjh5ViPSD0PtwoPBrknV0Higrc MznQ== X-Gm-Message-State: APjAAAUyybj7bC9S2nrXyMTmLZITMfbWfbWqpZchPWWPiz3Aw8hqdUQV iHn73CNE3J6zKK8HzltsmtiCJnkIB8/8HKfWjJzq085d X-Google-Smtp-Source: APXvYqzKZM2rKZnBN/E10mDqldvy3tfx9bFlcTZ9LlCzlBfdwTRiD0S/sMo3XiKUEsUI8Y8/3crOIfiYJKCkaWFJEkA= X-Received: by 2002:a19:6d1b:: with SMTP id i27mr35577535lfc.156.1556557586863; Mon, 29 Apr 2019 10:06:26 -0700 (PDT) MIME-Version: 1.0 References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> In-Reply-To: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> From: Alan Somers Date: Mon, 29 Apr 2019 11:06:15 -0600 Message-ID: Subject: Re: ZFS... To: Michelle Sullivan Cc: freebsd-stable Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 431FA8CAF3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.52 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-3.79 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_SHORT(-0.88)[-0.883,0]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[52.167.85.209.list.dnswl.org : 127.0.5.0]; SUBJ_ALL_CAPS(0.45)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; IP_SCORE(-1.35)[ip: (-0.60), ipnet: 209.85.128.0/17(-3.86), asn: 15169(-2.24), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 17:06:35 -0000 On Mon, Apr 29, 2019 at 10:23 AM Michelle Sullivan wrote: > > I know I'm not going to be popular for this, but I'll just drop it here > anyhow. > > http://www.michellesullivan.org/blog/1726 > > Perhaps one should reconsider either: > > 1. Looking at tools that may be able to recover corrupt ZFS metadata, or > 2. Defaulting to non ZFS filesystems on install. > > -- > Michelle Sullivan > http://www.mhix.org/ Wow, losing multiple TB sucks for anybody. I'm sorry for your loss. But I want to respond to a few points from the blog post. 1) When ZFS says that "the data is always correct and there's no need for fsck", they mean metadata as well as data. The spacemap is protected in exactly the same way as all other data and metadata. (to be pedantically correct, the labels and uberblocks are protected in a different way, but still protected). The only way to get metadata corruption is due a disk failure (3-disk failure when using RAIDZ2), or due to a software bug. Sadly, those do happen, and they're devilishly tricky to track down. The difference between ZFS and older filesystems is that older filesystems experience corruption during power loss _by_design_, not merely due to software bugs. A perfectly functioning UFS implementation will experience corruption during power loss, and that's why it needs to be fscked. It's not just theoretical, either. I use UFS on my development VMs, and they frequently experience corruption after a panic (which happens all the time because I'm working on kernel code). 2) Backups are essential with any filesystem, not just ZFS. After all, no amount of RAID will protect you from an accidental "rm -rf /". 3) ZFS hotspares can be swapped in automatically, though they don't be default. It sounds like you already figured out how to assign a spare to the pool. To use it automatically, you must set the "autoreplace" pool property and enable zfsd. The latter can be done with "sysrc zfsd_enable="YES"". 4) It sounds like you're having a lot of power trouble. Have you tried sysutils/apcupsd from ports? It's fairly handy. It can talk to a wide range of UPSes, and can be configured to do stuff like send you an email on power loss, and power down the server if the battery gets too low. Better luck next time, -Alan From owner-freebsd-stable@freebsd.org Mon Apr 29 17:13:47 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 948DE1596F35 for ; Mon, 29 Apr 2019 17:13:47 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 200298D058 for ; Mon, 29 Apr 2019 17:13:47 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from pi by home.opsec.eu with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hL9qd-0002Ed-UV; Mon, 29 Apr 2019 19:13:47 +0200 Date: Mon, 29 Apr 2019 19:13:47 +0200 From: Kurt Jaeger To: Michelle Sullivan Cc: freebsd-stable Subject: Re: ZFS... Message-ID: <20190429171347.GV72200@home.opsec.eu> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 17:13:47 -0000 Hi! > I know I'm not going to be popular for this, but I'll just drop it here > anyhow. > > http://www.michellesullivan.org/blog/1726 With all due respect, I think if that filesystem/server you describe has not kept with all those mishaps, I think it's not perfect, but nothing is. > Perhaps one should reconsider either: > > 2. Defaulting to non ZFS filesystems on install. I had more cases of UFS being toast than ZFS until now. > 1. Looking at tools that may be able to recover corrupt ZFS metadata, or Here I agree! Making tools available to dig around zombie zpools, which is icky in itself, would be helpful! -- pi@opsec.eu +49 171 3101372 One year to go ! From owner-freebsd-stable@freebsd.org Mon Apr 29 19:31:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 198A3159AB6B; Mon, 29 Apr 2019 19:31:44 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A9B46E3AA; Mon, 29 Apr 2019 19:31:41 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id LBzshjHmcGusjLBzuhmeZ0; Mon, 29 Apr 2019 13:31:33 -0600 X-Authority-Analysis: v=2.3 cv=fOdHIqSe c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=kj9zAlcOel0A:10 a=oexKYjalfGEA:10 a=iKhvJSA4AAAA:8 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=zoJ4fAklMHesfqNpwjwA:9 a=CjuIK1q_8ugA:10 a=odh9cflL3HIXMm4fY7Wr:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy8 [10.2.2.6]) by spqr.komquats.com (Postfix) with ESMTPS id D3E2B3105; Mon, 29 Apr 2019 12:31:26 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x3TJV7Bg079805; Mon, 29 Apr 2019 12:31:07 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x3TJV73d079802; Mon, 29 Apr 2019 12:31:07 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201904291931.x3TJV73d079802@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: "Rodney W. Grimes" cc: Kris Moore , FreeBSD Stable , freebsd-ports@freebsd.org, =?UTF-8?Q?Goran_Meki=C4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, Emmanuel Vadot Subject: Re: CFT: FreeBSD Package Base In-Reply-To: Message from "Rodney W. Grimes" of "Mon, 29 Apr 2019 07:41:22 -0700." <201904291441.x3TEfMid072751@gndrsh.dnsmgr.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 29 Apr 2019 12:31:07 -0700 X-CMAE-Envelope: MS4wfNzDWI+zWN/ORISOq2YqftqFPdS4edEpOvpU680vr/0sjvuXehwqqYw0ygP6+3T7Viuz3PQ57eRy7ntso0zPA0JjpvAP5KpCIGDydVICtnz/bDuBZkz4 VbYnvaZwzvtOLW8kh/E7iOCeqvOdfrhDDuiTa+Y3tiipLD5wFQ7WpOMBu8d6YrXpgO+nSY1ixyrUmJWDQMUI4uT6jZd+oryQwP57gTjGBm4H7h/9mgQjhYAY SQ1w9uhrKaTp77U96HuJm588nPRfFMJaHYZx/ujMsDcLJ2shhxEYacWnLVCukybqG59i/LE/HnqNK70ShU64HBWLAE0+UKo/j0aEmoKiYmImmhTpfe1quzvx eZfp65MULKH35mmEeRiGdaCZKvJl34o+2v5R9iwIpNcP+v2lEFdlES8D3hdvr1zkghhJAd92ZKzqvEq+P0/RrZed+EmdU0ZCe8iDz1f+E+g4/cZQBThOnlBZ elX6kJv4tFCEr9Dbre3Bz9SgHVyRWEGx7ERsHS2PMdNGgjxvUJk20eu6jc8= X-Rspamd-Queue-Id: 9A9B46E3AA X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-3.49 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; HAS_XAW(0.00)[]; MX_GOOD(-0.01)[spqr.komquats.com]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; RCPT_COUNT_SEVEN(0.00)[10]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.zen.spamhaus.org : 127.0.0.11]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_LOW(-0.10)[139.136.59.64.list.dnswl.org : 127.0.5.1]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_COUNT_FIVE(0.00)[5]; REPLYTO_EQ_FROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-2.29)[ip: (-5.71), ipnet: 64.59.128.0/20(-3.21), asn: 6327(-2.45), country: CA(-0.09)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_NA(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-Mailman-Approved-At: Mon, 29 Apr 2019 19:54:55 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 19:31:44 -0000 In message <201904291441.x3TEfMid072751@gndrsh.dnsmgr.net>, "Rodney W. Grimes" writes: > > On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < > > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > > > > > > > > > Correct, this is ZFS only. And it's something we're using specific to > > > FreeNAS / TrueOS, which is why I didn't originally mention it as apart of > > > our CFT. > > > > > > Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", > > > calling this FreeBSD pkg base when it is not was wrong, > > > and miss leading. > > > > > > > Sorry, I disagree. > Which is fine. > > > This pkg base is independent of the ZFS tool we're using > > to wrangle boot-environments. Hence why it wasn't mentioned in the CFT. > > These base packages work the same as existing in-tree pkg base on UFS, no > > difference. If anything are probably safer due to being able to update all > > of userland in single extract operation, so you don't have out of order > > extraction of libc or some such. > > You missed the major string change and focused on the edge, > No comment on calling iXsystems :stuff: FreeBSD instead of FreeNAS/TrueOS? > > That was the major point of my statement, your miss leading the user > community, you yourself said this would never be imported into FreeBSD > base, so I see no reason that it should be called "FreeBSD package Base", > as it is not, that is a different project. Taking the last comment on this thread to ask a question and maybe refocus a little. The discussion about granularity begs the question, why pkgbase in the first place? My impression was that it allowed people to select which components they wanted to either create a lean installation or mix and match base packages and ports (possibly with flavours to install in /usr rather than $LOCALBASE) such that maybe person A wanted a stock install while person B wanted to replace, picking a random example, BSD tar with GNU tar. Isn't that the real advantage of pkgbase? If OTOH it's binary updates V 2.0, what's the point? I'm a little rhetorical here but you get my point. If I want ipfw instead pf or ipfilter instead of the others I should have the freedom. Similarly if I want vim instead of vi I should have the choice to install vim as /usr/bin/vi. Otherwise all the effort to replace binary updates makes no sense. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-stable@freebsd.org Mon Apr 29 20:11:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DAB8C159C3BD; Mon, 29 Apr 2019 20:11:27 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D9E771048; Mon, 29 Apr 2019 20:11:26 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1556568683; bh=/CohqdqlEu9l7zyZzFBroyuRIPeYMCAEkqd5kWSNrHQ=; h=X-UI-Sender-Class:From:To:Subject:Date; b=BauOpAtNH5+UmAjdWGbA55JFXURk9F/XRyIruuKlh0lgZc6Lzf1f+YalBWxty+vRv 45KVaqpqphjsSe/SKOekQZNC7uMGT8OoyYx2n+b0Xl16FBspU98h65f9P6Jxi0LF2f k0dlbMagAabPS+OgjcxUGjgtz511RlZSeBY7G65U= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [89.244.165.76] ([89.244.165.76]) by web-mail.gmx.net (3c-app-gmx-bs47.server.lan [172.19.170.99]) (via HTTP); Mon, 29 Apr 2019 21:58:25 +0200 Message-ID: From: "Martin Jakob" To: freebsd-pkgbase@freebsd.org, freebsd-stable@freebsd.org Subject: Creating jails from packagebase Date: Mon, 29 Apr 2019 21:58:25 +0200 Importance: normal Sensitivity: Normal X-Priority: 3 X-Provags-ID: V03:K1:BvPKLBcTXcq8IIif+hdB8/qCX8ywM3ewIddm/CE6fz6UhE8chufnS5+WzXgu8z0KCkP0m CgYM36kzeAf1qvFFOi2W4ePUomnr4lIz5U+NFI0PxQOA4Z9viDx0aJcNnYAgHP0oGKi1lDX0aV6O N2oSOIikdG/QtiuZl9x+leyD7s2CA2uqP+oN6t5zpQ4zGy0lkY08L2o4gun36UsDjpU/PMxSmeDD +mZGu9J2svMceX5q5Wx0S8VWpLdBRKEEDNo/xyhiML4viqXvisFfF2gnJ5h+kaYcgBGgeoAAf+yh rM= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:IqJibukkDaM=:4HZwr2TVn6Lk6o0c6GOW+/ S5zju2Vl8DfSGF8pBa0ApeUq6IFqAMb1EM7Ajwq0x412O8aNxfRrZ0qRuxAIstw1FnKH3rcP4 trbEPXhg3zrnIuXbU01UvKDauuDrEjIg8zlVH24Bd7BYHlFEXMwCkVA9FNij/KmbmhPHR0C82 3lKWGtORztXkSUbfy6NGoBdHMAMGKyVs9do9PVPAThUvPXGrEC3w9h5OZ5PyAJnf76hqQ6Kki l/oXdgYNGMFzdZpeRBEzqWZccaJMJoOSLlwRTUlG8OqBmJyAOzYRbUWvDtuF6sX/Zyeo/2ggY lsfC36qMqKVCOOEqJKZmPFylfIpeG3ZcGy9e9qh6bWhzFdHwvLB+dO4O4cW/kr1+RUK+urMJ6 2zNegX+gINXv0YSq2wkDHHJNnmQOIEH77ZRfuhFsBAHh0PLU7uu4vNWfUxZE4J0ao4D+y9z3m elGAwrm3+Rr8lYSa7Kz+pM8TxWezsRxJwDlQD8u/SLPSvq7TL+3U9H+i1ivbKvSWvI6C8sRIo +ibMqjrOPu08G/wJEP+FAFXgHNQxIgJ0UIymPwDL97J9tPrnAjwb1qQckDl0hn93MPimFLt6K Q+uCKqPHf38NXvUlv+6Dn8pOXHEM0JCtUbmXHYstqBA0sE9zZKQl+KrQ== X-Rspamd-Queue-Id: 9D9E771048 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=BauOpAtN; spf=pass (mx1.freebsd.org: domain of mj-mailinglist@gmx.de designates 212.227.15.19 as permitted sender) smtp.mailfrom=mj-mailinglist@gmx.de X-Spamd-Result: default: False [-3.13 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RECEIVED_SPAMHAUS_PBL(0.00)[76.165.244.89.zen.spamhaus.org : 127.0.0.10]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.227.15.0/25]; FREEMAIL_FROM(0.00)[gmx.de]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[gmx.de]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[gmx.net:+]; RCPT_COUNT_TWO(0.00)[2]; HAS_X_PRIO_THREE(0.00)[3]; MX_GOOD(-0.01)[mx00.emig.gmx.net,mx01.emig.gmx.net]; MIME_HTML_ONLY(0.20)[]; IP_SCORE(-0.99)[ip: (-5.39), ipnet: 212.227.0.0/16(-1.40), asn: 8560(1.87), country: DE(-0.01)]; NEURAL_HAM_SHORT(-0.34)[-0.338,0]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_ENVFROM(0.00)[gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; RCVD_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_LOW(-0.10)[19.15.227.212.list.dnswl.org : 127.0.3.1] MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 20:11:28 -0000 From owner-freebsd-stable@freebsd.org Mon Apr 29 20:12:48 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4BFE159C4C7 for ; Mon, 29 Apr 2019 20:12:48 +0000 (UTC) (envelope-from jmaloney@ixsystems.com) Received: from mail-yw1-xc44.google.com (mail-yw1-xc44.google.com [IPv6:2607:f8b0:4864:20::c44]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FB6171341 for ; Mon, 29 Apr 2019 20:12:47 +0000 (UTC) (envelope-from jmaloney@ixsystems.com) Received: by mail-yw1-xc44.google.com with SMTP id i66so4245868ywe.5 for ; Mon, 29 Apr 2019 13:12:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6MjsstOZD7psHVBmtlDM1lEe80ANjy/dSzbHny/7BdI=; b=k9foXJI/JSbZgDBP4fCHwVrMBUYq2kvPzTJYVw3Nrjse5oywo8ebK8buMKJIkz5OBm Xs9LGrM+EbX3Sy7b2OJbDT+H+2M4iR1OYRp3j5OZvhAcq6f9WuxrqqPPAY2gKHwsmhGR vsq7NB+JUsqDBH/cH+XPUg45yNdtyRUyNugAn2eDQP2BswakIiaZQiXRZjieIQP418hq +grTgUQzoFUm1hgN9wOhBtw7djqss1NloSPbjYsed7TnFFkWzCwphcMzL9ei2xzZa/Y+ GZoac09S/itq8iVI1JXBqrwfvf9Fp0jd3Bhj1GGl041sLcBaCka4c894oN/cu7TysJuC 5+1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=6MjsstOZD7psHVBmtlDM1lEe80ANjy/dSzbHny/7BdI=; b=HAz8d0WUZrjtELQd//sTOOdDgDIEwDhiUJV9Dg5GKFFqwIZ8XSOiqyTQX1YXNd+c2r Xe7oNu1+cZXbrU0PxYn+xSuNNUzgP3zAu5OxDuCCVvoEbpSSkobppXiz/AGqTOgguQRk 7UhiW+fmdljmw+RGZdvRr26Wcai/HEYKhRu4Sk6rdZKgybXrXgrCxwokDUPG9oGMEEDf 8e6ZFukXHXuphvpNgtIxbFhJubmdvpP0nBWuvHOt7NuhxltPdwYc5cMYbJVvNO7ryBO4 TBbHy3AmIeluiQGpZpfpQdlQKzjvLblYP9qF7FqRpujOreB5D6oBfAerdzPo7He7iNpc wYMA== X-Gm-Message-State: APjAAAV1b2FvMqQZdenQJW80RD1PFJLxvj+B5Bo6Jhrgsw1dvSpfb71y KW1WFcjVRNp2cSVDNshrjJ6TlhYWLoJQng== X-Google-Smtp-Source: APXvYqxXTtel0TlWm5Rw3TF3NV29b2J9wd5tI5zBCRkqB2n1zoArV3CzMNIGd6dXvgKhLNXQhc7qWQ== X-Received: by 2002:a81:6608:: with SMTP id a8mr22566339ywc.229.1556568766490; Mon, 29 Apr 2019 13:12:46 -0700 (PDT) Received: from [10.238.1.100] ([12.189.233.131]) by smtp.gmail.com with ESMTPSA id j83sm10920165ywj.93.2019.04.29.13.12.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Apr 2019 13:12:45 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: CFT: FreeBSD Package Base From: Joe Maloney In-Reply-To: <201904291931.x3TJV73d079802@slippy.cwsent.com> Date: Mon, 29 Apr 2019 16:12:43 -0400 Cc: "Rodney W. Grimes" , Kris Moore , FreeBSD Stable , freebsd-ports@freebsd.org, =?utf-8?Q?Goran_Meki=C4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, Emmanuel Vadot Content-Transfer-Encoding: quoted-printable Message-Id: <6EB65F49-47CD-48B2-8C58-BF387D8C7A58@ixsystems.com> References: <201904291931.x3TJV73d079802@slippy.cwsent.com> To: Cy Schubert X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 1FB6171341 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=k9foXJI/; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of jmaloney@ixsystems.com designates 2607:f8b0:4864:20::c44 as permitted sender) smtp.mailfrom=jmaloney@ixsystems.com X-Spamd-Result: default: False [-2.82 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; RCPT_COUNT_SEVEN(0.00)[11]; NEURAL_HAM_SHORT(-0.93)[-0.933,0]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-0.88)[ip: (1.06), ipnet: 2607:f8b0::/32(-3.16), asn: 15169(-2.24), country: US(-0.06)]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[4.4.c.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; SUSPICIOUS_RECIPS(1.50)[] X-Mailman-Approved-At: Mon, 29 Apr 2019 20:40:04 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 20:12:49 -0000 With CFT version you chose to build, and package individual components = such as sendmail with a port option. That does entirely solve the = problem of being able to reinstall sendmail after the fact without a = rebuild of the userland (base) port but perhaps base flavors could solve = that problem assuming flavors could extend beyond python. Joe Maloney Quality Engineering Manager / iXsystems Enterprise Storage & Servers Driven By Open Source > On Apr 29, 2019, at 3:31 PM, Cy Schubert = wrote: >=20 > In message <201904291441.x3TEfMid072751@gndrsh.dnsmgr.net>, "Rodney W.=20= > Grimes" > writes: >>> On Mon, Apr 29, 2019 at 10:09 AM Rodney W. Grimes < >>> freebsd-rwg@gndrsh.dnsmgr.net> wrote: >>>=20 >>>>>=20 >>>>> Correct, this is ZFS only. And it's something we're using specific = to >>>> FreeNAS / TrueOS, which is why I didn't originally mention it as = apart of >>>> our CFT. >>>>=20 >>>> Then please it is "CFT: FreeNAS/TrueOS pkg base, ZFS only", >>>> calling this FreeBSD pkg base when it is not was wrong, >>>> and miss leading. >>>>=20 >>>=20 >>> Sorry, I disagree. >> Which is fine. >>=20 >>> This pkg base is independent of the ZFS tool we're using >>> to wrangle boot-environments. Hence why it wasn't mentioned in the = CFT. >>> These base packages work the same as existing in-tree pkg base on = UFS, no >>> difference. If anything are probably safer due to being able to = update all >>> of userland in single extract operation, so you don't have out of = order >>> extraction of libc or some such. >>=20 >> You missed the major string change and focused on the edge, >> No comment on calling iXsystems :stuff: FreeBSD instead of = FreeNAS/TrueOS? >>=20 >> That was the major point of my statement, your miss leading the user >> community, you yourself said this would never be imported into = FreeBSD >> base, so I see no reason that it should be called "FreeBSD package = Base", >> as it is not, that is a different project. >=20 > Taking the last comment on this thread to ask a question and maybe=20 > refocus a little. >=20 > The discussion about granularity begs the question, why pkgbase in the=20= > first place? My impression was that it allowed people to select which=20= > components they wanted to either create a lean installation or mix and=20= > match base packages and ports (possibly with flavours to install in=20 > /usr rather than $LOCALBASE) such that maybe person A wanted a stock=20= > install while person B wanted to replace, picking a random example, = BSD=20 > tar with GNU tar. Isn't that the real advantage of pkgbase? >=20 > If OTOH it's binary updates V 2.0, what's the point? I'm a little=20 > rhetorical here but you get my point. If I want ipfw instead pf or=20 > ipfilter instead of the others I should have the freedom. Similarly if=20= > I want vim instead of vi I should have the choice to install vim as=20 > /usr/bin/vi. Otherwise all the effort to replace binary updates makes=20= > no sense. >=20 >=20 > --=20 > Cheers, > Cy Schubert > FreeBSD UNIX: Web: http://www.FreeBSD.org >=20 > The need of the many outweighs the greed of the few. >=20 >=20 > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Tue Apr 30 00:24:27 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91E2815A1C2B for ; Tue, 30 Apr 2019 00:24:27 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 352DA83F81; Tue, 30 Apr 2019 00:24:27 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQR005VI0FVZY40@hades.sorbs.net>; Mon, 29 Apr 2019 17:38:22 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <20190429171347.GV72200@home.opsec.eu> Date: Tue, 30 Apr 2019 10:24:22 +1000 Cc: freebsd-stable Content-transfer-encoding: quoted-printable Message-id: References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <20190429171347.GV72200@home.opsec.eu> To: Kurt Jaeger X-Rspamd-Queue-Id: 352DA83F81 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.93 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.93)[-0.930,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 00:24:27 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 30 Apr 2019, at 03:13, Kurt Jaeger wrote: >=20 > Hi! >=20 >> I know I'm not going to be popular for this, but I'll just drop it here >> anyhow. >>=20 >> http://www.michellesullivan.org/blog/1726 >=20 > With all due respect, I think if that filesystem/server you describe > has not kept with all those mishaps, I think it's not perfect, but > nothing is. The killer was the catalog of errors, where you have resolver in progress, a= nd not one but two power failures...and just to let you know... one of the 6= kva upses caught fire the other no longer recognizes AC input... so it was n= ot your normal event. I had a good run with 8 years of running ZFS on this s= erver. >=20 >> Perhaps one should reconsider either: >>=20 >> 2. Defaulting to non ZFS filesystems on install. >=20 > I had more cases of UFS being toast than ZFS until now. I=E2=80=99ve toasted many, however I=E2=80=99ve always been able to get majo= rity(if not all) of the data. =20 >=20 >> 1. Looking at tools that may be able to recover corrupt ZFS metadata, or >=20 > Here I agree! Making tools available to dig around zombie zpools, > which is icky in itself, would be helpful! The one tool that I could think would be useful that is denied =E2=80=9Cbeca= use the data on disk is always right=E2=80=9D is not a fact for zfs, but a z= fs send with -AAA (like zdb) or a =E2=80=9Czfs walk=E2=80=9D tool that works= similar to zfs send but where you can tell it to ignore the checksum errors= (particularly in the structures of zfs rather than on the data itself) so y= ou can send what=E2=80=99s left of your data to another box either in part o= r fully. Particularly as in my case all the tools tell me all the data is th= ere and intact and it=E2=80=99s just the metadata that can=E2=80=99t be reco= vered/repaired. Regards, Michelle= From owner-freebsd-stable@freebsd.org Tue Apr 30 00:41:25 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1A0C15A23DC for ; Tue, 30 Apr 2019 00:41:24 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id ABF8D84AA5; Tue, 30 Apr 2019 00:41:22 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQR0020C181Y400@hades.sorbs.net>; Mon, 29 Apr 2019 17:55:17 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Tue, 30 Apr 2019 10:41:17 +1000 Cc: freebsd-stable Content-transfer-encoding: quoted-printable Message-id: <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> To: Alan Somers X-Rspamd-Queue-Id: ABF8D84AA5 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.92 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-1.30)[ip: (-3.65), ipnet: 72.12.192.0/19(-1.60), asn: 11114(-1.20), country: US(-0.06)]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.85)[-0.854,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 00:41:25 -0000 Comments inline.. Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 30 Apr 2019, at 03:06, Alan Somers wrote: >=20 >> On Mon, Apr 29, 2019 at 10:23 AM Michelle Sullivan w= rote: >>=20 >> I know I'm not going to be popular for this, but I'll just drop it here >> anyhow. >>=20 >> http://www.michellesullivan.org/blog/1726 >>=20 >> Perhaps one should reconsider either: >>=20 >> 1. Looking at tools that may be able to recover corrupt ZFS metadata, or >> 2. Defaulting to non ZFS filesystems on install. >>=20 >> -- >> Michelle Sullivan >> http://www.mhix.org/ >=20 > Wow, losing multiple TB sucks for anybody. I'm sorry for your loss. > But I want to respond to a few points from the blog post. >=20 > 1) When ZFS says that "the data is always correct and there's no need > for fsck", they mean metadata as well as data. The spacemap is > protected in exactly the same way as all other data and metadata. (to > be pedantically correct, the labels and uberblocks are protected in a > different way, but still protected). The only way to get metadata > corruption is due a disk failure (3-disk failure when using RAIDZ2), > or due to a software bug. Sadly, those do happen, and they're > devilishly tricky to track down. The difference between ZFS and older > filesystems is that older filesystems experience corruption during > power loss _by_design_, not merely due to software bugs. A perfectly > functioning UFS implementation will experience corruption during power > loss, and that's why it needs to be fscked. It's not just > theoretical, either. I use UFS on my development VMs, and they > frequently experience corruption after a panic (which happens all the > time because I'm working on kernel code). I know, which is why I have ZVOLs with UFS filesystems in them for the devel= opment VMs... in a perfect world the power would have been all good, the up= ses would not be damaged and the generator would not run out of fuel because= of extended outage... in fact if it was a perfect world I wouldn=E2=80=99t= have my own mini dc at home. >=20 > 2) Backups are essential with any filesystem, not just ZFS. After > all, no amount of RAID will protect you from an accidental "rm -rf /". You only do it once... I did it back in 1995... haven=E2=80=99t ever done i= t again. >=20 > 3) ZFS hotspares can be swapped in automatically, though they don't be > default. It sounds like you already figured out how to assign a spare > to the pool. To use it automatically, you must set the "autoreplace" > pool property and enable zfsd. The latter can be done with "sysrc > zfsd_enable=3D"YES"". The system was originally built on 9.0, and got upgraded through out the yea= rs... zfsd was not available back then. So get your point, but maybe you di= dn=E2=80=99t realize this blog was a history of 8+ years? >=20 > 4) It sounds like you're having a lot of power trouble. Have you > tried sysutils/apcupsd from ports? I did... Malta was notorious for it. Hence 6kva upses in the bottom of each= rack (4 racks), cross connected with the rack next to it and a backup gener= ator... Australia on the otherhand is a lot more stable (at least where I a= m)... 2 power issues in 2 years... both within 10 hours... one was a transf= ormer, the other when some idiot took out a power pole (and I mean actually t= ook it out, it was literally snapped in half... how they got out of the car a= nd did a runner before the police or Ambos got there I=E2=80=99ll never know= .) > It's fairly handy. It can talk to > a wide range of UPSes, and can be configured to do stuff like send you > an email on power loss, and power down the server if the battery gets > too low. >=20 They could help this... all 4 upses are toast now. One caught fire, one no l= onger detects AC input, the other two I=E2=80=99m not even trying after the f= irst catching fire... the lot are being replaced on insurance. It=E2=80=99s a catalog of errors that most wouldn=E2=80=99t normally experie= nce. However it does show (to me) that ZFS on everything is a really bad id= ea... particularly for home users where there is unknown hardware and you kn= ow they will mistreat it... they certainly won=E2=80=99t have ECC RAM in lap= tops etc... unknown caching facilities etc.. it=E2=80=99s a recipe for losin= g the root drive... Regards, Michelle= From owner-freebsd-stable@freebsd.org Tue Apr 30 06:03:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F6DC1584785 for ; Tue, 30 Apr 2019 06:03:34 +0000 (UTC) (envelope-from stable@umpquanet.com) Received: from g5.umpquanet.com (ns.umpquanet.com [209.216.177.146]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DB8738DB00 for ; Tue, 30 Apr 2019 06:03:27 +0000 (UTC) (envelope-from stable@umpquanet.com) Received: from g5.umpquanet.com (localhost [127.0.0.1]) by g5.umpquanet.com (8.15.2/8.15.2) with ESMTP id x3U5lYJe062043 for ; Mon, 29 Apr 2019 22:47:34 -0700 (PDT) (envelope-from stable@umpquanet.com) Received: (from james@localhost) by g5.umpquanet.com (8.15.2/8.15.2/Submit) id x3U5lYoN062042 for freebsd-stable@freebsd.org; Mon, 29 Apr 2019 22:47:34 -0700 (PDT) (envelope-from stable@umpquanet.com) X-Authentication-Warning: g5.umpquanet.com: james set sender to stable@umpquanet.com using -f Date: Mon, 29 Apr 2019 22:47:34 -0700 From: Jim Long To: freebsd-stable@freebsd.org Subject: Buildworld error: use of undeclared identifier '__fnclex' Message-ID: <20190430054734.GA96884@g5.umpquanet.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: DB8738DB00 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of stable@umpquanet.com designates 209.216.177.146 as permitted sender) smtp.mailfrom=stable@umpquanet.com X-Spamd-Result: default: False [-2.38 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.88)[-0.876,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.99)[-0.993,0]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; MX_GOOD(-0.01)[g5.umpquanet.com]; NEURAL_HAM_SHORT(-0.18)[-0.184,0]; DMARC_NA(0.00)[umpquanet.com]; IP_SCORE(-0.01)[country: US(-0.06)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11233, ipnet:209.216.160.0/19, country:US]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 06:03:34 -0000 I'm upgrading from source from 11.1-STABLE r321596 to 11-STABLE #346941. Nothing in make.conf or src.conf. I believe I am following the procedure at https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html but I keep hitting an error: --snip c++ -O2 -pipe -I/usr/obj/usr/src/tmp/usr/src/lib/clang/libllvm -I/usr/src/= contrib/llvm/lib/Target/AArch64 -I/usr/src/contrib/llvm/lib/Target/ARM -I/u= sr/src/contrib/llvm/lib/Target/Mips -I/usr/src/contrib/llvm/lib/Target/Powe= rPC -I/usr/src/contrib/llvm/lib/Target/Sparc -I/usr/src/contrib/llvm/lib/Ta= rget/X86 -I/usr/src/lib/clang/include -I/usr/src/contrib/llvm/include -DLLV= M_BUILD_GLOBAL_ISEL -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd11.2\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd11.2\" -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/= src/tmp\" -DLLVM_TARGET_ENABLE_AARCH64 -DLLVM_TARGET_ENABLE_ARM -DLLVM_TARG= ET_ENABLE_MIPS -DLLVM_TARGET_ENABLE_POWERPC -DLLVM_TARGET_ENABLE_SPARC -DLL= VM_TARGET_ENABLE_X86 -DLLVM_NATIVE_ASMPARSER=3DLLVMInitializeX86AsmParser -= DLLVM_NATIVE_ASMPRINTER=3DLLVMInitializeX86AsmPrinter -DLLVM_NATIVE_DISASSE= MBLER=3DLLVMInitializeX86Disassembler -DLLVM_NATIVE_TARGET=3DLLVMInitialize= X86Target -DLLVM_NATIVE_TARGETINFO=3DLLVMInitializeX86TargetInfo -DLLVM_NAT= IVE_TARGETMC=3DLLVMInitializeX86TargetMC -ffunction-sections -fdata-section= s -MD -MF.depend.Analysis_CodeMetrics.o -MTAnalysis/CodeMetrics.o -Qunused-= arguments -I/usr/obj/usr/src/tmp/legacy/usr/include -std=3Dc++11 -fno-exce= ptions -fno-rtti -stdlib=3Dlibc++ -Wno-c++11-extensions -c /usr/src/contri= b/llvm/lib/Analysis/CodeMetrics.cpp -o Analysis/CodeMetrics.o c++ -O2 -pipe -I/usr/obj/usr/src/tmp/usr/src/lib/clang/libllvm -I/usr/src/= contrib/llvm/lib/Target/AArch64 -I/usr/src/contrib/llvm/lib/Target/ARM -I/u= sr/src/contrib/llvm/lib/Target/Mips -I/usr/src/contrib/llvm/lib/Target/Powe= rPC -I/usr/src/contrib/llvm/lib/Target/Sparc -I/usr/src/contrib/llvm/lib/Ta= rget/X86 -I/usr/src/lib/clang/include -I/usr/src/contrib/llvm/include -DLLV= M_BUILD_GLOBAL_ISEL -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd11.2\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd11.2\" -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/= src/tmp\" -DLLVM_TARGET_ENABLE_AARCH64 -DLLVM_TARGET_ENABLE_ARM -DLLVM_TARG= ET_ENABLE_MIPS -DLLVM_TARGET_ENABLE_POWERPC -DLLVM_TARGET_ENABLE_SPARC -DLL= VM_TARGET_ENABLE_X86 -DLLVM_NATIVE_ASMPARSER=3DLLVMInitializeX86AsmParser -= DLLVM_NATIVE_ASMPRINTER=3DLLVMInitializeX86AsmPrinter -DLLVM_NATIVE_DISASSE= MBLER=3DLLVMInitializeX86Disassembler -DLLVM_NATIVE_TARGET=3DLLVMInitialize= X86Target -DLLVM_NATIVE_TARGETINFO=3DLLVMInitializeX86TargetInfo -DLLVM_NAT= IVE_TARGETMC=3DLLVMInitializeX86TargetMC -ffunction-sections -fdata-section= s -MD -MF.depend.Analysis_ConstantFolding.o -MTAnalysis/ConstantFolding.o -= Qunused-arguments -I/usr/obj/usr/src/tmp/legacy/usr/include -std=3Dc++11 -= fno-exceptions -fno-rtti -stdlib=3Dlibc++ -Wno-c++11-extensions -c /usr/sr= c/contrib/llvm/lib/Analysis/ConstantFolding.cpp -o Analysis/ConstantFolding= =2Eo In file included from /usr/src/contrib/llvm/lib/Analysis/ConstantFolding.cp= p:49: In file included from /usr/include/c++/v1/cfenv:57: /usr/include/fenv.h:270:3: error: use of undeclared identifier '__fnclex' __fnclex(); ^ 1 error generated. *** Error code 1 Stop. make[4]: stopped in /usr/src/lib/clang/libllvm *** Error code 1 Stop. make[3]: stopped in /usr/src/lib/clang *** Error code 1 Stop. make[2]: stopped in /usr/src *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src --snip How can I get past this to upgrade to 11.2 and then on to 12.0? I'm happy to provide additional info on request. Thank you! Jim My build environment: # uname -a FreeBSD diatom2.smate.wwu.edu 11.1-STABLE FreeBSD 11.1-STABLE #0 r321596: W= ed Jul 26 18:21:47 PDT 2017 root@jimsdesk:/usr/obj/usr/src/sys/GENERIC = amd64 # svn info /usr/src Path: /usr/src Working Copy Root Path: /usr/src URL: svn://svn.freebsd.org/base/stable/11 Relative URL: ^/stable/11 Repository Root: svn://svn.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 346941 Node Kind: directory Schedule: normal Last Changed Author: np Last Changed Rev: 346940 Last Changed Date: 2019-04-29 18:25:02 -0700 (Mon, 29 Apr 2019) # cat /etc/make.conf /etc/src.conf cat: /etc/make.conf: No such file or directory cat: /etc/src.conf: No such file or directory # find /usr/obj /usr/obj # cd /usr/src # make buildworld > /tmp/mbw-20190429-222910.out 2>&1 # tail -28 /tmp/mbw-20190429-222910.out=20 c++ -O2 -pipe -I/usr/obj/usr/src/tmp/usr/src/lib/clang/libllvm -I/usr/src/= contrib/llvm/lib/Target/AArch64 -I/usr/src/contrib/llvm/lib/Target/ARM -I/u= sr/src/contrib/llvm/lib/Target/Mips -I/usr/src/contrib/llvm/lib/Target/Powe= rPC -I/usr/src/contrib/llvm/lib/Target/Sparc -I/usr/src/contrib/llvm/lib/Ta= rget/X86 -I/usr/src/lib/clang/include -I/usr/src/contrib/llvm/include -DLLV= M_BUILD_GLOBAL_ISEL -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd11.2\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd11.2\" -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/= src/tmp\" -DLLVM_TARGET_ENABLE_AARCH64 -DLLVM_TARGET_ENABLE_ARM -DLLVM_TARG= ET_ENABLE_MIPS -DLLVM_TARGET_ENABLE_POWERPC -DLLVM_TARGET_ENABLE_SPARC -DLL= VM_TARGET_ENABLE_X86 -DLLVM_NATIVE_ASMPARSER=3DLLVMInitializeX86AsmParser -= DLLVM_NATIVE_ASMPRINTER=3DLLVMInitializeX86AsmPrinter -DLLVM_NATIVE_DISASSE= MBLER=3DLLVMInitializeX86Disassembler -DLLVM_NATIVE_TARGET=3DLLVMInitialize= X86Target -DLLVM_NATIVE_TARGETINFO=3DLLVMInitializeX86TargetInfo -DLLVM_NAT= IVE_TARGETMC=3DLLVMInitializeX86TargetMC -ffunction-sections -fdata-section= s -MD -MF.depend.Analysis_CodeMetrics.o -MTAnalysis/CodeMetrics.o -Qunused-= arguments -I/usr/obj/usr/src/tmp/legacy/usr/include -std=3Dc++11 -fno-exce= ptions -fno-rtti -stdlib=3Dlibc++ -Wno-c++11-extensions -c /usr/src/contri= b/llvm/lib/Analysis/CodeMetrics.cpp -o Analysis/CodeMetrics.o c++ -O2 -pipe -I/usr/obj/usr/src/tmp/usr/src/lib/clang/libllvm -I/usr/src/= contrib/llvm/lib/Target/AArch64 -I/usr/src/contrib/llvm/lib/Target/ARM -I/u= sr/src/contrib/llvm/lib/Target/Mips -I/usr/src/contrib/llvm/lib/Target/Powe= rPC -I/usr/src/contrib/llvm/lib/Target/Sparc -I/usr/src/contrib/llvm/lib/Ta= rget/X86 -I/usr/src/lib/clang/include -I/usr/src/contrib/llvm/include -DLLV= M_BUILD_GLOBAL_ISEL -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd11.2\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd11.2\" -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/= src/tmp\" -DLLVM_TARGET_ENABLE_AARCH64 -DLLVM_TARGET_ENABLE_ARM -DLLVM_TARG= ET_ENABLE_MIPS -DLLVM_TARGET_ENABLE_POWERPC -DLLVM_TARGET_ENABLE_SPARC -DLL= VM_TARGET_ENABLE_X86 -DLLVM_NATIVE_ASMPARSER=3DLLVMInitializeX86AsmParser -= DLLVM_NATIVE_ASMPRINTER=3DLLVMInitializeX86AsmPrinter -DLLVM_NATIVE_DISASSE= MBLER=3DLLVMInitializeX86Disassembler -DLLVM_NATIVE_TARGET=3DLLVMInitialize= X86Target -DLLVM_NATIVE_TARGETINFO=3DLLVMInitializeX86TargetInfo -DLLVM_NAT= IVE_TARGETMC=3DLLVMInitializeX86TargetMC -ffunction-sections -fdata-section= s -MD -MF.depend.Analysis_ConstantFolding.o -MTAnalysis/ConstantFolding.o -= Qunused-arguments -I/usr/obj/usr/src/tmp/legacy/usr/include -std=3Dc++11 -= fno-exceptions -fno-rtti -stdlib=3Dlibc++ -Wno-c++11-extensions -c /usr/sr= c/contrib/llvm/lib/Analysis/ConstantFolding.cpp -o Analysis/ConstantFolding= =2Eo In file included from /usr/src/contrib/llvm/lib/Analysis/ConstantFolding.cp= p:49: In file included from /usr/include/c++/v1/cfenv:57: /usr/include/fenv.h:270:3: error: use of undeclared identifier '__fnclex' __fnclex(); ^ 1 error generated. *** Error code 1 Stop. make[4]: stopped in /usr/src/lib/clang/libllvm *** Error code 1 Stop. make[3]: stopped in /usr/src/lib/clang *** Error code 1 Stop. make[2]: stopped in /usr/src *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src From owner-freebsd-stable@freebsd.org Tue Apr 30 07:10:51 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F06601585A38 for ; Tue, 30 Apr 2019 07:10:50 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.netfence.it (net-2-44-121-52.cust.vodafonedsl.it [2.44.121.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mailserver.netfence.it", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BEA6A8FCFA for ; Tue, 30 Apr 2019 07:10:48 +0000 (UTC) (envelope-from ml@netfence.it) Received: from alamar.ventu (alamar.local.netfence.it [10.1.2.18]) (authenticated bits=0) by soth.netfence.it (8.15.2/8.15.2) with ESMTPSA id x3U7AZDP003298 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 30 Apr 2019 09:10:44 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.netfence.it: Host alamar.local.netfence.it [10.1.2.18] claimed to be alamar.ventu Subject: Re: ZFS... To: Michelle Sullivan Cc: freebsd-stable References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> From: Andrea Venturoli Message-ID: <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> Date: Tue, 30 Apr 2019 09:10:35 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: BEA6A8FCFA X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of ml@netfence.it designates 2.44.121.52 as permitted sender) smtp.mailfrom=ml@netfence.it X-Spamd-Result: default: False [-2.92 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.989,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:2.44.121.52]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; DMARC_NA(0.00)[netfence.it]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: mailserver.netfence.it]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.42)[ip: (-1.21), ipnet: 2.44.0.0/16(-0.61), asn: 30722(-0.31), country: IT(0.05)]; NEURAL_HAM_SHORT(-0.65)[-0.653,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:30722, ipnet:2.44.0.0/16, country:IT]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 07:10:51 -0000 On 4/30/19 2:41 AM, Michelle Sullivan wrote: > The system was originally built on 9.0, and got upgraded through out the years... zfsd was not available back then. So get your point, but maybe you didn’t realize this blog was a history of 8+ years? That's one of the first things I thought about while reading the original post: what can be inferred from it is that ZFS might not have been that good in the past. It *could* still suffer from the same problems or it *could* have improved and be more resilient. Answering that would be interesting... bye av. From owner-freebsd-stable@freebsd.org Tue Apr 30 08:09:13 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2CCF1587B27 for ; Tue, 30 Apr 2019 08:09:12 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id BB3219253F for ; Tue, 30 Apr 2019 08:09:11 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQR002HSLYFY400@hades.sorbs.net> for freebsd-stable@freebsd.org; Tue, 30 Apr 2019 01:23:06 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> Date: Tue, 30 Apr 2019 18:09:06 +1000 Cc: freebsd-stable Content-transfer-encoding: quoted-printable Message-id: <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> To: Andrea Venturoli X-Rspamd-Queue-Id: BB3219253F X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.95 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-1.21)[ip: (-3.37), ipnet: 72.12.192.0/19(-1.50), asn: 11114(-1.13), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 08:09:13 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 30 Apr 2019, at 17:10, Andrea Venturoli wrote: >=20 >> On 4/30/19 2:41 AM, Michelle Sullivan wrote: >>=20 >> The system was originally built on 9.0, and got upgraded through out the y= ears... zfsd was not available back then. So get your point, but maybe you d= idn=E2=80=99t realize this blog was a history of 8+ years? >=20 > That's one of the first things I thought about while reading the original p= ost: what can be inferred from it is that ZFS might not have been that good i= n the past. > It *could* still suffer from the same problems or it *could* have improved= and be more resilient. > Answering that would be interesting... >=20 Without a doubt it has come a long way, but in my opinion, until there is a t= ool to walk the data (to transfer it out) or something that can either repai= r or invalidate metadata (such as a spacemap corruption) there is still a fa= tal flaw that makes it questionable to use... and that is for one reason alo= ne (regardless of my current problems.) Consider.. If one triggers such a fault on a production server, how can one justify tra= nsferring from backup multiple terabytes (or even petabytes now) of data to r= epair an unmountable/faulted array.... because all backup solutions I know c= urrently would take days if not weeks to restore the sort of store ZFS is to= uted with supporting. =20 Now, yes most production environments have multiple backing stores so will h= ave a server or ten to switch to whilst the store is being recovered, but it= still wouldn=E2=80=99t be a pleasant experience... not to mention the possi= bility that if one store is corrupted there is a chance that the other store= (s) would also be affected in the same way if in the same DC... (Eg a DC fir= e - which I have seen) .. and if you have multi DC stores to protect from th= at.. size of the pipes between DCs comes clearly into play. Thoughts? Michelle From owner-freebsd-stable@freebsd.org Tue Apr 30 08:26:30 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02AFB158888C for ; Tue, 30 Apr 2019 08:26:30 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 86C129325A for ; Tue, 30 Apr 2019 08:26:29 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from pi by home.opsec.eu with local (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hLO5r-0003ze-1Y; Tue, 30 Apr 2019 10:26:27 +0200 Date: Tue, 30 Apr 2019 10:26:27 +0200 From: Kurt Jaeger To: Michelle Sullivan Cc: freebsd-stable Subject: Re: ZFS... Message-ID: <20190430082626.GX72200@home.opsec.eu> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 08:26:30 -0000 Hi! > If one triggers such a fault on a production server, how can one > justify transferring from backup multiple terabytes (or even petabytes > now) of data to repair an unmountable/faulted array.... because all > backup solutions I know currently would take days if not weeks to > restore the sort of store ZFS is touted with supporting. Isn't that the problem with all large storage systems ? Even mainframe storage (with very different concepts behind it) can become messed up so much that there's no hope left. -- pi@opsec.eu +49 171 3101372 One year to go ! From owner-freebsd-stable@freebsd.org Tue Apr 30 08:46:17 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 434D81589012; Tue, 30 Apr 2019 08:46:17 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id B2AA893D34; Tue, 30 Apr 2019 08:46:15 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 30 Apr 2019 10:45:03 +0200 Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 2A3CD7DA-B9BA-428B-9283-2F3763BC7073.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 30 Apr 2019 10:45:00 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 30 Apr 2019 10:44:59 +0200 From: rainer@ultra-secure.de To: Michelle Sullivan Cc: Andrea Venturoli , freebsd-stable , owner-freebsd-stable@freebsd.org Subject: Re: ZFS... In-Reply-To: <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> Message-ID: <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 927, bad: 0, connections: 933, history: 927, pass:all_good, relaying X-Rspamd-Queue-Id: B2AA893D34 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [2.59 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.01)[-0.011,0]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.72)[0.722,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[ultra-secure.de]; AUTH_NA(1.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[connect.ultra-secure.de]; NEURAL_SPAM_LONG(0.62)[0.620,0]; FROM_NO_DN(0.00)[]; SUBJ_ALL_CAPS(0.45)[6]; R_SPF_NA(0.00)[]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:88.198.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-0.18)[ipnet: 88.198.0.0/16(0.80), asn: 24940(-1.71), country: DE(-0.01)] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 08:46:17 -0000 Am 2019-04-30 10:09, schrieb Michelle Sullivan: > Now, yes most production environments have multiple backing stores so > will have a server or ten to switch to whilst the store is being > recovered, but it still wouldn’t be a pleasant experience... not to > mention the possibility that if one store is corrupted there is a > chance that the other store(s) would also be affected in the same way > if in the same DC... (Eg a DC fire - which I have seen) .. and if you > have multi DC stores to protect from that.. size of the pipes between > DCs comes clearly into play. I have one customer with about 13T of ZFS - and because it would take a while to restore (actual backups), it zfs-sends delta-snapshots every hour to a standby-system. It was handy when we had to rebuild the system with different HBAs. From owner-freebsd-stable@freebsd.org Tue Apr 30 09:05:56 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5DF7A1589A67; Tue, 30 Apr 2019 09:05:56 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 4153094CA0; Tue, 30 Apr 2019 09:05:55 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQR002K7OKZY400@hades.sorbs.net>; Tue, 30 Apr 2019 02:19:50 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> Date: Tue, 30 Apr 2019 19:05:50 +1000 Cc: Andrea Venturoli , freebsd-stable , owner-freebsd-stable@freebsd.org Content-transfer-encoding: quoted-printable Message-id: <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> To: rainer@ultra-secure.de X-Rspamd-Queue-Id: 4153094CA0 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.58 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; NEURAL_HAM_SHORT(-0.69)[-0.689,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-1.13)[ip: (-3.13), ipnet: 72.12.192.0/19(-1.41), asn: 11114(-1.07), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 09:05:56 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 30 Apr 2019, at 18:44, rainer@ultra-secure.de wrote: >=20 > Am 2019-04-30 10:09, schrieb Michelle Sullivan: >=20 >> Now, yes most production environments have multiple backing stores so >> will have a server or ten to switch to whilst the store is being >> recovered, but it still wouldn=E2=80=99t be a pleasant experience... not t= o >> mention the possibility that if one store is corrupted there is a >> chance that the other store(s) would also be affected in the same way >> if in the same DC... (Eg a DC fire - which I have seen) .. and if you >> have multi DC stores to protect from that.. size of the pipes between >> DCs comes clearly into play. >=20 >=20 > I have one customer with about 13T of ZFS - and because it would take a wh= ile to restore (actual backups), it zfs-sends delta-snapshots every hour to a= standby-system. >=20 > It was handy when we had to rebuild the system with different HBAs. >=20 >=20 I wonder what would happen if you scaled that up by just 10 (storage) and ha= d the master blow up where it needs to be restored from backup.. how long wo= uld one be praying to higher powers that there is no problem with the backup= ...? (As in no outage or error causing a complete outAge.)... don=E2=80=99t g= et me wrong.. we all get to that position at sometime, but in my recent expe= rience 2 issues colliding at the same time results in disaster. 13T is real= ly not something I have issues with as I can usually cobble something togeth= er with 16T.. (at least until 6T drives became a viable (cost and availabili= ty at short notice) option... even 10T is becoming easier to get a hold of n= ow.. but I have a measly 96T here and it takes weeks even with gigabit bonde= d interfaces when I need to restore.= From owner-freebsd-stable@freebsd.org Tue Apr 30 09:23:37 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 42970158A48C; Tue, 30 Apr 2019 09:23:37 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 94CFA95873; Tue, 30 Apr 2019 09:23:35 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Tue, 30 Apr 2019 11:23:34 +0200 Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 127.0.0.10 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=127.0.0.10; helo=connect.ultra-secure.de; envelope-from= Received: from connect.ultra-secure.de (webmail [127.0.0.10]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 17A9EBBD-58B2-4C0A-8594-6CB75BDD3822.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=AES256-SHA verify=NO); Tue, 30 Apr 2019 11:23:32 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 30 Apr 2019 11:23:31 +0200 From: rainer@ultra-secure.de To: Michelle Sullivan Cc: Andrea Venturoli , freebsd-stable , owner-freebsd-stable@freebsd.org Subject: Re: ZFS... In-Reply-To: <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> Message-ID: <3634a7b3fc29b09ad0a662a4ac0e1b7a@ultra-secure.de> X-Sender: rainer@ultra-secure.de User-Agent: Roundcube Webmail/1.2.0 X-Haraka-GeoIP: --, , NaNkm X-Haraka-GeoIP-Received: X-Haraka-p0f: os="undefined undefined" link_type="undefined" distance=undefined total_conn=undefined shared_ip=Y X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 928, bad: 0, connections: 934, history: 928, pass:all_good, relaying X-Rspamd-Queue-Id: 94CFA95873 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [2.45 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: connect.ultra-secure.de]; SUBJ_ALL_CAPS(0.45)[6]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:88.198.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.03)[-0.027,0]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.61)[0.611,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[ultra-secure.de]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.61)[0.612,0]; FROM_NO_DN(0.00)[]; R_SPF_NA(0.00)[]; IP_SCORE(-0.18)[ipnet: 88.198.0.0/16(0.79), asn: 24940(-1.71), country: DE(-0.01)] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 09:23:37 -0000 Am 2019-04-30 11:05, schrieb Michelle Sullivan: > I wonder what would happen if you scaled that up by just 10 (storage) > and had the master blow up where it needs to be restored from backup.. > how long would one be praying to higher powers that there is no > problem with the backup...? Well, the backup itself takes over a day, AFAIK. I'm not sure why it is that slow. Maybe the old SAS6 HBAs. It's also lots of files. > (As in no outage or error causing a > complete outAge.)... don’t get me wrong.. we all get to that position > at sometime, but in my recent experience 2 issues colliding at the > same time results in disaster. 13T is really not something I have > issues with as I can usually cobble something together with 16T.. (at > least until 6T drives became a viable (cost and availability at short > notice) option... even 10T is becoming easier to get a hold of now.. > but I have a measly 96T here and it takes weeks even with gigabit > bonded interfaces when I need to restore. Those are all SAS drives, actually. 600 and 1200 GB. From owner-freebsd-stable@freebsd.org Tue Apr 30 09:50:59 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAFB4158AD77; Tue, 30 Apr 2019 09:50:59 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D713F967CB; Tue, 30 Apr 2019 09:50:58 +0000 (UTC) (envelope-from delphij@gmail.com) Received: by mail-io1-xd2c.google.com with SMTP id u12so11685174iop.0; Tue, 30 Apr 2019 02:50:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rL0uEn9GIJsVfF474I0qV3xC8yy/pTExmK8Xhy+g5Jc=; b=NMs9EaolCWZVWbVkDP2qpQ1m/2qYQFUgibcuM/1qWgXT7sA26qQX/3Moym1imaemsl smxbfDi8H3c/Hl7qlWIi1m8bQrQQHcuS7qJs0BSlSdzk6ptxW0Rc5d6383XJcMOGE4Yz HpawlT5dz+5yOKMHVbNh44hVbZH+E9W1U9Wt5DJaCnEH0DHPA/nbnHHTI8sP/Wgqslns aBRF7C9RPZfnmkMcVitOSHhCrKDFOA9VdA9ACn8vah/b+3wcsKppb6Ljjs3KCPJURNs/ 78tmZHWE773DpiWihlqz2eDTJegZo+drD16GrVdpvCqGDUav3m/K5GGMN7M9NepXT3en uGKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rL0uEn9GIJsVfF474I0qV3xC8yy/pTExmK8Xhy+g5Jc=; b=mOINMek+G40pucncVcZ8csrwJ2tgtczb4C/hO6E594vbWykWW4HaiHi1clARKfvRDx Wpigjoa6xY02mG6lrJGBmnjCPpaCrPCPyVupq0/x5imuVRpLVUv6D8N3lwlmcBGTVZe9 DxLGre0GXnlXcL5D8T0F3KYN1RtiEfqHbJCyDy0UHHn0mhwvqJOKfJfnUy7gUIRNLGbZ SShIcGEPxtJXQIgXXa7tfAuSwlXz42k6OauN09cbQt19bMXFY9JrE/laV0NxT64q08M3 f/zX4oI7glhQmhZz/Yic4SflFI/fb7/wbrqc+tsk3jq8WKl3rXiqtucgkBmDalyHveHa GJ2g== X-Gm-Message-State: APjAAAXHQZGBkV9W7lnNBuI2ELrBWJEXUIMXsXClCDuKIkstAbC4zZCL yRhRNAvtJNbn+Uzy7wHIGq26IOPRRxEIhZKWbNDIp/Tl X-Google-Smtp-Source: APXvYqwHMSHx6G3Dol+ayowyBt0nlwUVmk5HdrdAMKfiweIF4CiTJHMUt3/+0Lp/3+9lQTFvEP1ANRQASiPH2eMy+5U= X-Received: by 2002:a5d:8e19:: with SMTP id e25mr26907698iod.139.1556617857959; Tue, 30 Apr 2019 02:50:57 -0700 (PDT) MIME-Version: 1.0 References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> In-Reply-To: <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> From: Xin LI Date: Tue, 30 Apr 2019 17:50:40 +0800 Message-ID: Subject: Re: ZFS... To: Michelle Sullivan Cc: rainer@ultra-secure.de, owner-freebsd-stable@freebsd.org, freebsd-stable , Andrea Venturoli X-Rspamd-Queue-Id: D713F967CB X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=NMs9Eaol; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of delphij@gmail.com designates 2607:f8b0:4864:20::d2c as permitted sender) smtp.mailfrom=delphij@gmail.com X-Spamd-Result: default: False [-6.42 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_HAM_SHORT(-0.96)[-0.963,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[c.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-2.90)[ip: (-9.04), ipnet: 2607:f8b0::/32(-3.16), asn: 15169(-2.24), country: US(-0.06)]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 09:51:00 -0000 On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wrote: > but in my recent experience 2 issues colliding at the same time results in > disaster > Do we know exactly what kind of corruption happen to your pool? If you see it twice in a row, it might suggest a software bug that should be investigated. Note that ZFS stores multiple copies of its essential metadata, and in my experience with my old, consumer grade crappy hardware (non-ECC RAM, with several faulty, single hard drive pool: bad enough to crash almost monthly and damages my data from time to time), I've never seen a corruption this bad and I was always able to recover the pool. At previous employer, the only case that we had the pool corrupted enough to the point that mount was not allowed was because two host nodes happen to import the pool at the same time, which is a situation that can be avoided with SCSI reservation; their hardware was of much better quality, though. Speaking for a tool like 'fsck': I think I'm mostly convinced that it's *not* necessary, because at the point ZFS says the metadata is corrupted, it means that these metadata was really corrupted beyond repair (all replicas were corrupted; otherwise it would recover by finding out the right block and rewrite the bad ones). An interactive tool may be useful (e.g. "I saw data structure version 1, 2, 3 available, and all with bad checksum, choose which one you would want to try"), but I think they wouldn't be very practical for use with large data pools -- unlike traditional filesystems, ZFS uses copy-on-write and heavily depends on the metadata to find where the data is, and a regular "scan" is not really useful. I'd agree that you need a full backup anyway, regardless what storage system is used, though. From owner-freebsd-stable@freebsd.org Tue Apr 30 10:10:41 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32206158B282 for ; Tue, 30 Apr 2019 10:10:41 +0000 (UTC) (envelope-from emss.mail@gmail.com) Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0272D97086 for ; Tue, 30 Apr 2019 10:10:40 +0000 (UTC) (envelope-from emss.mail@gmail.com) Received: by mail-wr1-x432.google.com with SMTP id s15so20267196wra.12 for ; Tue, 30 Apr 2019 03:10:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:user-agent:date:message-id:mime-version; bh=PPDCtVjh3cRRQzGcYZvfhFqIAXMp5ZHp7hnCxCAIPsA=; b=irvikHZIHf4o96sztHHK24t5+SBDS89qOno5Z8VcAhNXePdxN5LjdUSegBMwh7Rc3/ bOEOxlbuxuRvKTk1zpJDZic3RbtTpQCITdIiLxxTbA3MQxt5GsAGu4FfNVCF885+dYBy YBFwQ039V7spD6LNR/Y5PROUml59DmR/eevKGN9RoQHOn8P6GlGqCpezzbFfH8XDOpG6 rGcycVXrskcJxaDqmsnamsHv1JOU0vDxiv05ujEu7CT0ikkd+EQr1bsyT8hD9siQ9pem 2yYjiGb+pv4c7y9y+vBpb1eT2SYa4RNIrzj1+MrunoXkdYtUz4OBsS88khfYL8WTpROn mm7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:user-agent:date :message-id:mime-version; bh=PPDCtVjh3cRRQzGcYZvfhFqIAXMp5ZHp7hnCxCAIPsA=; b=VSJbIcH68C35l98Ui9b6qiGttvfrCL2Dv+uz0kI9ktN660VxVvGdK9ypZYr3KrQAIo 3HnPHuZ3NAIQJ16Uf3O5bw1n2V85bW73tOk3RdSmYfJ4jGg/D30s2I91SuKf7TuXDwbg YPz0JvdLb65HRWZ0qp7LrSSMr+vQyKRTLHxUcES3qJ+tAho/hsz8QBQgomq2nSkN6+37 GuEkdGkLFYsolcCL4KwXddEHypTDRucPjHpIHe1MSm6BtFcHGaCUjI6WV1aZOYu8sN1L oe8SdFAcjiu9jGRZS6isSzBTyBK1UT7Dhy+9MU7k1HFfeHXLSxUo6uK9PsWanRSYOQ0f iItA== X-Gm-Message-State: APjAAAUPOP6fbQREL+pXJaLVepSvrIHhAMnoUkKE/tqe8ZX37FPr+4Wy BDZhsnyR1LkTysH+y7aajQBDfVpv X-Google-Smtp-Source: APXvYqxW9G5W2LNsBoD6dsyg26noTJI/Lt26WRj1v9ybeBVkttuWbasmfYyAPj6WFJpjD/5LwlkrCw== X-Received: by 2002:adf:e88b:: with SMTP id d11mr24206572wrm.327.1556619038617; Tue, 30 Apr 2019 03:10:38 -0700 (PDT) Received: from srvbsdfenssv.interne.associated-bears.org (lstlambert-658-1-110-48.w217-128.abo.wanadoo.fr. [217.128.200.48]) by smtp.gmail.com with ESMTPSA id q4sm12065439wrg.24.2019.04.30.03.10.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2019 03:10:37 -0700 (PDT) Sender: Eric Masson Received: from srvbsdfenssv.interne.associated-bears.org (localhost [127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (Postfix) with ESMTP id 8DD31122 for ; Tue, 30 Apr 2019 12:10:36 +0200 (CEST) X-Virus-Scanned: amavisd-new at interne.associated-bears.org Received: from srvbsdfenssv.interne.associated-bears.org ([127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (srvbsdfenssv.interne.associated-bears.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UFFuH_Conoxv for ; Tue, 30 Apr 2019 12:10:34 +0200 (CEST) Received: by srvbsdfenssv.interne.associated-bears.org (Postfix, from userid 1001) id 38600115; Tue, 30 Apr 2019 12:10:34 +0200 (CEST) From: Eric Masson To: Mailing List FreeBSD Stable Subject: zpool dumps core User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (berkeley-unix) X-Operating-System: FreeBSD 11.2-STABLE amd64 Date: Tue, 30 Apr 2019 12:10:33 +0200 Message-ID: <86sgtzvm5i.fsf@srvbsdfenssv.interne.associated-bears.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Rspamd-Queue-Id: 0272D97086 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=irvikHZI; spf=pass (mx1.freebsd.org: domain of emssmail@gmail.com designates 2a00:1450:4864:20::432 as permitted sender) smtp.mailfrom=emssmail@gmail.com X-Spamd-Result: default: False [-5.03 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[free.fr]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; CTYPE_MIXED_BOGUS(1.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FORGED_SENDER(0.30)[emss@free.fr,emssmail@gmail.com]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[emss@free.fr,emssmail@gmail.com]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[free.fr]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(-2.82)[ip: (-9.55), ipnet: 2a00:1450::/32(-2.26), asn: 15169(-2.24), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[2.3.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-1.00)[-0.996,0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 10:10:41 -0000 --=-=-= Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Hi, zpool dumps core on my box. FreeBSD srvbsdfenssv.xxx 11.2-STABLE FreeBSD 11.2-STABLE #2 r346942M: Tue Apr 30 09:54:06 CEST 2019 emss@srvbsdfenssv.xxx:/usr/obj/usr/src/sys/SE7525GP2 amd64 gdb bt attached. Any idea, please ? Regards ric Masson --=-=-= Content-Type: text/plain Content-Disposition: inline; filename=zpool_bt.txt root@srvbsdfenssv:~ # gdb zpool GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... (gdb) core zpool.core Core was generated by `zpool status'. Program terminated with signal 11, Segmentation fault. Reading symbols from /lib/libgeom.so.5...Reading symbols from /usr/lib/debug//lib/libgeom.so.5.debug...done. done. Loaded symbols for /lib/libgeom.so.5 Reading symbols from /lib/libnvpair.so.2...Reading symbols from /usr/lib/debug//lib/libnvpair.so.2.debug...done. done. Loaded symbols for /lib/libnvpair.so.2 Reading symbols from /lib/libuutil.so.2...Reading symbols from /usr/lib/debug//lib/libuutil.so.2.debug...done. done. Loaded symbols for /lib/libuutil.so.2 Reading symbols from /lib/libzfs.so.2...Reading symbols from /usr/lib/debug//lib/libzfs.so.2.debug...done. done. Loaded symbols for /lib/libzfs.so.2 Reading symbols from /lib/libc.so.7...Reading symbols from /usr/lib/debug//lib/libc.so.7.debug...done. done. Loaded symbols for /lib/libc.so.7 Reading symbols from /lib/libbsdxml.so.4...Reading symbols from /usr/lib/debug//lib/libbsdxml.so.4.debug...done. done. Loaded symbols for /lib/libbsdxml.so.4 Reading symbols from /lib/libsbuf.so.6...Reading symbols from /usr/lib/debug//lib/libsbuf.so.6.debug...done. done. Loaded symbols for /lib/libsbuf.so.6 Reading symbols from /lib/libmd.so.6...Reading symbols from /usr/lib/debug//lib/libmd.so.6.debug...done. done. Loaded symbols for /lib/libmd.so.6 Reading symbols from /lib/libumem.so.2...Reading symbols from /usr/lib/debug//lib/libumem.so.2.debug...done. done. Loaded symbols for /lib/libumem.so.2 Reading symbols from /lib/libutil.so.9...Reading symbols from /usr/lib/debug//lib/libutil.so.9.debug...done. done. Loaded symbols for /lib/libutil.so.9 Reading symbols from /lib/libm.so.5...Reading symbols from /usr/lib/debug//lib/libm.so.5.debug...done. done. Loaded symbols for /lib/libm.so.5 Reading symbols from /lib/libavl.so.2...Reading symbols from /usr/lib/debug//lib/libavl.so.2.debug...done. done. Loaded symbols for /lib/libavl.so.2 Reading symbols from /lib/libz.so.6...Reading symbols from /usr/lib/debug//lib/libz.so.6.debug...done. done. Loaded symbols for /lib/libz.so.6 Reading symbols from /lib/libzfs_core.so.2...Reading symbols from /usr/lib/debug//lib/libzfs_core.so.2.debug...done. done. Loaded symbols for /lib/libzfs_core.so.2 Reading symbols from /lib/libthr.so.3...Reading symbols from /usr/lib/debug//lib/libthr.so.3.debug...done. done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /libexec/ld-elf.so.1...Reading symbols from /usr/lib/debug//libexec/ld-elf.so.1.debug...done. done. Loaded symbols for /libexec/ld-elf.so.1 #0 avl_first (tree=0x28) at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:187 187 size_t off = tree->avl_offset; Current language: auto; currently minimal (gdb) bt #0 avl_first (tree=0x28) at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:187 #1 0x0000000800e8a806 in zpool_iter (hdl=0x802c42000, func=0x410920 , data=0x802c35020) at /usr/src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_config.c:417 #2 0x00000000004108cb in pool_list_get (argc=0, argv=0x7fffffffec20, proplist=0x0, err=0x7fffffffd90c) at /usr/src/cddl/contrib/opensolaris/cmd/zpool/zpool_iter.c:128 #3 0x0000000000410bfd in for_each_pool (argc=, argv=, unavail=B_TRUE, proplist=, func=0x406fd0 , data=0x7fffffffd950) at /usr/src/cddl/contrib/opensolaris/cmd/zpool/zpool_iter.c:246 #4 0x000000000040a4e3 in zpool_do_status (argc=0, argv=0x7fffffffec20) at /usr/src/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:5245 #5 0x0000000000408501 in main (argc=, argv=0x7fffffffec10) at /usr/src/cddl/contrib/opensolaris/cmd/zpool/zpool_main.c:6338 #6 0x000000000040481d in _start () #7 0x000000080063e000 in ?? () #8 0x0000000000000000 in ?? () (gdb) --=-=-=-- From owner-freebsd-stable@freebsd.org Tue Apr 30 10:14:26 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 33741158B5D8; Tue, 30 Apr 2019 10:14:26 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id D582897491; Tue, 30 Apr 2019 10:14:24 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQR002N8RR3Y400@hades.sorbs.net>; Tue, 30 Apr 2019 03:28:19 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Tue, 30 Apr 2019 20:14:19 +1000 Cc: rainer@ultra-secure.de, owner-freebsd-stable@freebsd.org, freebsd-stable , Andrea Venturoli Message-id: <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> To: Xin LI X-Rspamd-Queue-Id: D582897491 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.81 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[sorbs.net]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-1.06)[ip: (-2.92), ipnet: 72.12.192.0/19(-1.33), asn: 11114(-1.01), country: US(-0.06)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 10:14:26 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 30 Apr 2019, at 19:50, Xin LI wrote: >=20 >=20 >> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wr= ote: >> but in my recent experience 2 issues colliding at the same time results i= n disaster >=20 > Do we know exactly what kind of corruption happen to your pool? If you se= e it twice in a row, it might suggest a software bug that should be investig= ated. All I know is it=E2=80=99s a checksum error on a meta slab (122) and from wh= at I can gather it=E2=80=99s the spacemap that is corrupt... but I am no exp= ert. I don=E2=80=99t believe it=E2=80=99s a software fault as such, because= this was cause by a hard outage (damaged UPSes) whilst resilvering a single= (but completely failed) drive. ...and after the first outage a second occu= rred (same as the first but more damaging to the power hardware)... the host= itself was not damaged nor were the drives or controller. >=20 > Note that ZFS stores multiple copies of its essential metadata, and in my e= xperience with my old, consumer grade crappy hardware (non-ECC RAM, with sev= eral faulty, single hard drive pool: bad enough to crash almost monthly and d= amages my data from time to time), This was a top end consumer grade mb with non ecc ram that had been running f= or 8+ years without fault (except for hard drive platter failures.). Uptime w= ould have been years if it wasn=E2=80=99t for patching. > I've never seen a corruption this bad and I was always able to recover the= pool.=20 So far, same. > At previous employer, the only case that we had the pool corrupted enough t= o the point that mount was not allowed was because two host nodes happen to i= mport the pool at the same time, which is a situation that can be avoided wi= th SCSI reservation; their hardware was of much better quality, though. >=20 > Speaking for a tool like 'fsck': I think I'm mostly convinced that it's no= t necessary, because at the point ZFS says the metadata is corrupted, it mea= ns that these metadata was really corrupted beyond repair (all replicas were= corrupted; otherwise it would recover by finding out the right block and re= write the bad ones). I see this message all the time and mostly agree.. actually I do agree with p= ossibly a minor exception, but so minor it=E2=80=99s probably not worth it. = However as I suggested in my original post.. the pool says the files are th= ere, a tool that would send them (aka zfs send) but ignoring errors to space= maps etc would be real useful (to me.) >=20 > An interactive tool may be useful (e.g. "I saw data structure version 1, 2= , 3 available, and all with bad checksum, choose which one you would want to= try"), but I think they wouldn't be very practical for use with large data p= ools -- unlike traditional filesystems, ZFS uses copy-on-write and heavily d= epends on the metadata to find where the data is, and a regular "scan" is no= t really useful. Zdb -AAA showed (shows) 36m files.. which suggests the data is intact, but i= t aborts the mount with I/o error because it says metadata has three errors.= . 2 =E2=80=98metadata=E2=80=99 and one =E2=80=9C=E2=80=9D (stor= age being the pool name).. it does import, and it attempts to resilver but r= eports the resilver finishes at some 780M (ish).. export import and it does i= t all again... zdb without -AAA aborts loading metaslab 122. >=20 > I'd agree that you need a full backup anyway, regardless what storage syst= em is used, though. Yeah.. unlike UFS that has to get really really hosed to restore from backup= with nothing recoverable it seems ZFS can get hosed where issues occur in j= ust the wrong bit... but mostly it is recoverable (and my experience has bee= n some nasty shit that always ended up being recoverable.) Michelle=20= From owner-freebsd-stable@freebsd.org Tue Apr 30 08:23:12 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8927158867E; Tue, 30 Apr 2019 08:23:12 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from theravensnest.org (theravensnest.org [46.226.110.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "theravensnest.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22C2A931AD; Tue, 30 Apr 2019 08:23:11 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from [10.164.186.150] ([167.220.197.150]) (authenticated bits=0) by theravensnest.org (8.15.2/8.15.2) with ESMTPSA id x3U8MAWr015104 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 30 Apr 2019 08:22:10 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: mail: Host [167.220.197.150] claimed to be [10.164.186.150] Subject: Re: CFT: FreeBSD Package Base To: Joe Maloney , Cy Schubert Cc: "Rodney W. Grimes" , Kris Moore , FreeBSD Stable , freebsd-ports@freebsd.org, =?UTF-8?Q?Goran_Meki=c4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, Emmanuel Vadot References: <201904291931.x3TJV73d079802@slippy.cwsent.com> <6EB65F49-47CD-48B2-8C58-BF387D8C7A58@ixsystems.com> From: David Chisnall Message-ID: Date: Tue, 30 Apr 2019 09:22:56 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <6EB65F49-47CD-48B2-8C58-BF387D8C7A58@ixsystems.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 22C2A931AD X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.976,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; ASN(0.00)[asn:29169, ipnet:46.226.108.0/22, country:FR] X-Mailman-Approved-At: Tue, 30 Apr 2019 10:21:09 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 08:23:13 -0000 On 29/04/2019 21:12, Joe Maloney wrote: > With CFT version you chose to build, and package individual components such as sendmail with a port option. That does entirely solve the problem of being able to reinstall sendmail after the fact without a rebuild of the userland (base) port but perhaps base flavors could solve that problem assuming flavors could extend beyond python. This sounds very much like local optimisation. It's now easy to create a custom base image. Great. But how do I express dependencies in ports on a specific base configuration? This is easy if I depend on a specific base package, but how does this work in your model? For example, if I have a package that depends on a library that is an optional part of the base system, how do I express that pkg needs to either refuse to install it, or install a userland pkg that includes that library in place of my existing version as part of the install process? More importantly for the container use case, if I want to take a completely empty jail and do pkg ins nginx (for example), what does the maintainer of the nginx port need to do to express the minimum set of the base system that needs to be installed to allow nginx to work? One of the goals for the pkg base concept was to allow this kind of use case, easily creating a minimal environment required to run a single service. With a monolithic base package set, you're going to need some mechanism other than packages to express the specific base subset package that you need and I think that you need to justify why this mechanism is better than using small individual packages. David From owner-freebsd-stable@freebsd.org Tue Apr 30 11:24:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B1F9158D5CD; Tue, 30 Apr 2019 11:24:16 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DEFB16B6E4; Tue, 30 Apr 2019 11:24:14 +0000 (UTC) (envelope-from mj-mailinglist@gmx.de) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1556623444; bh=C48Y2DAFDvdSncQPnRp9EYT/pMwLrRMv8yayxyDZlIU=; h=X-UI-Sender-Class:From:To:Subject:Date; b=Vbmz8EC0XPWBxF8fDBSybwT1pcjBarTJcrcew6W1uANckhbJ8PcJQjvdfIK/tzvFR PRp1zfBSng4KGBGUk+9YqxQ2V7FTX1htiNUCjoSFDo0MwH+T0C4465gSysCMLd8VFP oGZVPDE6wSylwNf+BRldTgKSBjOvmkDGpUWN49CU= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [194.59.36.88] ([194.59.36.88]) by web-mail.gmx.net (3c-app-gmx-bs07.server.lan [172.19.170.56]) (via HTTP); Tue, 30 Apr 2019 13:24:04 +0200 MIME-Version: 1.0 Message-ID: From: "Martin Jakob" To: freebsd-pkgbase@freebsd.org, freebsd-stable@freebsd.org Subject: Creating jails from pkgbase packages Content-Type: text/plain; charset=UTF-8 Date: Tue, 30 Apr 2019 13:24:04 +0200 Importance: normal Sensitivity: Normal Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-Provags-ID: V03:K1:tLU9i8MVgGDKBk2v80XuLSUrQJhK5X/hN4cPMCASgMLS+yH6CSD4AHSf5VDYu8vw/a9A0 WkV/5UwVeeCCKbvb1+Rlm1HZ7V0mkqret1zFY9OFthT5j6iqusnI82jTjkknr9jMRxn9IK9PvWyc +4L8HJSPSfZXtt1qz7sUuJF+Oy57kQXPSZYebSRw4HGhO85S+kdCIVI/OHNLy2rzx3ISZzeK13ue c1VNgwdpBT+8kYqt2tntRWYKQSa/1DCWucY8aLpbuH/lB9QWwzzd7I8teiwyjj5MpaGMHxOviJaD SY= X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:j6NES14qW/Q=:F5EETivzkJXM15DfCvMA7z fgKbYi/Prx72FrWmT6wQxalUYmUKQQg5u+eGpeVR8FWQFvzwQs9nMykas8NpUPCRUWXcV+ox6 0ZwCRektshKyl+HLbU7pPIjU7v4io7NZ9v4/hYWbArRklexs79K7WopcUg0wAD8qmgAdONWmA h1dATRYzzuH5OJU8ouVtzff+3E0lgAinErhRNT4UxjbVtxdm+czh0VjB4choemWxSncXbLnNR MU09f3QpVR5VGHKbGUxZ1cKGzzRIsXbkLe7MwK5HL8tfgD4D9LERdD2raK0F/kkfx/EnCdYIE C4iFnRr3W/2IKYxIK7BAZuxuZzAkT320MjS0qvQXsUPjsaT8/MbEtF1Vu75cniQNEcoVYT8tt 6qwe0P2pIUJ86BKx6Tpx4ot3GedLfjxatz10VIqVFJoLL1txjBhABiqgGMV4OdrgwyTZEYS1+ fQo28klv9QY/Dnq7n9tlm5F2RxYmbGuUoEhWj7zMyu0eXJ1X0RFHaf9Qsh8/6UZDkdNWQMxvj ASQLmSLpHOeXIiBwVbhx2bhAOUL1xVXAEoR4u2Bm5N2L+noi4mRjLDhixuSU4vS3zNQado3LH 6D+cbevu9F8ZRIodqfqEZMXKdmKN4B376+cGPiKhCMY/K76sVqMJziKw== X-Rspamd-Queue-Id: DEFB16B6E4 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=Vbmz8EC0; spf=pass (mx1.freebsd.org: domain of mj-mailinglist@gmx.de designates 212.227.17.22 as permitted sender) smtp.mailfrom=mj-mailinglist@gmx.de X-Spamd-Result: default: False [-3.86 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; RCVD_IN_DNSWL_LOW(-0.10)[22.17.227.212.list.dnswl.org : 127.0.3.1]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; FREEMAIL_FROM(0.00)[gmx.de]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[gmx.de]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[cached: mx00.emig.gmx.net]; DKIM_TRACE(0.00)[gmx.net:+]; HAS_X_PRIO_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.90)[-0.897,0]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-0.85)[ip: (-4.73), ipnet: 212.227.0.0/16(-1.40), asn: 8560(1.87), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; FREEMAIL_ENVFROM(0.00)[gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 11:24:16 -0000 My first attempt seems to be empty when its arrived at the maillinglist, so= i try it again in pure text format (i hope)=2E Sorry, if there are unwante= d duplicates=2E=20 Since a few days i was experimenting with jails build from pkgbase (i got = the idea from Michael W=2E Lucas Freebsd Jails book)=2E Now there is some d= iscussion about pkgbase again :) So i want to share my experience, i hope this is not offtopic to this conv= ersation and my writing style is not too offputting=2E=2E=2E =C2=A0 This experiments were made on an resently updated 12-stable system=2E Sorr= y, i dont have a system with head=C2=A0available right now=2E =C2=A0 Anyway, this is what i did: - I first set the variable "REPODIR=3D/usr/repo" in /etc/make=2Econf, to h= ave a nonvolatile repository (is this the right place/variable to do this?)= =2E - then after the standard "make buildworld|installworld|buildkernel|instal= lkernel" in /usr/src, i created the pkgbase packages with the command "make= packages"=2E =C2=A0 the files are placed in "/usr/repo/FreeBSD:12:amd64/latest" where l= atest is a symlink to the, well latest build=2E =C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 14 Apr=2E 14:59 12=2E= 0=2Es20190414123806/ =C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 21 Apr=2E 09:27 12=2E= 0=2Es20190419155807/ =C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 22 Apr=2E 13:53 12=2E= 0=2Es20190422094219/ =C2=A0 drwxr-xr-x =C2=A02 root =C2=A0wheel =C2=A0805 27 Apr=2E 14:12 12=2E= 0=2Es20190427084356/ =C2=A0 lrwxr-xr-x =C2=A01 root =C2=A0wheel =C2=A0 20 27 Apr=2E 11:04 lates= t@ -> 12=2E0=2Es20190427084356 =C2=A0=C2=A0 - this repo is published via the file "/usr/local/etc/pkg/repos/FreeBSD-ba= se=2Econf"which has this content: root@betablock:~ # cat "/usr/local/etc/pkg/repos/FreeBSD-base=2Econf" # FreeBSD base system repository FreeBSD-base: { =C2=A0 url: "file:///usr/repo/${ABI}/latest", =C2=A0 mirror_type: "none", =C2=A0 enabled: yes } =C2=A0 now to the jail stuff: - this command installs the FreeBSD-runtime and the 72 neccesary dependenc= ies in the jails root directory /jails/test03: root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru= e' install -r FreeBSD-base FreeBSD-runtime =C2=A0 - the newly created jail "test03" (an entry in /etc/jail=2Econf already ex= ists) is started with the command: root@betablock:~ # jail -vc test03 =C2=A0 sadly the jail wont start, it exits with this message: =2E=2E=2E test03: created test03: run command in jail: /bin/sh /etc/rc jail: test03: getpwnam: No such file or directory jail: test03: /bin/sh /etc/rc: failed test03: removed =2E=2E=2E =C2=A0 This happens, because there are no user-database files (like master=2Epass= wd, pwd=2Edb, spwd=2Edb) and no group files in the jails /etc/ directory=2E= hmmm=2E=2E=2E I saw in the pkg install output of the FreeBSD-runtime =C2=A0pkg this mess= age: =2E=2E=2E [2/72] Extracting FreeBSD-runtime-12=2E0=2Es20190414123806: 100% pwd_mkdb: /jails/test03/etc/master=2Epasswd: No such file or directory =2E=2E=2E so pkg tries to run the pwd_mkdb as defined in the runtime=2Eucl file, or = embedded in the pkg file, it is viewable with this command: root@betablock:~ # pkg info --raw --file /usr/repo/FreeBSD:12:amd64/latest= /FreeBSD-runtime-12=2E0=2Es20190414123806=2Etxz =2E=2E=2E scripts: { =C2=A0 =C2=A0 post-install: "cap_mkdb -l ${PKG_ROOTDIR}/etc/login=2Econf\n= \tpwd_mkdb -i -p -d =C2=A0${PKG_ROOTDIR}/etc ${PKG_ROOTDIR}/etc/master=2Epa= sswd\n\tservices_mkdb -l -q -o ${PKG_ROOTDIR}/var/db/services=2Edb ${PKG_RO= OTDIR}/etc/services\n\tchmod 1777 ${PKG_ROOTDIR}/tmp" } =2E=2E=2E - so for a quick fix, i copied the "missing" files (master=2Epasswd, pwd= =2Edb, spwd=2Edb, group) from a FreeBSD base=2Etxz package to my jails etc = directory and tried to start the jail again=2E Read below for a "better" so= lution=2E Now the jail starts: root@betablock:~ # jls =C2=A0 =C2=A0 =C2=A0JID =C2=A0IP Address =C2=A0 =C2=A0 =C2=A0Hostname =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Pa= th =C2=A0 =C2=A0 =C2=A0=2E=2E=2E =C2=A0 =C2=A0 =C2=A05 =C2=A0192=2E168=2E0=2E113 =C2=A0 test03=2Elocal =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/jails/test03 =C2=A0 i can "enter" it with this command: root@betablock:~ # jexec -l test03 =C2=A0=C2=A0 now to some more experiments with the jail which results in some questions= (also, see below) =C2=A0 - trying to ping something: root@test03:~ # ping google=2Ecom ld-elf=2Eso=2E1: Shared object "libcap_dns=2Eso=2E1" not found, required b= y "ping" =C2=A0 the required library libcap_dns=2Eso=2E1 is in the file FreeBSD-libcasper-= 12=2E0=2Es20190414123806=2Etxz, so lets install it: Outside of the jail: root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru= e' install -r FreeBSD-base FreeBSD-libcasper =C2=A0 and inside the jail: root@test03:~ # ping google=2Ecom PING google=2Ecom (216=2E58=2E213=2E206): 56 data bytes 64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D0 ttl=3D52 time=3D23=2E398 = ms 64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D1 ttl=3D52 time=3D23=2E975 = ms 64 bytes from 216=2E58=2E213=2E206: icmp_seq=3D2 ttl=3D52 time=3D23=2E252 = ms =2E=2E=2E success! =C2=A0 The size of this freshly installed jail: du -h -d 1 /jails/ =2E=2E=2E 90M =C2=A0 =C2=A0/jails/test03 =C2=A0 - now lets install some packages from outside of the jail: root@betablock:~ # pkg --rootdir /jails/test03 install nginx-full Updating FreeBSD repository catalogue=2E=2E=2E pkg: Repository FreeBSD load error: access repo file(/jails/test03/var/db/= pkg/repo-FreeBSD=2Esqlite) failed: No such file or directory Fetching meta=2Etxz: 100% =C2=A0 =C2=A0944 B =C2=A0 0=2E9kB/s =C2=A0 =C2= =A000:01 Fetching packagesite=2Etxz: 100% =C2=A0 =C2=A06 MiB =C2=A0 2=2E2MB/s =C2= =A0 =C2=A000:03 Processing entries: 100% FreeBSD repository update completed=2E 31883 packages processed=2E Updating Synth repository catalogue=2E=2E=2E Synth repository is up to date=2E Updating FreeBSD-base repository catalogue=2E=2E=2E FreeBSD-base repository is up to date=2E All repositories are up to date=2E The following 146 package(s) will be affected (of 0 checked): New packages to be INSTALLED: =C2=A0 =C2=A0 =C2=A0 =C2=A0 nginx-full: 1=2E16=2E0_2,2 [FreeBSD] =C2=A0 =C2=A0 =C2=A0 =C2=A0 apache24: 2=2E4=2E39 [FreeBSD] =C2=A0 =C2=A0 =C2=A0 =C2=A0 libnghttp2: 1=2E38=2E0 [FreeBSD] =2E=2E=2E =C2=A0 =C2=A0 =C2=A0 =C2=A0 libdrizzle: 0=2E8_6 [FreeBSD] =C2=A0 =C2=A0 =C2=A0 =C2=A0 brotli: 1=2E0=2E7_1,1 [FreeBSD] Number of packages to be installed: 146 The process will require 953 MiB more space=2E 232 MiB to be downloaded=2E =2E=2E=2E =C2=A0 Funfact -=C2=A0the jailsize after this installation: 1,0G =C2=A0 =C2=A0/jails/test03 =C2=A0 - lets enable the nginx service root@betablock:~ # sysrc -R /jails/test03 nginx_enable=3DYES nginx_enable: =C2=A0-> YES =C2=A0 - and start it (after a jail restart, without it nginx does not find its l= ibraries, installed above) root@test03:~ # service nginx start Performing sanity check on nginx configuration: ld-elf=2Eso=2E1: Shared object "libexecinfo=2Eso=2E1" not found, required = by "libprofiler=2Eso=2E0" =C2=A0 - lets install this missing library with the command=C2=A0 root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru= e' install -r FreeBSD-base FreeBSD-libexecinfo =C2=A0 - and lets also install vi: root@betablock:~ # pkg --rootdir /jails/test03 -o 'ASSUME_ALWAYS_YES=3Dtru= e' install -r FreeBSD-base FreeBSD-vi =C2=A0 - so lets try again (after creating the missing config files nginx=2Econf = and mime=2Etypes): root@betablock:~ # jexec -l test03 service nginx start Starting nginx=2E nginx: [emerg] gethostbyname() failed in /usr/local/etc/nginx/nginx=2Econf= :122 =C2=A0 - and the check: root@betablock:~ # jexec -l test03 service nginx status nginx is running as pid 41359=2E =C2=A0 (Browser access works too=2E=2E=2E) =C2=A0 - Now Lets work with packages within the jail: root@test03:~ # pkg update The package management tool is not yet installed on your system=2E Do you want to fetch and install it now? [y/N]: y Bootstrapping pkg from pkg+http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/lat= est, please wait=2E=2E=2E pkg: Error loading revoked certificates =C2=A0 - Hm, this is because the directory "/usr/share/keys/pkg/revoked/" is miss= ing, lets create it: root@test03:~ # mkdir /usr/share/keys/pkg/revoked/ =C2=A0 - and now: root@test03:~ # pkg update The package management tool is not yet installed on your system=2E Do you want to fetch and install it now? [y/N]: y Bootstrapping pkg from pkg+http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/lat= est[http://pkg=2EFreeBSD=2Eorg/FreeBSD:12:amd64/latest], please wait=2E=2E= =2E Verifying signature with trusted certificate pkg=2Efreebsd=2Eorg=2E2013102= 301=2E=2E=2E done [test03=2Elocal] Installing pkg-1=2E10=2E5_5=2E=2E=2E [test03=2Elocal] Extracting pkg-1=2E10=2E5_5: 100% Updating FreeBSD repository catalogue=2E=2E=2E FreeBSD repository is up to date=2E All repositories are up to date=2E root@test03:~ # pkg info FreeBSD-casper-12=2E0=2Es20190414123806 casper package FreeBSD-clibs-12=2E0=2Es20190414123806 Core C Libraries FreeBSD-jail-12=2E0=2Es20190414123806 Jail Utilities FreeBSD-lib-12=2E0=2Es20190414123806 lib package FreeBSD-lib80211-12=2E0=2Es20190414123806 lib80211 package =2E=2E=2E x265-3=2E0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 H=2E265/High Efficiency Video Coding (HEVC) format xorg-fonts-truetype-7=2E7_1 =C2=A0 =C2=A0 =C2=A0X=2EOrg TrueType fonts xorgproto-2018=2E4 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 xorg p= rotocol headers xvid-1=2E3=2E5,1 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 Opensource MPEG-4 codec, based on OpenDivx yajl-2=2E1=2E0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Portable JSON parsing and serialization library in ANSI C Btw=2E The missing "revoked" directory problem shoud be fixed with https:/= /reviews=2Efreebsd=2Eorg/D20056[https://reviews=2Efreebsd=2Eorg/D20056] --- So this is a basic jail installation with pkgbase, now to some further exp= eriments and resulting questions: =C2=A0 - i looked into why the userdatabase was missing in the newly created jail= =2E Well, it is, because the files are not included in the default Freebsd-= runtime package=2E =C2=A0 To check if it would work, i built my own Freebsd-runtime package w= ith these files added=2E to do this, i copied the runtime=2Eplist file from= the $WSTAGEDIR and modified it=2E =C2=A0 - first i removed a lot of duplicated directory entries like =C2=A0 @dir(root,wheel,0755,) /boot =C2=A0 @dir(root,wheel,0755,) /boot =C2=A0 @dir(root,wheel,0755,) /boot =C2=A0 @dir(root,wheel,0755,) /boot =C2=A0 =2E=2E=2E (and many more) =C2=A0 - then i added the "missing" files and directory =C2=A0 @config(root,wheel,0600,) /etc/master=2Epasswd =C2=A0 @config(root,wheel,0644,) /etc/group =C2=A0 @dir(root,wheel,0755,) /usr/share/keys/pkg/revoked =C2=A0=C2=A0 =C2=A0 - after that i rebuild the Freebsd-runtime package with these chang= es (found in /usr/src/Makefile=2Einc1): =C2=A0 root@betablock:~ # pkg -o ABI_FILE=3D/usr/obj/usr/src/amd64=2Eamd64= /worldstage/usr/bin/uname -o ALLOW_BASE_SHLIBS=3Dyes create -M /usr/obj/usr= /src/amd64=2Eamd64/worldstage/runtime=2Eucl -p /root/pkgs/runtime=2Eplist -= r /usr/obj/usr/src/amd64=2Eamd64/worldstage --output-dir /usr/repo/FreeBSD:= 12:amd64/latest =C2=A0=C2=A0 =C2=A0 and rebuilding the repo metadata with: =C2=A0 root@betablock:~ # pkg repo /usr/repo/FreeBSD:12:amd64/latest =C2=A0 a jail created with this Freebsd-runtime starts directly, without t= he steps mentioned above=2E =C2=A0=C2=A0 =C2=A0 My Question: Is the ommission of these files intentional?=C2=A0 =C2=A0=C2=A0 =C2=A0 - one last experiment:=C2=A0i deleted the manpages, the examples, a= nd the /boot directory (i guessed a jail has no need for it), the jail buil= d from this file is 75M, the installation of packages like nginx=C2=A0works= like in the original file=2E =C2=A0=C2=A0 =C2=A0 - should the FreeBSD-libcasper package be installed as a dependency= of the FreeBSD-casper package? =C2=A0 =C2=A0 - i did not observe more missing libraries like libcasper above, bu= t depending on the tasks that are=C2=A0executed or packages that are instal= led, there might be more that are missing=2E =C2=A0 So that was my pkgbase-jail experiment, it was fun and i learned some thin= gs about pkgbase=2E Any comments, tips, tricks and (mild) critique are welc= ome=2E =C2=A0 Greetings =C2=A0 Martin =C2=A0 From owner-freebsd-stable@freebsd.org Tue Apr 30 13:12:08 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4067D1590A34 for ; Tue, 30 Apr 2019 13:12:08 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 79B326FCD9 for ; Tue, 30 Apr 2019 13:12:07 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 891462110B0 for ; Tue, 30 Apr 2019 09:11:30 -0400 (EDT) Received: from [192.168.10.16] (D6.Denninger.Net [192.168.10.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id EEBE8C6A28 for ; Tue, 30 Apr 2019 08:11:29 -0500 (CDT) Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Tue, 30 Apr 2019 08:11:29 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms050501030002040403040807" X-Rspamd-Queue-Id: 79B326FCD9 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.14 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[px.denninger.net]; NEURAL_HAM_SHORT(-0.87)[-0.869,0]; SUBJ_ALL_CAPS(0.45)[6]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.51)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.10), asn: 14061(1.48), country: US(-0.06)]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 13:12:08 -0000 This is a cryptographically signed message in MIME format. --------------ms050501030002040403040807 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 4/30/2019 05:14, Michelle Sullivan wrote: >> On 30 Apr 2019, at 19:50, Xin LI wrote: >>> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wrote: >>> but in my recent experience 2 issues colliding at the same time resul= ts in disaster >> Do we know exactly what kind of corruption happen to your pool? If yo= u see it twice in a row, it might suggest a software bug that should be i= nvestigated. >> >> All I know is it=E2=80=99s a checksum error on a meta slab (122) and f= rom what I can gather it=E2=80=99s the spacemap that is corrupt... but I = am no expert. I don=E2=80=99t believe it=E2=80=99s a software fault as s= uch, because this was cause by a hard outage (damaged UPSes) whilst resil= vering a single (but completely failed) drive. ...and after the first ou= tage a second occurred (same as the first but more damaging to the power = hardware)... the host itself was not damaged nor were the drives or contr= oller. =2E.... >> Note that ZFS stores multiple copies of its essential metadata, and in= my experience with my old, consumer grade crappy hardware (non-ECC RAM, = with several faulty, single hard drive pool: bad enough to crash almost m= onthly and damages my data from time to time), > This was a top end consumer grade mb with non ecc ram that had been run= ning for 8+ years without fault (except for hard drive platter failures.)= =2E Uptime would have been years if it wasn=E2=80=99t for patching. Yuck. I'm sorry, but that may well be what nailed you. ECC is not just about the random cosmic ray.=C2=A0 It also saves your bac= on when there are power glitches. Unfortunately however there is also cache memory on most modern hard drives, most of the time (unless you explicitly shut it off) it's on for write caching, and it'll nail you too.=C2=A0 Oh, and it's never, in my experience, ECC. In addition, however, and this is something I learned a LONG time ago (think Z-80 processors!) is that as in so many very important things "two is one and one is none." In other words without a backup you WILL lose data eventually, and it WILL be important. Raidz2 is very nice, but as the name implies it you have two redundancies.=C2=A0 If you take three errors, or if, God forbid, you *wri= te* a block that has a bad checksum in it because it got scrambled while in RAM, you're dead if that happens in the wrong place. > Yeah.. unlike UFS that has to get really really hosed to restore from b= ackup with nothing recoverable it seems ZFS can get hosed where issues oc= cur in just the wrong bit... but mostly it is recoverable (and my experie= nce has been some nasty shit that always ended up being recoverable.) > > Michelle=20 Oh that is definitely NOT true.... again, from hard experience, including (but not limited to) on FreeBSD. My experience is that ZFS is materially more-resilient but there is no such thing as "can never be corrupted by any set of events."=C2=A0 Backup= strategies for moderately large (e.g. many Terabytes) to very large (e.g. Petabytes and beyond) get quite complex but they're also very necessary. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms050501030002040403040807 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNDMwMTMxMTI5 WjBPBgkqhkiG9w0BCQQxQgRAGAIvKJ+W0vCr8nN8QeH2+yE/h15nnjuM5tDjCwHwv585JUJb aIabJFN8ondbeULNS8NqXUzO0hCbiq0efe13mDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgAp1ODWucVojzEgJb42eXySKQDQwlWmpnPJZ6ftfpQNw3jPoBdzsKAtT1SY5QOczMQN gJOGv53NGcEzrfqnp6ZSv6dHlA4kxuoYen/rBHNC/GvczKBIXSvAh5ptR37JWpvB7iR++66i 1q+rJix9Zjk7D0L3HjM1VW1Ov20lxVqljZlBxWfEzggHZ5tXYY50Qsloai8Ih+ThdcIyE9Kg SlGcQbhIVpe74lZNuIMHCbibfU3NXumseK6kHC1kTj+qy3rf+y+peN18Wjt78MfVRtYPBsOq vDfFM+g0QHsayA1u2DYp8LL6WAk/4MbdmLS1iA5Qw3qFzUrLeimQy3kte04ojdiHy13MugNK WlphNyw01AHLZArF7OUrJcBe2hCfwdrNSWZulkU3xn0SyxhQ62nmdb4uuIEc9s3R1zJV+mJ2 zyOKRxrBLl3i+gFkyP6QiYok65hsI2NhcwissOinc+ldSqxIE3f46/tYfXVrArFeYWXNlisX Yp6viR7PeF1YMBxs3FZIwKox21UTElmYTrbz+50w9JnnvZnBuG3Vt5BSpSLuKrLgctcqlWgW Wgc7IIfYhtS0XdYJR/IZbP0euTgpG3Q93vCTdKuaLngAld2HS9EJwpq3jZPlmxabDAvidbpU SHm1MoDNQjFnorHUSr+h6QPOIeDgSejaW7LF1+HqEQAAAAAAAA== --------------ms050501030002040403040807-- From owner-freebsd-stable@freebsd.org Tue Apr 30 13:30:08 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8DC7015910C7 for ; Tue, 30 Apr 2019 13:30:08 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 1710770C0A for ; Tue, 30 Apr 2019 13:30:06 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-transfer-encoding: 8BIT Content-type: text/plain; charset=UTF-8; format=flowed Received: from isux.com (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQS002VG0TCY400@hades.sorbs.net> for freebsd-stable@freebsd.org; Tue, 30 Apr 2019 06:44:01 -0700 (PDT) Subject: Re: ZFS... To: Karl Denninger , freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> From: Michelle Sullivan Message-id: <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> Date: Tue, 30 Apr 2019 23:30:03 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-reply-to: X-Rspamd-Queue-Id: 1710770C0A X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.86)[ip: (-2.30), ipnet: 72.12.192.0/19(-1.09), asn: 11114(-0.83), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.99)[-0.989,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 13:30:08 -0000 Karl Denninger wrote: > On 4/30/2019 05:14, Michelle Sullivan wrote: >>> On 30 Apr 2019, at 19:50, Xin LI wrote: >>>> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wrote: >>>> but in my recent experience 2 issues colliding at the same time results in disaster >>> Do we know exactly what kind of corruption happen to your pool? If you see it twice in a row, it might suggest a software bug that should be investigated. >>> >>> All I know is it’s a checksum error on a meta slab (122) and from what I can gather it’s the spacemap that is corrupt... but I am no expert. I don’t believe it’s a software fault as such, because this was cause by a hard outage (damaged UPSes) whilst resilvering a single (but completely failed) drive. ...and after the first outage a second occurred (same as the first but more damaging to the power hardware)... the host itself was not damaged nor were the drives or controller. > ..... >>> Note that ZFS stores multiple copies of its essential metadata, and in my experience with my old, consumer grade crappy hardware (non-ECC RAM, with several faulty, single hard drive pool: bad enough to crash almost monthly and damages my data from time to time), >> This was a top end consumer grade mb with non ecc ram that had been running for 8+ years without fault (except for hard drive platter failures.). Uptime would have been years if it wasn’t for patching. > Yuck. > > I'm sorry, but that may well be what nailed you. > > ECC is not just about the random cosmic ray. It also saves your bacon > when there are power glitches. No. Sorry no. If the data is only half to disk, ECC isn't going to save you at all... it's all about power on the drives to complete the write. > > Unfortunately however there is also cache memory on most modern hard > drives, most of the time (unless you explicitly shut it off) it's on for > write caching, and it'll nail you too. Oh, and it's never, in my > experience, ECC. No comment on that - you're right in the first part, I can't comment if there are drives with ECC. > > In addition, however, and this is something I learned a LONG time ago > (think Z-80 processors!) is that as in so many very important things > "two is one and one is none." > > In other words without a backup you WILL lose data eventually, and it > WILL be important. > > Raidz2 is very nice, but as the name implies it you have two > redundancies. If you take three errors, or if, God forbid, you *write* > a block that has a bad checksum in it because it got scrambled while in > RAM, you're dead if that happens in the wrong place. Or in my case you write part data therefore invalidating the checksum... > >> Yeah.. unlike UFS that has to get really really hosed to restore from backup with nothing recoverable it seems ZFS can get hosed where issues occur in just the wrong bit... but mostly it is recoverable (and my experience has been some nasty shit that always ended up being recoverable.) >> >> Michelle > Oh that is definitely NOT true.... again, from hard experience, > including (but not limited to) on FreeBSD. > > My experience is that ZFS is materially more-resilient but there is no > such thing as "can never be corrupted by any set of events." The latter part is true - and my blog and my current situation is not limited to or aimed at FreeBSD specifically, FreeBSD is my experience. The former part... it has been very resilient, but I think (based on this certain set of events) it is easily corruptible and I have just been lucky. You just have to hit a certain write to activate the issue, and whilst that write and issue might be very very difficult (read: hit and miss) to hit in normal every day scenarios it can and will eventually happen. > Backup > strategies for moderately large (e.g. many Terabytes) to very large > (e.g. Petabytes and beyond) get quite complex but they're also very > necessary. > and there in lies the problem. If you don't have a many 10's of thousands of dollars backup solutions, you're either: 1/ down for a looooong time. 2/ losing all data and starting again... ..and that's the problem... ufs you can recover most (in most situations) and providing the *data* is there uncorrupted by the fault you can get it all off with various tools even if it is a complete mess.... here I am with the data that is apparently ok, but the metadata is corrupt (and note: as I had stopped writing to the drive when it started resilvering the data - all of it - should be intact... even if a mess.) Michelle -- Michelle Sullivan http://www.mhix.org/ From owner-freebsd-stable@freebsd.org Tue Apr 30 13:33:50 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE9C215913D8 for ; Tue, 30 Apr 2019 13:33:49 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3FBC971090 for ; Tue, 30 Apr 2019 13:33:48 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 218BF2110B0 for ; Tue, 30 Apr 2019 09:33:48 -0400 (EDT) Received: from [192.168.10.16] (D6.Denninger.Net [192.168.10.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 82F48C6AF5 for ; Tue, 30 Apr 2019 08:33:47 -0500 (CDT) Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Tue, 30 Apr 2019 08:33:47 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms050404080707080201050301" X-Rspamd-Queue-Id: 3FBC971090 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.24 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.97)[-0.971,0]; SUBJ_ALL_CAPS(0.45)[6]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-2.51)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.11), asn: 14061(1.48), country: US(-0.06)]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 13:33:50 -0000 This is a cryptographically signed message in MIME format. --------------ms050404080707080201050301 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 4/30/2019 03:09, Michelle Sullivan wrote: > Consider.. > > If one triggers such a fault on a production server, how can one justif= y transferring from backup multiple terabytes (or even petabytes now) of = data to repair an unmountable/faulted array.... because all backup soluti= ons I know currently would take days if not weeks to restore the sort of = store ZFS is touted with supporting. =20 Had it happen on a production server a few years back with ZFS.=C2=A0 The= *hardware* went insane (disk adapter) and scribbled on *all* of the vdevs= =2E The machine crashed and would not come back up -- at all.=C2=A0 I insist = on (and had) emergency boot media physically in the box (a USB key) in any production machine and it was quite-quickly obvious that all of the vdevs were corrupted beyond repair.=C2=A0 There was no rational option ot= her than to restore. It was definitely not a pleasant experience, but this is why when you get into systems and data store sizes where it's a five-alarm pain in the neck you must figure out some sort of strategy that covers you 99% of the time without a large amount of downtime involved, and in the 1% case accept said downtime.=C2=A0 In this particular circumstance the cust= omer didn't want to spend on a doubled-and-transaction-level protected on-site (in the same DC) redundancy setup originally so restore, as opposed to fail-over/promote and then restore and build a new "redundant" box where the old "primary" resided was the most-viable option.=C2=A0 Time to recover essential functions was ~8 hours (and over = 24 hours for everything to be restored.) Incidentally that's not the first time I've had a disk adapter failure on a production machine in my career as a systems dude; it was, in fact, the *third* such failure.=C2=A0 Then again I've been doing this stuff sin= ce the 1980s and learned long ago that if it can break it eventually will, and that Murphy is a real b******. The answer to your question Michelle is that when restore times get into "seriously disruptive" amounts of time (e.g. hours, days or worse depending on the application involved and how critical it is) you spend the time and money to have redundancy in multiple places and via paths that do not destroy the redundant copies when things go wrong, and you spend the engineering time to figure out what those potential faults are and how to design such that a fault which can destroy the data set does not propagate to the redundant copies before it is detected. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms050404080707080201050301 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNDMwMTMzMzQ3 WjBPBgkqhkiG9w0BCQQxQgRA4Xydx+FaIhQIZPGx00ETVAqMmnh0wBsZLpdXLCcvSNtOA3nJ AHfB2Fv3yeAgpJw7Zq1pegY4M4Z7lkcvzwc/8zBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgAIrGd3/aKBnCQMaShStflcJIJT4eFz/CujNShmtcBUphkKx+4y5aUKQ+KbdJUPADJL p3cVDLVLAhfwEg47nKkzquGfbIl2ypVnD8Mn3oo72//XiFER7z3Uao1YguWQHHObsuYkun16 eeJeHxihiRLNw+VIu9sSpD2YPYt5e4SPLHhTshxcDqCxRyrmp18kjMEGUAiHexjAxGkcm6nv t80Ov3FguWz/F2FFyPZqTgKX1kO6yJnQVZYiAn/xhoK7UMk5oF9IofClTVuB2C99KdlJ3KgE 2WHgBumbkioOllj+nTgh/IeNuG5BT6UwwYfb7QVay1yO47XNFbYsj7eAki8kYthF6gmRH7iT mnV4PK/j5XSxB7BenrQ/ABh7/xBGJc27dCgPGDQk29yGg6KbQi0px2Pw26I1bwbpzjdZz08c yh8NO9dMY0RTurzUtcXH3ly1fPAHiZb1GBF7UDzmamU3KMtR6+LQbT+4epNPpMWcJOAbu7Je sVe4wqazz0Ngoea3P+LE6x0D3z4B1yX+hJz2ERc0LX4iZ7MakicQ+DlI2h74mNEIG+nguP72 0BCJJhXSVMuHJV6jFmDAEIGnZWsLkgeuhDXTZiTxgrjkDZghSu4XhP6IOdbPuSxEhYmr/srH XEaqk9XyjTYmUOUswtM0sHeSXGNRPyTgeUPlWS2LLAAAAAAAAA== --------------ms050404080707080201050301-- From owner-freebsd-stable@freebsd.org Tue Apr 30 13:38:47 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCE96159164A for ; Tue, 30 Apr 2019 13:38:47 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 4173F714DB for ; Tue, 30 Apr 2019 13:38:47 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from isux.com (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQS002VU17JY400@hades.sorbs.net> for freebsd-stable@freebsd.org; Tue, 30 Apr 2019 06:52:33 -0700 (PDT) Subject: Re: ZFS... To: Karl Denninger , freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> From: Michelle Sullivan Message-id: Date: Tue, 30 Apr 2019 23:38:34 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-reply-to: X-Rspamd-Queue-Id: 4173F714DB X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.06 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.82)[ip: (-2.18), ipnet: 72.12.192.0/19(-1.04), asn: 11114(-0.80), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.99)[-0.987,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; CTE_CASE(0.50)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 13:38:48 -0000 Karl Denninger wrote: > On 4/30/2019 03:09, Michelle Sullivan wrote: >> Consider.. >> >> If one triggers such a fault on a production server, how can one justify transferring from backup multiple terabytes (or even petabytes now) of data to repair an unmountable/faulted array.... because all backup solutions I know currently would take days if not weeks to restore the sort of store ZFS is touted with supporting. > Had it happen on a production server a few years back with ZFS. The > *hardware* went insane (disk adapter) and scribbled on *all* of the vdevs. > > The machine crashed and would not come back up -- at all. I insist on > (and had) emergency boot media physically in the box (a USB key) in any > production machine and it was quite-quickly obvious that all of the > vdevs were corrupted beyond repair. There was no rational option other > than to restore. > > It was definitely not a pleasant experience, but this is why when you > get into systems and data store sizes where it's a five-alarm pain in > the neck you must figure out some sort of strategy that covers you 99% > of the time without a large amount of downtime involved, and in the 1% > case accept said downtime. In this particular circumstance the customer > didn't want to spend on a doubled-and-transaction-level protected > on-site (in the same DC) redundancy setup originally so restore, as > opposed to fail-over/promote and then restore and build a new > "redundant" box where the old "primary" resided was the most-viable > option. Time to recover essential functions was ~8 hours (and over 24 > hours for everything to be restored.) > How big was the storage area? -- Michelle Sullivan http://www.mhix.org/ From owner-freebsd-stable@freebsd.org Tue Apr 30 13:53:01 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20BA31591E0A for ; Tue, 30 Apr 2019 13:53:01 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.21.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp-sofia.digsys.bg", Issuer "Digital Systems Operational CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9D0B372171 for ; Tue, 30 Apr 2019 13:52:53 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from [193.68.6.100] ([193.68.6.100]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.15.2/8.15.2) with ESMTPSA id x3UDe6rp061034 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Tue, 30 Apr 2019 16:40:07 +0300 (EEST) (envelope-from daniel@digsys.bg) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Daniel Kalchev In-Reply-To: Date: Tue, 30 Apr 2019 16:40:06 +0300 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <31095B59-630A-44E3-B68F-530C6A0AEC35@digsys.bg> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> To: Karl Denninger X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 9D0B372171 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of daniel@digsys.bg designates 193.68.21.123 as permitted sender) smtp.mailfrom=daniel@digsys.bg X-Spamd-Result: default: False [-1.28 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.90)[-0.904,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:193.68.21.123]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[digsys.bg]; NEURAL_HAM_LONG(-0.88)[-0.876,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[relay.digsys.bg,xt.digsys.bg]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.09)[asn: 3245(-0.48), country: BG(0.04)]; NEURAL_HAM_SHORT(-0.06)[-0.057,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3245, ipnet:193.68.0.0/19, country:BG]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 13:53:01 -0000 > On 30 Apr 2019, at 16:11, Karl Denninger wrote: >=20 >=20 > My experience is that ZFS is materially more-resilient but there is no > such thing as "can never be corrupted by any set of events." Backup > strategies for moderately large (e.g. many Terabytes) to very large > (e.g. Petabytes and beyond) get quite complex but they're also very > necessary. >=20 I can only second that statement. Being paranoid with your data (keep = many copies, have many backups) is never enough. A colleague just complained the other day, that they lost a zpool and = that ZFS didn=E2=80=99t save their data=E2=80=A6. by not making a = redundant pool and the hard drive trashing heads. And no backups. The = unreadable part of the drive happened in metadata and the pool can not = be imported. I keep an HDD around, that since it was brand new, runs perfectly under = any OS. Rock solid, that is=E2=80=A6 and only ZFS complains that it = reads things back it didn=E2=80=99t write. Before that, I would think = UFS was ok=E2=80=A6 since then, I don=E2=80=99t build a single = installation, that does not have at least a mirrored ZFS pool. And = =E2=80=9Carchive servers=E2=80=9D (stands for backup) have become the = central focus of my work. These are never enough.. Daniel From owner-freebsd-stable@freebsd.org Tue Apr 30 14:02:17 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 77C8015924D4 for ; Tue, 30 Apr 2019 14:02:17 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf1-f68.google.com (mail-lf1-f68.google.com [209.85.167.68]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A82E7285F for ; Tue, 30 Apr 2019 14:02:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf1-f68.google.com with SMTP id v1so10953872lfg.5 for ; Tue, 30 Apr 2019 07:02:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=5Z06XhSQGYOzkYDTybJzyMHCdv4sg7c458iB9t4MccI=; b=ZIJtjaLtR/PM4UHa1xI26TNp/RDJl3zgAuje3/I8rjX38JPdB7q+XDAgqwuwYs/dzd C9dUF0TJgfn+hC5zu5BS7pcwMVndx7bXMZ71BH7jef5ugvcx0h9pNMPKYOhfyt/k9UDo XzDHhUOBoS6jjqrZ+KCdwKhpP2sPJ2hQX4d2nM7wDOzyaTR/PaTV0nE7gBbdWqZS4+wo NNwiSy8AIZKdWnovmOx5W0t7H0KjTrAwuxLPp4y0rxTAdrLPWRh7CPnkCWyeOpaoaiNQ utSsCeAFZFJSftW/ASTloBaRHcrLaog/u88PMq2HXbMN7/RvTXP/+ixC1dVZjybILbSY VsbQ== X-Gm-Message-State: APjAAAUuxOArzZIJo5bNqE5S1bHGOGZnmrIYho22O+jL0iZFQYjPFamp Fjb+uQN6T0NBJwrVraDEw5+/xaXxu0gZ61quPk7/mUNz X-Google-Smtp-Source: APXvYqxEBOHkspWcZkQgoUiNncsnR4xLwwReYGFrY7MdAD4eYKt+JKZ4sZPIqXXyYXq9VjXPX3mRO90Kpoky2+gk0R4= X-Received: by 2002:ac2:4301:: with SMTP id l1mr20384238lfh.54.1556632927762; Tue, 30 Apr 2019 07:02:07 -0700 (PDT) MIME-Version: 1.0 References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> In-Reply-To: <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> From: Alan Somers Date: Tue, 30 Apr 2019 08:01:55 -0600 Message-ID: Subject: Re: ZFS... To: Michelle Sullivan Cc: Karl Denninger , FreeBSD Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 3A82E7285F X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.68 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-3.63 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_SHORT(-0.84)[-0.843,0]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[68.167.85.209.list.dnswl.org : 127.0.5.0]; SUBJ_ALL_CAPS(0.45)[6]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[68.167.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; IP_SCORE(-1.23)[ipnet: 209.85.128.0/17(-3.86), asn: 15169(-2.24), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 14:02:17 -0000 On Tue, Apr 30, 2019 at 7:30 AM Michelle Sullivan wrot= e: > > Karl Denninger wrote: > > On 4/30/2019 05:14, Michelle Sullivan wrote: > >>> On 30 Apr 2019, at 19:50, Xin LI wrote: > >>>> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wrote: > >>>> but in my recent experience 2 issues colliding at the same time resu= lts in disaster > >>> Do we know exactly what kind of corruption happen to your pool? If y= ou see it twice in a row, it might suggest a software bug that should be in= vestigated. > >>> > >>> All I know is it=E2=80=99s a checksum error on a meta slab (122) and = from what I can gather it=E2=80=99s the spacemap that is corrupt... but I a= m no expert. I don=E2=80=99t believe it=E2=80=99s a software fault as such= , because this was cause by a hard outage (damaged UPSes) whilst resilverin= g a single (but completely failed) drive. ...and after the first outage a = second occurred (same as the first but more damaging to the power hardware)= ... the host itself was not damaged nor were the drives or controller. > > ..... > >>> Note that ZFS stores multiple copies of its essential metadata, and i= n my experience with my old, consumer grade crappy hardware (non-ECC RAM, w= ith several faulty, single hard drive pool: bad enough to crash almost mont= hly and damages my data from time to time), > >> This was a top end consumer grade mb with non ecc ram that had been ru= nning for 8+ years without fault (except for hard drive platter failures.).= Uptime would have been years if it wasn=E2=80=99t for patching. > > Yuck. > > > > I'm sorry, but that may well be what nailed you. > > > > ECC is not just about the random cosmic ray. It also saves your bacon > > when there are power glitches. > > No. Sorry no. If the data is only half to disk, ECC isn't going to save > you at all... it's all about power on the drives to complete the write. ECC RAM isn't about saving the last few seconds' worth of data from before a power crash. It's about not corrupting the data that gets written long before a crash. If you have non-ECC RAM, then a cosmic ray/alpha ray/row hammer attack/bad luck can corrupt data after it's been checksummed but before it gets DMAed to disk. Then disk will contain corrupt data and you won't know it until you try to read it back. -Alan > > > > Unfortunately however there is also cache memory on most modern hard > > drives, most of the time (unless you explicitly shut it off) it's on fo= r > > write caching, and it'll nail you too. Oh, and it's never, in my > > experience, ECC. Fortunately, ZFS never sends non-checksummed data to the hard drive. So an error in the hard drive's cache ram will usually get detected by the ZFS checksum. > > No comment on that - you're right in the first part, I can't comment if > there are drives with ECC. > > > > > In addition, however, and this is something I learned a LONG time ago > > (think Z-80 processors!) is that as in so many very important things > > "two is one and one is none." > > > > In other words without a backup you WILL lose data eventually, and it > > WILL be important. > > > > Raidz2 is very nice, but as the name implies it you have two > > redundancies. If you take three errors, or if, God forbid, you *write* > > a block that has a bad checksum in it because it got scrambled while in > > RAM, you're dead if that happens in the wrong place. > > Or in my case you write part data therefore invalidating the checksum... > > > >> Yeah.. unlike UFS that has to get really really hosed to restore from = backup with nothing recoverable it seems ZFS can get hosed where issues occ= ur in just the wrong bit... but mostly it is recoverable (and my experience= has been some nasty shit that always ended up being recoverable.) > >> > >> Michelle > > Oh that is definitely NOT true.... again, from hard experience, > > including (but not limited to) on FreeBSD. > > > > My experience is that ZFS is materially more-resilient but there is no > > such thing as "can never be corrupted by any set of events." > > The latter part is true - and my blog and my current situation is not > limited to or aimed at FreeBSD specifically, FreeBSD is my experience. > The former part... it has been very resilient, but I think (based on > this certain set of events) it is easily corruptible and I have just > been lucky. You just have to hit a certain write to activate the issue, > and whilst that write and issue might be very very difficult (read: hit > and miss) to hit in normal every day scenarios it can and will > eventually happen. > > > Backup > > strategies for moderately large (e.g. many Terabytes) to very large > > (e.g. Petabytes and beyond) get quite complex but they're also very > > necessary. > > > and there in lies the problem. If you don't have a many 10's of > thousands of dollars backup solutions, you're either: > > 1/ down for a looooong time. > 2/ losing all data and starting again... > > ..and that's the problem... ufs you can recover most (in most > situations) and providing the *data* is there uncorrupted by the fault > you can get it all off with various tools even if it is a complete > mess.... here I am with the data that is apparently ok, but the > metadata is corrupt (and note: as I had stopped writing to the drive > when it started resilvering the data - all of it - should be intact... > even if a mess.) > > Michelle > > -- > Michelle Sullivan > http://www.mhix.org/ > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Tue Apr 30 14:05:49 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2A851592676 for ; Tue, 30 Apr 2019 14:05:48 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id B3C0F72AD7; Tue, 30 Apr 2019 14:05:46 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQS002WZ2GQY400@hades.sorbs.net>; Tue, 30 Apr 2019 07:19:41 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Wed, 01 May 2019 00:05:41 +1000 Cc: Karl Denninger , FreeBSD Content-transfer-encoding: quoted-printable Message-id: <34539589-162B-4891-A68F-88F879B59650@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> To: Alan Somers X-Rspamd-Queue-Id: B3C0F72AD7 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.13 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; NEURAL_HAM_SHORT(-0.60)[-0.595,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.78)[ip: (-2.07), ipnet: 72.12.192.0/19(-1.00), asn: 11114(-0.76), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 14:05:49 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 01 May 2019, at 00:01, Alan Somers wrote: >=20 >> On Tue, Apr 30, 2019 at 7:30 AM Michelle Sullivan wr= ote: >>=20 >> Karl Denninger wrote: >>> On 4/30/2019 05:14, Michelle Sullivan wrote: >>>>>> On 30 Apr 2019, at 19:50, Xin LI wrote: >>>>>> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wrote: >>>>>> but in my recent experience 2 issues colliding at the same time resul= ts in disaster >>>>> Do we know exactly what kind of corruption happen to your pool? If yo= u see it twice in a row, it might suggest a software bug that should be inve= stigated. >>>>>=20 >>>>> All I know is it=E2=80=99s a checksum error on a meta slab (122) and f= rom what I can gather it=E2=80=99s the spacemap that is corrupt... but I am n= o expert. I don=E2=80=99t believe it=E2=80=99s a software fault as such, be= cause this was cause by a hard outage (damaged UPSes) whilst resilvering a s= ingle (but completely failed) drive. ...and after the first outage a second= occurred (same as the first but more damaging to the power hardware)... the= host itself was not damaged nor were the drives or controller. >>> ..... >>>>> Note that ZFS stores multiple copies of its essential metadata, and in= my experience with my old, consumer grade crappy hardware (non-ECC RAM, wit= h several faulty, single hard drive pool: bad enough to crash almost monthly= and damages my data from time to time), >>>> This was a top end consumer grade mb with non ecc ram that had been run= ning for 8+ years without fault (except for hard drive platter failures.). U= ptime would have been years if it wasn=E2=80=99t for patching. >>> Yuck. >>>=20 >>> I'm sorry, but that may well be what nailed you. >>>=20 >>> ECC is not just about the random cosmic ray. It also saves your bacon >>> when there are power glitches. >>=20 >> No. Sorry no. If the data is only half to disk, ECC isn't going to save >> you at all... it's all about power on the drives to complete the write. >=20 > ECC RAM isn't about saving the last few seconds' worth of data from > before a power crash. It's about not corrupting the data that gets > written long before a crash. If you have non-ECC RAM, then a cosmic > ray/alpha ray/row hammer attack/bad luck can corrupt data after it's > been checksummed but before it gets DMAed to disk. Then disk will > contain corrupt data and you won't know it until you try to read it > back. I know this... unless I misread Karl=E2=80=99s message he implied the ECC wo= uld have saved the corruption in the crash... which is patently false... I t= hink you=E2=80=99ll agree.. Michelle >=20 > -Alan >=20 >>>=20 >>> Unfortunately however there is also cache memory on most modern hard >>> drives, most of the time (unless you explicitly shut it off) it's on for= >>> write caching, and it'll nail you too. Oh, and it's never, in my >>> experience, ECC. >=20 > Fortunately, ZFS never sends non-checksummed data to the hard drive. > So an error in the hard drive's cache ram will usually get detected by > the ZFS checksum. >=20 >>=20 >> No comment on that - you're right in the first part, I can't comment if >> there are drives with ECC. >>=20 >>>=20 >>> In addition, however, and this is something I learned a LONG time ago >>> (think Z-80 processors!) is that as in so many very important things >>> "two is one and one is none." >>>=20 >>> In other words without a backup you WILL lose data eventually, and it >>> WILL be important. >>>=20 >>> Raidz2 is very nice, but as the name implies it you have two >>> redundancies. If you take three errors, or if, God forbid, you *write* >>> a block that has a bad checksum in it because it got scrambled while in >>> RAM, you're dead if that happens in the wrong place. >>=20 >> Or in my case you write part data therefore invalidating the checksum... >>>=20 >>>> Yeah.. unlike UFS that has to get really really hosed to restore from b= ackup with nothing recoverable it seems ZFS can get hosed where issues occur= in just the wrong bit... but mostly it is recoverable (and my experience ha= s been some nasty shit that always ended up being recoverable.) >>>>=20 >>>> Michelle >>> Oh that is definitely NOT true.... again, from hard experience, >>> including (but not limited to) on FreeBSD. >>>=20 >>> My experience is that ZFS is materially more-resilient but there is no >>> such thing as "can never be corrupted by any set of events." >>=20 >> The latter part is true - and my blog and my current situation is not >> limited to or aimed at FreeBSD specifically, FreeBSD is my experience. >> The former part... it has been very resilient, but I think (based on >> this certain set of events) it is easily corruptible and I have just >> been lucky. You just have to hit a certain write to activate the issue, >> and whilst that write and issue might be very very difficult (read: hit >> and miss) to hit in normal every day scenarios it can and will >> eventually happen. >>=20 >>> Backup >>> strategies for moderately large (e.g. many Terabytes) to very large >>> (e.g. Petabytes and beyond) get quite complex but they're also very >>> necessary. >>>=20 >> and there in lies the problem. If you don't have a many 10's of >> thousands of dollars backup solutions, you're either: >>=20 >> 1/ down for a looooong time. >> 2/ losing all data and starting again... >>=20 >> ..and that's the problem... ufs you can recover most (in most >> situations) and providing the *data* is there uncorrupted by the fault >> you can get it all off with various tools even if it is a complete >> mess.... here I am with the data that is apparently ok, but the >> metadata is corrupt (and note: as I had stopped writing to the drive >> when it started resilvering the data - all of it - should be intact... >> even if a mess.) >>=20 >> Michelle >>=20 >> -- >> Michelle Sullivan >> http://www.mhix.org/ >>=20 >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"= From owner-freebsd-stable@freebsd.org Tue Apr 30 14:12:36 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86DE41592B91 for ; Tue, 30 Apr 2019 14:12:36 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf1-f68.google.com (mail-lf1-f68.google.com [209.85.167.68]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B76DE7306F for ; Tue, 30 Apr 2019 14:12:35 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf1-f68.google.com with SMTP id j11so11023110lfm.0 for ; Tue, 30 Apr 2019 07:12:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=8QjjSl9eEUAdwTjVUxd+F43h3w03QzC58axBLeNWB4U=; b=AQcZ08jp/bph9pIYYu+LffPairkHgZ6CNRxrYNDmLcokrzkyfRdDq64KlH9exazKu9 R+NSEbI4MB2s/voAK5X9ltcxr99dDR/JtahKIDuJ8+rH1sHeQqtFiB4ILvZZVJgltB6w Y5ceNI1dJDTOKQmE04g5Ljex3K775Xq+4hBC1i9+cQG8KSE1Zzy7TybRI6r9s1miDMl+ tfFEHJLedndhgwyQBUy8/qZyeGDnV7JitM9JmmfWRb/FmrSgAmxz8byDvFJY1UekqB/C imJH7EadPbG5C0aRNGeb0H6paXB+eU/SBKY7RZrY1fZUhqjqTnjbeWRvJP/lR5hsq9V4 X8sQ== X-Gm-Message-State: APjAAAWgDiT9CNWdVbjlC8V9ihAcDaFIfRQQ0pjEnK9Zclu23WiNUu1f SqOVSL//d6dGK+WG9NrOWFM+izeKRqTOu3YMWNk= X-Google-Smtp-Source: APXvYqz8y/1SOtTTmwB3aE0uvgGjUdXS3nxUafDmxn4iEGcywv3QwDCQy5QBmpqpIX7md75t32m8jJlscT9ci5k2DMQ= X-Received: by 2002:a05:6512:c8:: with SMTP id c8mr35851962lfp.138.1556633554055; Tue, 30 Apr 2019 07:12:34 -0700 (PDT) MIME-Version: 1.0 References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> In-Reply-To: <34539589-162B-4891-A68F-88F879B59650@sorbs.net> From: Alan Somers Date: Tue, 30 Apr 2019 08:12:22 -0600 Message-ID: Subject: Re: ZFS... To: Michelle Sullivan Cc: Karl Denninger , FreeBSD Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: B76DE7306F X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.68 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-3.63 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_SHORT(-0.84)[-0.843,0]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[68.167.85.209.list.dnswl.org : 127.0.5.0]; SUBJ_ALL_CAPS(0.45)[6]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[68.167.85.209.rep.mailspike.net : 127.0.0.17]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; IP_SCORE(-1.23)[ipnet: 209.85.128.0/17(-3.86), asn: 15169(-2.24), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 14:12:36 -0000 On Tue, Apr 30, 2019 at 8:05 AM Michelle Sullivan wrot= e: > > > > Michelle Sullivan > http://www.mhix.org/ > Sent from my iPad > > > On 01 May 2019, at 00:01, Alan Somers wrote: > > > >> On Tue, Apr 30, 2019 at 7:30 AM Michelle Sullivan = wrote: > >> > >> Karl Denninger wrote: > >>> On 4/30/2019 05:14, Michelle Sullivan wrote: > >>>>>> On 30 Apr 2019, at 19:50, Xin LI wrote: > >>>>>> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wrote: > >>>>>> but in my recent experience 2 issues colliding at the same time re= sults in disaster > >>>>> Do we know exactly what kind of corruption happen to your pool? If= you see it twice in a row, it might suggest a software bug that should be = investigated. > >>>>> > >>>>> All I know is it=E2=80=99s a checksum error on a meta slab (122) an= d from what I can gather it=E2=80=99s the spacemap that is corrupt... but I= am no expert. I don=E2=80=99t believe it=E2=80=99s a software fault as su= ch, because this was cause by a hard outage (damaged UPSes) whilst resilver= ing a single (but completely failed) drive. ...and after the first outage = a second occurred (same as the first but more damaging to the power hardwar= e)... the host itself was not damaged nor were the drives or controller. > >>> ..... > >>>>> Note that ZFS stores multiple copies of its essential metadata, and= in my experience with my old, consumer grade crappy hardware (non-ECC RAM,= with several faulty, single hard drive pool: bad enough to crash almost mo= nthly and damages my data from time to time), > >>>> This was a top end consumer grade mb with non ecc ram that had been = running for 8+ years without fault (except for hard drive platter failures.= ). Uptime would have been years if it wasn=E2=80=99t for patching. > >>> Yuck. > >>> > >>> I'm sorry, but that may well be what nailed you. > >>> > >>> ECC is not just about the random cosmic ray. It also saves your baco= n > >>> when there are power glitches. > >> > >> No. Sorry no. If the data is only half to disk, ECC isn't going to sa= ve > >> you at all... it's all about power on the drives to complete the write= . > > > > ECC RAM isn't about saving the last few seconds' worth of data from > > before a power crash. It's about not corrupting the data that gets > > written long before a crash. If you have non-ECC RAM, then a cosmic > > ray/alpha ray/row hammer attack/bad luck can corrupt data after it's > > been checksummed but before it gets DMAed to disk. Then disk will > > contain corrupt data and you won't know it until you try to read it > > back. > > I know this... unless I misread Karl=E2=80=99s message he implied the ECC= would have saved the corruption in the crash... which is patently false...= I think you=E2=80=99ll agree.. I don't think that's what Karl meant. I think he meant that the non-ECC RAM could've caused latent corruption that was only detected when the crash forced a reboot and resilver. > > Michelle > > > > > > -Alan > > > >>> > >>> Unfortunately however there is also cache memory on most modern hard > >>> drives, most of the time (unless you explicitly shut it off) it's on = for > >>> write caching, and it'll nail you too. Oh, and it's never, in my > >>> experience, ECC. > > > > Fortunately, ZFS never sends non-checksummed data to the hard drive. > > So an error in the hard drive's cache ram will usually get detected by > > the ZFS checksum. > > > >> > >> No comment on that - you're right in the first part, I can't comment i= f > >> there are drives with ECC. > >> > >>> > >>> In addition, however, and this is something I learned a LONG time ago > >>> (think Z-80 processors!) is that as in so many very important things > >>> "two is one and one is none." > >>> > >>> In other words without a backup you WILL lose data eventually, and it > >>> WILL be important. > >>> > >>> Raidz2 is very nice, but as the name implies it you have two > >>> redundancies. If you take three errors, or if, God forbid, you *writ= e* > >>> a block that has a bad checksum in it because it got scrambled while = in > >>> RAM, you're dead if that happens in the wrong place. > >> > >> Or in my case you write part data therefore invalidating the checksum.= .. > >>> > >>>> Yeah.. unlike UFS that has to get really really hosed to restore fro= m backup with nothing recoverable it seems ZFS can get hosed where issues o= ccur in just the wrong bit... but mostly it is recoverable (and my experien= ce has been some nasty shit that always ended up being recoverable.) > >>>> > >>>> Michelle > >>> Oh that is definitely NOT true.... again, from hard experience, > >>> including (but not limited to) on FreeBSD. > >>> > >>> My experience is that ZFS is materially more-resilient but there is n= o > >>> such thing as "can never be corrupted by any set of events." > >> > >> The latter part is true - and my blog and my current situation is not > >> limited to or aimed at FreeBSD specifically, FreeBSD is my experience= . > >> The former part... it has been very resilient, but I think (based on > >> this certain set of events) it is easily corruptible and I have just > >> been lucky. You just have to hit a certain write to activate the issu= e, > >> and whilst that write and issue might be very very difficult (read: hi= t > >> and miss) to hit in normal every day scenarios it can and will > >> eventually happen. > >> > >>> Backup > >>> strategies for moderately large (e.g. many Terabytes) to very large > >>> (e.g. Petabytes and beyond) get quite complex but they're also very > >>> necessary. > >>> > >> and there in lies the problem. If you don't have a many 10's of > >> thousands of dollars backup solutions, you're either: > >> > >> 1/ down for a looooong time. > >> 2/ losing all data and starting again... > >> > >> ..and that's the problem... ufs you can recover most (in most > >> situations) and providing the *data* is there uncorrupted by the fault > >> you can get it all off with various tools even if it is a complete > >> mess.... here I am with the data that is apparently ok, but the > >> metadata is corrupt (and note: as I had stopped writing to the drive > >> when it started resilvering the data - all of it - should be intact... > >> even if a mess.) > >> > >> Michelle > >> > >> -- > >> Michelle Sullivan > >> http://www.mhix.org/ > >> > >> _______________________________________________ > >> freebsd-stable@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.o= rg" From owner-freebsd-stable@freebsd.org Tue Apr 30 14:12:58 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C54DE1592BBB for ; Tue, 30 Apr 2019 14:12:58 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 3C0FD73096; Tue, 30 Apr 2019 14:12:58 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQS002XD2SQY400@hades.sorbs.net>; Tue, 30 Apr 2019 07:26:53 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Wed, 01 May 2019 00:12:54 +1000 Cc: Karl Denninger , FreeBSD Content-transfer-encoding: quoted-printable Message-id: <6E471004-C8BC-4E9C-83B1-A9BECC0A54C4@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> To: Alan Somers X-Rspamd-Queue-Id: 3C0FD73096 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.05 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; NEURAL_HAM_SHORT(-0.55)[-0.545,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.75)[ip: (-1.98), ipnet: 72.12.192.0/19(-0.96), asn: 11114(-0.74), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 14:12:58 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 01 May 2019, at 00:01, Alan Somers wrote: >=20 >>>=20 >>> Unfortunately however there is also cache memory on most modern hard >>> drives, most of the time (unless you explicitly shut it off) it's on for= >>> write caching, and it'll nail you too. Oh, and it's never, in my >>> experience, ECC. >=20 > Fortunately, ZFS never sends non-checksummed data to the hard drive. > So an error in the hard drive's cache ram will usually get detected by > the ZFS checksum. True, but a drive losing power mid write will ensure the checksum doesn=E2=80= =99t match the data (even if it is written before the data)... you need to e= nsure all the data and the checksum is written before drive power down.. and= in the event of unexpected hard power fail, you can=E2=80=99t guarantee thi= s. Battery backup in the controller that has a write cache and re-writes th= e last few writes on power restore on the otherhand will save you.. which is= why the other machine at my disposal hasn=E2=80=99t failed to date. Michelle= From owner-freebsd-stable@freebsd.org Tue Apr 30 14:15:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C4D31592D13 for ; Tue, 30 Apr 2019 14:15:53 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 8184F73298; Tue, 30 Apr 2019 14:15:52 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQS002XI2XJY400@hades.sorbs.net>; Tue, 30 Apr 2019 07:29:47 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Wed, 01 May 2019 00:15:47 +1000 Cc: Karl Denninger , FreeBSD Content-transfer-encoding: quoted-printable Message-id: <9F250929-BAA1-4A18-9025-06F3EC13CD42@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> To: Alan Somers X-Rspamd-Queue-Id: 8184F73298 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.04 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; NEURAL_HAM_SHORT(-0.56)[-0.564,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.72)[ip: (-1.89), ipnet: 72.12.192.0/19(-0.92), asn: 11114(-0.71), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 14:15:53 -0000 This issue is definitely related to sudden unexpected loss of power during r= esilver.. not ECC/non-ECC issues. Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 01 May 2019, at 00:12, Alan Somers wrote: >=20 >> On Tue, Apr 30, 2019 at 8:05 AM Michelle Sullivan wr= ote: >>=20 >>=20 >>=20 >> Michelle Sullivan >> http://www.mhix.org/ >> Sent from my iPad >>=20 >>>> On 01 May 2019, at 00:01, Alan Somers wrote: >>>>=20 >>>> On Tue, Apr 30, 2019 at 7:30 AM Michelle Sullivan w= rote: >>>>=20 >>>> Karl Denninger wrote: >>>>> On 4/30/2019 05:14, Michelle Sullivan wrote: >>>>>>>> On 30 Apr 2019, at 19:50, Xin LI wrote: >>>>>>>> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan wrote: >>>>>>>> but in my recent experience 2 issues colliding at the same time res= ults in disaster >>>>>>> Do we know exactly what kind of corruption happen to your pool? If y= ou see it twice in a row, it might suggest a software bug that should be inv= estigated. >>>>>>>=20 >>>>>>> All I know is it=E2=80=99s a checksum error on a meta slab (122) and= from what I can gather it=E2=80=99s the spacemap that is corrupt... but I a= m no expert. I don=E2=80=99t believe it=E2=80=99s a software fault as such,= because this was cause by a hard outage (damaged UPSes) whilst resilvering a= single (but completely failed) drive. ...and after the first outage a seco= nd occurred (same as the first but more damaging to the power hardware)... t= he host itself was not damaged nor were the drives or controller. >>>>> ..... >>>>>>> Note that ZFS stores multiple copies of its essential metadata, and i= n my experience with my old, consumer grade crappy hardware (non-ECC RAM, wi= th several faulty, single hard drive pool: bad enough to crash almost monthl= y and damages my data from time to time), >>>>>> This was a top end consumer grade mb with non ecc ram that had been r= unning for 8+ years without fault (except for hard drive platter failures.).= Uptime would have been years if it wasn=E2=80=99t for patching. >>>>> Yuck. >>>>>=20 >>>>> I'm sorry, but that may well be what nailed you. >>>>>=20 >>>>> ECC is not just about the random cosmic ray. It also saves your bacon= >>>>> when there are power glitches. >>>>=20 >>>> No. Sorry no. If the data is only half to disk, ECC isn't going to sav= e >>>> you at all... it's all about power on the drives to complete the write.= >>>=20 >>> ECC RAM isn't about saving the last few seconds' worth of data from >>> before a power crash. It's about not corrupting the data that gets >>> written long before a crash. If you have non-ECC RAM, then a cosmic >>> ray/alpha ray/row hammer attack/bad luck can corrupt data after it's >>> been checksummed but before it gets DMAed to disk. Then disk will >>> contain corrupt data and you won't know it until you try to read it >>> back. >>=20 >> I know this... unless I misread Karl=E2=80=99s message he implied the ECC= would have saved the corruption in the crash... which is patently false... I= think you=E2=80=99ll agree.. >=20 > I don't think that's what Karl meant. I think he meant that the > non-ECC RAM could've caused latent corruption that was only detected > when the crash forced a reboot and resilver. >=20 >>=20 >> Michelle >>=20 >>=20 >>>=20 >>> -Alan >>>=20 >>>>>=20 >>>>> Unfortunately however there is also cache memory on most modern hard >>>>> drives, most of the time (unless you explicitly shut it off) it's on f= or >>>>> write caching, and it'll nail you too. Oh, and it's never, in my >>>>> experience, ECC. >>>=20 >>> Fortunately, ZFS never sends non-checksummed data to the hard drive. >>> So an error in the hard drive's cache ram will usually get detected by >>> the ZFS checksum. >>>=20 >>>>=20 >>>> No comment on that - you're right in the first part, I can't comment if= >>>> there are drives with ECC. >>>>=20 >>>>>=20 >>>>> In addition, however, and this is something I learned a LONG time ago >>>>> (think Z-80 processors!) is that as in so many very important things >>>>> "two is one and one is none." >>>>>=20 >>>>> In other words without a backup you WILL lose data eventually, and it >>>>> WILL be important. >>>>>=20 >>>>> Raidz2 is very nice, but as the name implies it you have two >>>>> redundancies. If you take three errors, or if, God forbid, you *write= * >>>>> a block that has a bad checksum in it because it got scrambled while i= n >>>>> RAM, you're dead if that happens in the wrong place. >>>>=20 >>>> Or in my case you write part data therefore invalidating the checksum..= . >>>>>=20 >>>>>> Yeah.. unlike UFS that has to get really really hosed to restore from= backup with nothing recoverable it seems ZFS can get hosed where issues occ= ur in just the wrong bit... but mostly it is recoverable (and my experience h= as been some nasty shit that always ended up being recoverable.) >>>>>>=20 >>>>>> Michelle >>>>> Oh that is definitely NOT true.... again, from hard experience, >>>>> including (but not limited to) on FreeBSD. >>>>>=20 >>>>> My experience is that ZFS is materially more-resilient but there is no= >>>>> such thing as "can never be corrupted by any set of events." >>>>=20 >>>> The latter part is true - and my blog and my current situation is not >>>> limited to or aimed at FreeBSD specifically, FreeBSD is my experience.= >>>> The former part... it has been very resilient, but I think (based on >>>> this certain set of events) it is easily corruptible and I have just >>>> been lucky. You just have to hit a certain write to activate the issue= , >>>> and whilst that write and issue might be very very difficult (read: hit= >>>> and miss) to hit in normal every day scenarios it can and will >>>> eventually happen. >>>>=20 >>>>> Backup >>>>> strategies for moderately large (e.g. many Terabytes) to very large >>>>> (e.g. Petabytes and beyond) get quite complex but they're also very >>>>> necessary. >>>>>=20 >>>> and there in lies the problem. If you don't have a many 10's of >>>> thousands of dollars backup solutions, you're either: >>>>=20 >>>> 1/ down for a looooong time. >>>> 2/ losing all data and starting again... >>>>=20 >>>> ..and that's the problem... ufs you can recover most (in most >>>> situations) and providing the *data* is there uncorrupted by the fault >>>> you can get it all off with various tools even if it is a complete >>>> mess.... here I am with the data that is apparently ok, but the >>>> metadata is corrupt (and note: as I had stopped writing to the drive >>>> when it started resilvering the data - all of it - should be intact... >>>> even if a mess.) >>>>=20 >>>> Michelle >>>>=20 >>>> -- >>>> Michelle Sullivan >>>> http://www.mhix.org/ >>>>=20 >>>> _______________________________________________ >>>> freebsd-stable@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g" From owner-freebsd-stable@freebsd.org Tue Apr 30 14:27:20 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17B47159342C for ; Tue, 30 Apr 2019 14:27:20 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf1-f47.google.com (mail-lf1-f47.google.com [209.85.167.47]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1329674217 for ; Tue, 30 Apr 2019 14:27:18 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf1-f47.google.com with SMTP id j20so11062094lfh.2 for ; Tue, 30 Apr 2019 07:27:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=xNTasiUZ+xOK432w62CXPJ7ZmpgP820CHzYzo0YOKqY=; b=VdOGM3wH1eyatLuLFlShpd7G5oUzy1UWnudxRHDNVMMzDEpkhozPVrOhARtpsfh/Ca Xi1wNbU8frsNd2s/C5NDmxoj2PjmO3lurgUgvWjBqN8+DNDVsEN1/76Cgr0Hl7PMEzNW oX7vQfJVXQ1Vss1BWYOFiZ5AaQIQNaHYSAeL137YG49BsHYBGOxfo9zSku86nNnNsBIt nlhtPLEhG1z8wDOS2CbIDdOwoesCcRbN1zw6rfYKvf4uniZCWGeFemS/5j6aekdljKRT MIRCoAzQaS3BwNXQo6yRfspSbmje+Fqi9b6aWRiaNWZ5SrEoEzl7yZcfK3CC7lZdI1LI 9tFQ== X-Gm-Message-State: APjAAAWt7185M30qUQAbZqKM7ICKGF1mqRfAXn9MnZR8hbCOP7tElqo8 0Avs2c0ad6O3GcgyclMsbvnZtksheozTbrS3fgs= X-Google-Smtp-Source: APXvYqxUrM3Lrg6vCpoYCm5pEicgji+723IJEb4h4r6arxrGwmRGnuNI14H0bksB0jj6w9Dn/rnqyxV28e7fBcz1Z8o= X-Received: by 2002:ac2:51a1:: with SMTP id f1mr26092786lfk.129.1556634436779; Tue, 30 Apr 2019 07:27:16 -0700 (PDT) MIME-Version: 1.0 References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <6E471004-C8BC-4E9C-83B1-A9BECC0A54C4@sorbs.net> In-Reply-To: <6E471004-C8BC-4E9C-83B1-A9BECC0A54C4@sorbs.net> From: Alan Somers Date: Tue, 30 Apr 2019 08:27:04 -0600 Message-ID: Subject: Re: ZFS... To: Michelle Sullivan Cc: Karl Denninger , FreeBSD Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 1329674217 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.47 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-3.88 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_HAM_SHORT(-0.98)[-0.975,0]; MIME_TRACE(0.00)[0:+]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[47.167.85.209.list.dnswl.org : 127.0.5.0]; SUBJ_ALL_CAPS(0.45)[6]; RCVD_TLS_LAST(0.00)[]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; IP_SCORE(-1.34)[ip: (-0.56), ipnet: 209.85.128.0/17(-3.85), asn: 15169(-2.24), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 14:27:20 -0000 On Tue, Apr 30, 2019 at 8:13 AM Michelle Sullivan wrot= e: > > > > Michelle Sullivan > http://www.mhix.org/ > Sent from my iPad > > > On 01 May 2019, at 00:01, Alan Somers wrote: > > > >>> > >>> Unfortunately however there is also cache memory on most modern hard > >>> drives, most of the time (unless you explicitly shut it off) it's on = for > >>> write caching, and it'll nail you too. Oh, and it's never, in my > >>> experience, ECC. > > > > Fortunately, ZFS never sends non-checksummed data to the hard drive. > > So an error in the hard drive's cache ram will usually get detected by > > the ZFS checksum. > > True, but a drive losing power mid write will ensure the checksum doesn= =E2=80=99t match the data (even if it is written before the data)... you ne= ed to ensure all the data and the checksum is written before drive power do= wn.. and in the event of unexpected hard power fail, you can=E2=80=99t guar= antee this. Battery backup in the controller that has a write cache and re= -writes the last few writes on power restore on the otherhand will save you= .. which is why the other machine at my disposal hasn=E2=80=99t failed to d= ate. No, ZFS was designed from the start to handle exactly that problem. It's solved by the transaction system. It works like this: Syncing a transaction group =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1) Write everything except the labels 2) Write labels 1 and 3 3) send FLUSH_CACHE_EXT, which synchronously flushes the HDD's write cache 4) Write labels 2 and 4 5) send FLUSH_CACHE_EXT again Importing a pool =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1) Interrogate all labels. Use the most recent label found whose embedded checksum is correct. Power loss possibilities =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 1) If power is lost during step 1 above, then on import ZFS will roll back to the last txg, usually < 5 seconds prior 2, 3) If power is lost during step 2 or 3 and labels 1 and 3 are corrupted, then ZFS will roll back to the last txg as stored in labels 2 and 4 4, 5) If power is lost during step 4 or 5 then labels 1 and 3 will be correct, and ZFS will use them. -Alan > > Michelle From owner-freebsd-stable@freebsd.org Tue Apr 30 14:28:46 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B39C715934E3; Tue, 30 Apr 2019 14:28:46 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gromit.dlib.vt.edu", Issuer "Chumby Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id BD898742E4; Tue, 30 Apr 2019 14:28:45 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from mather.gromit23.net (c-98-244-101-97.hsd1.va.comcast.net [98.244.101.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id 20CF727A; Tue, 30 Apr 2019 10:28:37 -0400 (EDT) Content-Type: text/plain; charset=utf-8; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Paul Mather In-Reply-To: <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> Date: Tue, 30 Apr 2019 10:28:36 -0400 Cc: rainer@ultra-secure.de, owner-freebsd-stable@freebsd.org, freebsd-stable , Andrea Venturoli Content-Transfer-Encoding: 8bit Message-Id: References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> To: Michelle Sullivan X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: BD898742E4 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dmarc=fail reason="" header.from=vt.edu (policy=none) X-Spamd-Result: default: False [-3.19 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; IP_SCORE(-1.19)[ip: (-3.07), ipnet: 128.173.0.0/16(-1.53), asn: 1312(-1.30), country: US(-0.06)]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[vt.edu : No valid SPF, No valid DKIM,none]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[chumby.dlib.vt.edu,gromit.dlib.vt.edu]; NEURAL_HAM_SHORT(-0.94)[-0.943,0]; SUBJ_ALL_CAPS(0.45)[6]; R_SPF_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[97.101.244.98.zen.spamhaus.org : 127.0.0.10]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1312, ipnet:128.173.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 14:28:46 -0000 On Apr 30, 2019, at 5:05 AM, Michelle Sullivan wrote: > > > Michelle Sullivan > http://www.mhix.org/ > Sent from my iPad > >> On 30 Apr 2019, at 18:44, rainer@ultra-secure.de wrote: >> >> Am 2019-04-30 10:09, schrieb Michelle Sullivan: >> >>> Now, yes most production environments have multiple backing stores so >>> will have a server or ten to switch to whilst the store is being >>> recovered, but it still wouldn’t be a pleasant experience... not to >>> mention the possibility that if one store is corrupted there is a >>> chance that the other store(s) would also be affected in the same way >>> if in the same DC... (Eg a DC fire - which I have seen) .. and if you >>> have multi DC stores to protect from that.. size of the pipes between >>> DCs comes clearly into play. >> >> >> I have one customer with about 13T of ZFS - and because it would take a >> while to restore (actual backups), it zfs-sends delta-snapshots every >> hour to a standby-system. >> >> It was handy when we had to rebuild the system with different HBAs. > > I wonder what would happen if you scaled that up by just 10 (storage) and > had the master blow up where it needs to be restored from backup.. how > long would one be praying to higher powers that there is no problem with > the backup...? (As in no outage or error causing a complete outAge.)... > don’t get me wrong.. we all get to that position at sometime, but in my > recent experience 2 issues colliding at the same time results in > disaster. 13T is really not something I have issues with as I can > usually cobble something together with 16T.. (at least until 6T drives > became a viable (cost and availability at short notice) option... even > 10T is becoming easier to get a hold of now.. but I have a measly 96T > here and it takes weeks even with gigabit bonded interfaces when I need > to restore. Such is the curse of large-scale storage when disaster befalls it. I guess you need to invent a home brew version of Amazon Snowball or Amazon Snowmobile. ;-) Cheers, Paul. From owner-freebsd-stable@freebsd.org Tue Apr 30 14:44:22 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3AAE1593F86 for ; Tue, 30 Apr 2019 14:44:22 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C8BC3751AE for ; Tue, 30 Apr 2019 14:44:21 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 9E87A21109F for ; Tue, 30 Apr 2019 10:43:50 -0400 (EDT) Received: from [192.168.10.16] (D6.Denninger.Net [192.168.10.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id CCFBCC6DBB for ; Tue, 30 Apr 2019 09:43:49 -0500 (CDT) Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <65b175f7-40ee-0902-564d-f72a70ce9c35@denninger.net> Date: Tue, 30 Apr 2019 09:43:49 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms070107030705050701070504" X-Rspamd-Queue-Id: C8BC3751AE X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.24 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.96)[-0.961,0]; SUBJ_ALL_CAPS(0.45)[6]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.52)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.12), asn: 14061(1.48), country: US(-0.06)]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 14:44:23 -0000 This is a cryptographically signed message in MIME format. --------------ms070107030705050701070504 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 4/30/2019 08:38, Michelle Sullivan wrote: > Karl Denninger wrote: >> On 4/30/2019 03:09, Michelle Sullivan wrote: >>> Consider.. >>> >>> If one triggers such a fault on a production server, how can one >>> justify transferring from backup multiple terabytes (or even >>> petabytes now) of data to repair an unmountable/faulted array.... >>> because all backup solutions I know currently would take days if not >>> weeks to restore the sort of store ZFS is touted with supporting. >> Had it happen on a production server a few years back with ZFS.=C2=A0 = The >> *hardware* went insane (disk adapter) and scribbled on *all* of the >> vdevs. >> >> .... >> Time to recover essential functions was ~8 hours (and over 24 >> hours for everything to be restored.) >> > How big was the storage area? > In that specific instance approximately 10Tb in-use.=C2=A0 The working se= t that allowed critical functions to come online (and which got restored first, obviously) was ~3Tb. BTW my personal "primary server" working set is approximately 20Tb.=C2=A0= There is data on that server dating back to 1982 -- yes, data all the way back to systems I owned that ran on a Z-80 processor with 64Kb (not MB) of RAM.=C2=A0 I started running ZFS a fairly long time ago on FreeBSD= -- 9.0, I believe, and have reorganized and upgraded drives over time.=C2=A0= If my storage fails "hard" in a way that I have no local backups available (e.g. building fire, adapter scribbles on drives including not-mounted ones, etc) critical functionality (e.g. email receipt, etc) can be back online in roughly 3-4 hours, assuming the bank is open and I can get to the backup volumes.=C2=A0 A full restore will require more than a day.=C2= =A0 I've tested restore of each individual piece of the backup structure but do not have the hardware available in the building to restore a complete clone.=C2=A0 With the segregated structure of it, however, I'm 100% certa= in it is all restorable.=C2=A0 That's tested regularly -- just to be sure. Now if we get nuked and the bank vault is also destroyed then it's over, but then again I'm probably a burnt piece of charcoal in such a circumstance so that's a risk I accept. When I ran my ISP in the 1990s we had both local copies and vault copies because a "scribbles on disk" failure on a Saturday couldn't be unable to be addressed until Monday morning.=C2=A0 We would have been out of business instantly if that had happened in any situation short of the office with our primary data center burning down.=C2=A0 Incidentally one = of my adapter failures was in exactly the worst possible place for it to occur while running that company -- the adapter on the machine that held our primary authentication and billing database. At the time the best option for "large" working sets was DLT.=C2=A0 Now f= or most purposes it's another disk.=C2=A0 Disks, however, must be re-verifie= d more-frequently than DLT -- MUCH more frequently.=C2=A0 Further, if you h= ave only ONE backup then it cannot be singular (e.g. there must be two or more, whether via mirroring or some other mechanism) on ANY media.=C2=A0 While DLT, for example, has a typical expected 30 year archival life that doesn't mean the ONE tape you have will be readable 30 years later. As data size expands noodling on how you segregate data into read-only, write-very-occasionally and read/write, along with how you handle backups of each component and how, or if, you subdivide those categories for backup purposes becomes quite important.=C2=A0 If performance matters= (and it usually does) then what goes where in what pool (and across pools of similar base storage types) matters too; in my personal working set there is both SSD (all "power-pull safe" drives, which cost more and tend to be somewhat slower than "consumer" SSDs) and spinning rust storage for that exact reason. Note that on this list right now I'm chasing a potential "gotcha" interaction between geli and ZFS that thus far has eluded isolation.=C2=A0= While it has yet to corrupt data the potential is there and the hair on the back of my neck is standing up a bit as a consequence.=C2=A0 It appea= rs to date to either 11.2 or 12.0 and *definitely* is present in 12-STABLE; it was *not* present on 11.1. The price of keeping your data intact is always eternal vigilance. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms070107030705050701070504 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNDMwMTQ0MzQ5 WjBPBgkqhkiG9w0BCQQxQgRA36/p/C3ogxe9e3z+rOREi4Sn1hZQ2Ht/0ak/YncAYkWrpGri hlMD8AeNKZ7+VSJUPoYagXz+okGFkLMN/zU2rTBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgBexOOgq2Fhy8iwgemwsLpS5nvjkde7G14vr/edDQw0i1c7DCjL6FtU6nm9lh6b03Tp yOZULKpiGfZxuLoJX05S0uDcV1cYrjrpgzIb0oAdJ3tKsCbkQOU35F121dCJ8XuuHr78rS3O M3b7EGxeE1t8R7tn035vV9QneA4aFFx0AmwnHvJW9Xt1Vml4GGJXNNnf5Duv1sEFOMpDBsHR i4RV88PywWFir9tErsvDWuqfdtYZ03nhvpgj+drOjhks1JM8hQi2el3YWHxaj/0DlNv65y50 RcC1PkXtCro8W1LBHfdJFxkaiR4cS4I6bnrRqbksRq04bww1cXoa81wFOhWNRqmMyvQwtmGv BeAa1H+BXsNDGkEtaX+UIePesHuEGa8aOSLf1mwBEVaSRK96AYpUo5AW0L60uv+g2gjG5eE+ mzV7e4ggBfdvARJBTaYEMQbcKHMYFBqjyUhVG7khlh1qocKk3aWqaGQWnOYVNa2iUHHSTj67 UEJ6ZbzcE0rgjC1SK3aT0wosJMqg9IR92XHT0d6wq8Byyw7iIqP2dmk9cjeuovykD1CsLIB6 mowtsKdr7GIBoqOgHXaDJm0Bh9RhCO6KEIzgkb6nFS1jF8WkeNcF0Ztk6yas3/36jrvu5gKy DSMYiBkOqr/6j2keJVySGmGP0qHQ/pIG389Ste7A9QAAAAAAAA== --------------ms070107030705050701070504-- From owner-freebsd-stable@freebsd.org Tue Apr 30 15:03:05 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 402AD1594B69 for ; Tue, 30 Apr 2019 15:03:05 +0000 (UTC) (envelope-from emss.mail@gmail.com) Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0DE1D7632C for ; Tue, 30 Apr 2019 15:03:04 +0000 (UTC) (envelope-from emss.mail@gmail.com) Received: by mail-wm1-x334.google.com with SMTP id y5so4242840wma.2 for ; Tue, 30 Apr 2019 08:03:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-transfer-encoding; bh=E4UXm7Jgz/y/Ax7oun2Xb34Ie2tqOaLjKqO50lQi2po=; b=UGkjnV6YB8pjFkBMUbKttqwGx3TwvDnzxsXKzFfgB2gMW7woJOrO/AYmSiuDIcAZRP 2WZnB0cBpzgv0ZKYuQrqRuiEmhRBhXegNYFuZ+0kFo4BVFgEKuW214g3xzq9mutI/Kk+ HH4U+eJ+Tn6A9KZsx83LriAtR4DyPboc0EDOo1Acflpz1sfrwTGgm/WdcMlgg3J1YHOV UNEJ2H5tuBlv6OEc7xovsGvE0hzTz5gcjivOA43/ij2jAoQvgskbZRfZDBDJMku+T6+L EM9M63lM3uxpJOfGSV1rYrnSFvr7evQu2ysvFhdw1OrM+Y0JgA92EjPHOGHi31QHoQ4/ 0fag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:in-reply-to:references :user-agent:date:message-id:mime-version:content-transfer-encoding; bh=E4UXm7Jgz/y/Ax7oun2Xb34Ie2tqOaLjKqO50lQi2po=; b=MkmvGs3OawHIYOzb1G1HFNhHEmevs+p8hji7fO4YzVjwwGJ3NYeWwVl4Jym+D1YHY7 OBbCesb9JVIbPcu7Ec6+lZX1T3o1v8KC7qZwCc81kGhack3b7cMUwjL7MydDjxFspQwz MSsFzivhbvhrjSifCNYY+Sj6ZAAKWyfYJYIe6c4XFWZLkFJl7svKYGc+WcVtqPM7yZNq Y3lHTSmTZNrYtw47+qUvPnFfJIZTVfOitZCUIu8VPbcEYdhrExNo+KIqHw1uuI1kL6On nTHeUHCi1Jq4cmzVwhYJAUkBvMykH+JIyi9zHTYnm9MdyEBzs1U4XHjydleXt7bTs2Ul wbzQ== X-Gm-Message-State: APjAAAXvC/3LyhXUKo6K+ie1BiDmQqh9bmlpElstcdVDCHhtIhMryq0S eigXX4ylFHZTGzEjRMpPPkXlZZN/ X-Google-Smtp-Source: APXvYqyU2BVZ7TiAG6v+Lpii3uhrS3qWjHCjJWtsu0Uj7FTNPsOujYTDCuaqz6jpbH8JkoPMGl4QFg== X-Received: by 2002:a1c:9ad8:: with SMTP id c207mr3444333wme.109.1556636581945; Tue, 30 Apr 2019 08:03:01 -0700 (PDT) Received: from srvbsdfenssv.interne.associated-bears.org (lstlambert-658-1-110-48.w217-128.abo.wanadoo.fr. [217.128.200.48]) by smtp.gmail.com with ESMTPSA id 67sm4023768wmz.41.2019.04.30.08.02.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 30 Apr 2019 08:02:56 -0700 (PDT) Sender: Eric Masson Received: from srvbsdfenssv.interne.associated-bears.org (localhost [127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (Postfix) with ESMTP id C17CEFA for ; Tue, 30 Apr 2019 17:02:50 +0200 (CEST) X-Virus-Scanned: amavisd-new at interne.associated-bears.org Received: from srvbsdfenssv.interne.associated-bears.org ([127.0.0.1]) by srvbsdfenssv.interne.associated-bears.org (srvbsdfenssv.interne.associated-bears.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wifs8-lFTnR2 for ; Tue, 30 Apr 2019 17:02:49 +0200 (CEST) Received: by srvbsdfenssv.interne.associated-bears.org (Postfix, from userid 1001) id 19619F2; Tue, 30 Apr 2019 17:02:49 +0200 (CEST) From: Eric Masson To: Mailing List FreeBSD Stable Subject: Re: zpool dumps core In-Reply-To: <86sgtzvm5i.fsf@srvbsdfenssv.interne.associated-bears.org> (Eric Masson's message of "Tue, 30 Apr 2019 12:10:33 +0200") References: <86sgtzvm5i.fsf@srvbsdfenssv.interne.associated-bears.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (berkeley-unix) X-Operating-System: FreeBSD 11.2-STABLE amd64 Date: Tue, 30 Apr 2019 17:02:48 +0200 Message-ID: <86v9yv7cyv.fsf@srvbsdfenssv.interne.associated-bears.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 0DE1D7632C X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=UGkjnV6Y; spf=pass (mx1.freebsd.org: domain of emssmail@gmail.com designates 2a00:1450:4864:20::334 as permitted sender) smtp.mailfrom=emssmail@gmail.com X-Spamd-Result: default: False [-5.95 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[free.fr]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; FORGED_SENDER(0.30)[emss@free.fr,emssmail@gmail.com]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; TAGGED_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[free.fr]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.76)[ip: (-9.22), ipnet: 2a00:1450::/32(-2.27), asn: 15169(-2.24), country: US(-0.06)]; FROM_NEQ_ENVFROM(0.00)[emss@free.fr,emssmail@gmail.com]; RCVD_IN_DNSWL_NONE(0.00)[4.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 15:03:05 -0000 Eric Masson writes: Hi, > zpool dumps core on my box. This box is tracking 11-STABLE and I'm using the NO_CLEAN flag to speed up buildworld/buildkernel. Full source rebuild & install erasing /usr/obj solved the issue. Sorry for the noise. ric Masson From owner-freebsd-stable@freebsd.org Tue Apr 30 15:15:37 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D5761595235 for ; Tue, 30 Apr 2019 15:15:37 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B3A8476A5D for ; Tue, 30 Apr 2019 15:15:36 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id 35A6421109F for ; Tue, 30 Apr 2019 11:15:35 -0400 (EDT) Received: from [192.168.10.16] (D6.Denninger.Net [192.168.10.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id B804EC6F21 for ; Tue, 30 Apr 2019 10:15:34 -0500 (CDT) Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> Date: Tue, 30 Apr 2019 10:15:34 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms010905040502030301010707" X-Rspamd-Queue-Id: B3A8476A5D X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.24 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.97)[-0.967,0]; SUBJ_ALL_CAPS(0.45)[6]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.52)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.13), asn: 14061(1.48), country: US(-0.06)]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 15:15:37 -0000 This is a cryptographically signed message in MIME format. --------------ms010905040502030301010707 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 4/30/2019 09:12, Alan Somers wrote: > On Tue, Apr 30, 2019 at 8:05 AM Michelle Sullivan = wrote: > . >> I know this... unless I misread Karl=E2=80=99s message he implied the = ECC would have saved the corruption in the crash... which is patently fal= se... I think you=E2=80=99ll agree.. > I don't think that's what Karl meant. I think he meant that the > non-ECC RAM could've caused latent corruption that was only detected > when the crash forced a reboot and resilver. Exactly. Non-ECC memory means you can potentially write data to *all* copies of a block (and its parity in the case of a Raidz) where the checksum is invalid and there is no way for the code to know it happened or defend against it.=C2=A0 Unfortunately since the checksum is very small compared= to the data size the odds are that IF that happens it's the *data* and not the checksum that's bad and there are *no* good copies. Contrary to popular belief the=C2=A0 "power good" signal on your PSU and = MB do not provide 100% protection against transient power problems causing this to occur with non-ECC memory either. IMHO non-ECC memory systems are ok for personal desktop and laptop machines where loss of stored data requiring a restore is acceptable (assuming you have a reasonable backup paradigm for same) but not for servers and *especially* not for ZFS storage.=C2=A0 I don't like the pric= e of ECC memory and I really don't like Intel's practices when it comes to only enabling ECC RAM on their "server" class line of CPUs either but it is what it is.=C2=A0 Pay up for the machines where it matters. One of the ironies is that there's better data *integrity* with ZFS than other filesystems in this circumstance; you're much more-likely to *know* you're hosed even if the situation is unrecoverable and requires a restore.=C2=A0 With UFS and other filesystems you can quite-easily wind= up with silent corruption that can go undetected; the filesystem "works" just fine but the data is garbage.=C2=A0 From my point of view that's *mu= ch* worse. In addition IMHO consumer drives are not exactly safe for online ZFS storage.=C2=A0 Ironically they're *safer* for archival use because when n= ot actively in use they're dismounted and thus not subject to "you're silently hosed" sort of failures.=C2=A0 What sort of "you're hosed" failures?=C2=A0 Oh, for example, claiming to have flushed their cache buf= fers before returning "complete" on that request when they really did not!=C2=A0= In combination with write re-ordering that can *really* screw you and there's nothing that any filesystem can defensively do about it either.=C2= =A0 This sort of "cheat" is much-more likely to be present in consumer drives than ones sold for either enterprise or NAS purposes and it's quite difficult to accurately test for this sort of thing on an individual basis too. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms010905040502030301010707 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNDMwMTUxNTM0 WjBPBgkqhkiG9w0BCQQxQgRAPKK9tNsOwnk1RpaC79Xnf1kXYAsIUHdSVvA7TbAXN+nUr6kI nwtBkGHWbdqX34r3fN/yNeSpcDRZtEg4TEaBdTBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCb3vrMU08Bl13yPaqF5p7TQ1HY1Qk27qbVJ1/CWNJhhqM3PRJ6gNDEK7aBLdwX3kL+ JIpRHRQgwgAi+HzvMsW3nRO9k9TOi/P92o4TlYno2ktdhiD7Nk3j5sTKM6jr7xBVVlHX4xJL UCONnOYkTsImcsRmF0IhHGwGL+1nxNcqs5j+/LHCF1/3zASVo6EEXTA88tv66cWXLd2SN97S Gq8eWnyPGNeFhLy8+IF/+/I81D6mV4ZQCZA4HFEU8S6GF8g15385MBBRbaRTKyHDSfudRoAr qZCUJxNbPWL+h4nbcJGoilFSJop43PkHzPHirfbO+7ehUp3o5yk8jlF60nxe+5RRjNQLqoCZ cpDngT/pOZjc5xioi6V/5AfP8/so3A07SXpRfKxP2vIhBSWD4UeP8sp3yjeeqkZmgJmrfk4k MxeiZ6WqPLQcdPeV4jTCwwklzZF1zJ4p1Kvp4JpzzIFCG1dWUdBT2kRJ4OzTsIKn18eZvFJZ 6/UU7OaK4y6+0samesa6uIxSbBjPrmTZ0bZv0frl3OzI6qae1jPd/UKoMtfJW94w4zE9+Yi1 iCcbkurgH8LIyWBBk23ACV+BdTZisThhqJis7lNpCres8jWYZZpykkfWJuLGYrRZQrOFJpT2 pI4yalC9s5dAbCmxEf8Vm3CPwJXT0VZhFQIQku8AUgAAAAAAAA== --------------ms010905040502030301010707-- From owner-freebsd-stable@freebsd.org Tue Apr 30 15:37:31 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A0271595972 for ; Tue, 30 Apr 2019 15:37:31 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CF8FD77D44 for ; Tue, 30 Apr 2019 15:37:29 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.250] (unknown [192.148.167.11]) by proxypop01.sare.net (Postfix) with ESMTPA id 0274D9DF4F2; Tue, 30 Apr 2019 17:37:19 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Borja Marcos In-Reply-To: <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> Date: Tue, 30 Apr 2019 17:37:17 +0200 Cc: Karl Denninger , freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <75A78DAC-DF85-481B-ABC5-70E5E3960341@sarenet.es> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> To: Michelle Sullivan X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: CF8FD77D44 X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.57 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:195.16.151.0/24]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.sarenet.es,smtp.sarenet.es,smtp.sarenet.es]; DMARC_POLICY_ALLOW(-0.50)[sarenet.es,reject]; RCVD_IN_DNSWL_NONE(0.00)[151.151.16.195.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-2.72)[ip: (-7.54), ipnet: 195.16.128.0/19(-3.49), asn: 3262(-2.59), country: ES(0.04)]; NEURAL_HAM_SHORT(-1.00)[-0.995,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 15:37:31 -0000 > On 30 Apr 2019, at 15:30, Michelle Sullivan = wrote: >=20 >> I'm sorry, but that may well be what nailed you. >>=20 >> ECC is not just about the random cosmic ray. It also saves your = bacon >> when there are power glitches. >=20 > No. Sorry no. If the data is only half to disk, ECC isn't going to = save you at all... it's all about power on the drives to complete the = write. Not necessarily. Depending on the power outage things can get really = funny during the power loss event. 25+ years ago I witnessed a severe 2 second voltage drop and during that time the hard disk in our = SCO Unix server got really crazy. Even the low level format was corrupted, damage was way beyond mere filesystem corruption. During the start of a power outage (especially when it=E2=80=99s not a = clean power cut, but it comes preceded by some voltage swings) data corruption can be extensive. As far as I know high end systems include = power management elements to reduce the impact.=20 I have other war stories about UPS systems providing an extremely dirty = waveform and causing format problems in disks. That happened in 1995 or so. >>=20 >> Unfortunately however there is also cache memory on most modern hard >> drives, most of the time (unless you explicitly shut it off) it's on = for >> write caching, and it'll nail you too. Oh, and it's never, in my >> experience, ECC. >=20 > No comment on that - you're right in the first part, I can't comment = if there are drives with ECC. Even with cache corruption, ZFS being transaction oriented should offer = a reasonable guarantee of integrity. You may lose 1 miunte, 5 minutes of changes, but there should be stable, committed = data on the disk. Unless the electronics got insane for some milliseconds during the = outage event (see above). >> Oh that is definitely NOT true.... again, from hard experience, >> including (but not limited to) on FreeBSD. >>=20 >> My experience is that ZFS is materially more-resilient but there is = no >> such thing as "can never be corrupted by any set of events." >=20 > The latter part is true - and my blog and my current situation is not = limited to or aimed at FreeBSD specifically, FreeBSD is my experience. = The former part... it has been very resilient, but I think (based on = this certain set of events) it is easily corruptible and I have just = been lucky. You just have to hit a certain write to activate the issue, = and whilst that write and issue might be very very difficult (read: hit = and miss) to hit in normal every day scenarios it can and will = eventually happen. >=20 >> Backup >> strategies for moderately large (e.g. many Terabytes) to very large >> (e.g. Petabytes and beyond) get quite complex but they're also very >> necessary. >>=20 > and there in lies the problem. If you don't have a many 10's of = thousands of dollars backup solutions, you're either: >=20 > 1/ down for a looooong time. > 2/ losing all data and starting again... >=20 > ..and that's the problem... ufs you can recover most (in most = situations) and providing the *data* is there uncorrupted by the fault = you can get it all off with various tools even if it is a complete = mess.... here I am with the data that is apparently ok, but the = metadata is corrupt (and note: as I had stopped writing to the drive = when it started resilvering the data - all of it - should be intact... = even if a mess.) The advantage of ZFS is that it makes it feasible to replicate data. If = you keep a mirror storage server your disaster recovery actions won=E2=80=99= t require the recovery of a full backup (which can take an inordinate = amount of time) but reconfiguring the replica server to assume the role = of the master one.=20 Again, being transaction based somewhat reduces the likelyhood of a = software bug on the master to propagate to the slave causing extensive = corruption. Rewinding to the previous snapshot should help. Borja. From owner-freebsd-stable@freebsd.org Tue Apr 30 16:08:03 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E38BB15963ED for ; Tue, 30 Apr 2019 16:08:02 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from scully.mintsol.com (scully.mintsol.com [199.182.77.206]) by mx1.freebsd.org (Postfix) with ESMTP id A7A7480E86 for ; Tue, 30 Apr 2019 16:08:01 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from mintsol.com (officecc.mintsol.com [96.85.114.33]) by scully.mintsol.com with esmtp; Tue, 30 Apr 2019 12:07:54 -0400 id 00ACDC40.000000005CC872DA.00005560 Received: from localhost (localhost [127.0.0.1]) (IDENT: uid 1002) by mintsol.com with esmtp; Tue, 30 Apr 2019 12:07:55 -0400 id 00000831.5CC872DB.00016B82 Date: Tue, 30 Apr 2019 12:07:55 -0400 (EDT) From: Walter Cramer To: Michelle Sullivan cc: Karl Denninger , freebsd-stable@freebsd.org Subject: Re: ZFS... In-Reply-To: <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> Message-ID: <20190430102024.E84286@mulder.mintsol.com> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> MIME-Version: 1.0 Content-ID: <20190430102102.Q84286@mulder.mintsol.com> X-Rspamd-Queue-Id: A7A7480E86 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of wfc@mintsol.com designates 199.182.77.206 as permitted sender) smtp.mailfrom=wfc@mintsol.com X-Spamd-Result: default: False [-4.60 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:scully.mintsol.com]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; DMARC_NA(0.00)[mintsol.com]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[bmx01.pofox.com]; CTYPE_MIXED_BOGUS(1.00)[]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.97)[-0.968,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:22768, ipnet:199.182.77.0/24, country:US]; IP_SCORE(-2.88)[ip: (-7.54), ipnet: 199.182.77.0/24(-3.77), asn: 22768(-3.02), country: US(-0.06)] Content-Type: TEXT/PLAIN; CHARSET=X-UNKNOWN; FORMAT=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 16:08:03 -0000 Brief "Old Man" summary/perspective here... Computers and hard drives are complex, sensitive physical things. They,=20 or the data on them, can be lost to fire, flood, lightning strikes, theft,= =20 transportation screw-ups, and more. Mass data corruption by faulty=20 hardware or software is mostly rare, but does happen. Then there's the=20 users - authorized or not - who are inept or malicious. You can spent a fortune to make loss of the "live" data in your home=20 server / server room / data center very unlikely. Is that worth the time= =20 and money? Depends on the business case. At any scale, it's best to have= =20 a manager - who understands both computers and the bottom line - keep a=20 close eye on this. "Real" protection from data loss means multiple off-site and generally=20 off-line backups. You could spend a fortune on that, too...but for your=20 use case (~21TB in an array that could hold ~39TB, and what sounds like a= =20 "home power user" budget), I'd say to put together two "backup servers" -= =20 cheap little (aka transportable) FreeBSD systems with, say 7x6GB HD's,=20 raidz1. With even a 1Gbit ethernet connection to your main system, savvy= =20 use of (say) rsync (net/rsync in Ports), and the sort of "know your data /= =20 divide & conquer" tactics that Karl mentions, you should be able to=20 complete initial backups (on both backup servers) in <1 month. After that= =20 - rsync can generally do incremental backups far, far faster. How often=20 you gently haul the backup servers to/from your off-site location(s)=20 depends on a bunch of factors - backup frequency, cost of bandwidth, etc. Never skimp on power supplies. -Walter [Credits: Nothing above is original. Others have already made most of my= =20 points in this thread. It's pretty much all decades-old computer wisdom=20 in any case.] On Tue, 30 Apr 2019, Michelle Sullivan wrote: > Karl Denninger wrote: >> On 4/30/2019 05:14, Michelle Sullivan wrote: >>>> On 30 Apr 2019, at 19:50, Xin LI wrote: >>>>> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan =20 > wrote: >>>>> but in my recent experience 2 issues colliding at the same time resul= ts=20 > in disaster >>>> Do we know exactly what kind of corruption happen to your pool? If yo= u=20 > see it twice in a row, it might suggest a software bug that should be=20 > investigated. >>>> >>>> All I know is it=E2=80=99s a checksum error on a meta slab (122) and f= rom what=20 > I can gather it=E2=80=99s the spacemap that is corrupt... but I am no exp= ert. I=20 > don=E2=80=99t believe it=E2=80=99s a software fault as such, because this= was cause by a=20 > hard outage (damaged UPSes) whilst resilvering a single (but completely= =20 > failed) drive. ...and after the first outage a second occurred (same as = the=20 > first but more damaging to the power hardware)... the host itself was not= =20 > damaged nor were the drives or controller. >> ..... >>>> Note that ZFS stores multiple copies of its essential metadata, and in= my=20 > experience with my old, consumer grade crappy hardware (non-ECC RAM, with= =20 > several faulty, single hard drive pool: bad enough to crash almost monthl= y=20 > and damages my data from time to time), >>> This was a top end consumer grade mb with non ecc ram that had been=20 > running for 8+ years without fault (except for hard drive platter failure= s.).=20 > Uptime would have been years if it wasn=E2=80=99t for patching. >> Yuck. >> >> I'm sorry, but that may well be what nailed you. >> >> ECC is not just about the random cosmic ray. It also saves your bacon >> when there are power glitches. > > No. Sorry no. If the data is only half to disk, ECC isn't going to save= =20 > you at all... it's all about power on the drives to complete the write. >> >> Unfortunately however there is also cache memory on most modern hard >> drives, most of the time (unless you explicitly shut it off) it's on for >> write caching, and it'll nail you too. Oh, and it's never, in my >> experience, ECC. > > No comment on that - you're right in the first part, I can't comment if= =20 > there are drives with ECC. > >> >> In addition, however, and this is something I learned a LONG time ago >> (think Z-80 processors!) is that as in so many very important things >> "two is one and one is none." >> >> In other words without a backup you WILL lose data eventually, and it >> WILL be important. >> >> Raidz2 is very nice, but as the name implies it you have two >> redundancies. If you take three errors, or if, God forbid, you *write* >> a block that has a bad checksum in it because it got scrambled while in >> RAM, you're dead if that happens in the wrong place. > > Or in my case you write part data therefore invalidating the checksum... >> >>> Yeah.. unlike UFS that has to get really really hosed to restore from= =20 > backup with nothing recoverable it seems ZFS can get hosed where issues o= ccur=20 > in just the wrong bit... but mostly it is recoverable (and my experience = has=20 > been some nasty shit that always ended up being recoverable.) >>> >>> Michelle >> Oh that is definitely NOT true.... again, from hard experience, >> including (but not limited to) on FreeBSD. >> >> My experience is that ZFS is materially more-resilient but there is no >> such thing as "can never be corrupted by any set of events." > > The latter part is true - and my blog and my current situation is not=20 > limited to or aimed at FreeBSD specifically, FreeBSD is my experience.= =20 > The former part... it has been very resilient, but I think (based on=20 > this certain set of events) it is easily corruptible and I have just=20 > been lucky. You just have to hit a certain write to activate the issue,= =20 > and whilst that write and issue might be very very difficult (read: hit= =20 > and miss) to hit in normal every day scenarios it can and will=20 > eventually happen. > >> Backup >> strategies for moderately large (e.g. many Terabytes) to very large >> (e.g. Petabytes and beyond) get quite complex but they're also very >> necessary. >> > and there in lies the problem. If you don't have a many 10's of=20 > thousands of dollars backup solutions, you're either: > > 1/ down for a looooong time. > 2/ losing all data and starting again... > > ..and that's the problem... ufs you can recover most (in most=20 > situations) and providing the *data* is there uncorrupted by the fault=20 > you can get it all off with various tools even if it is a complete=20 > mess.... here I am with the data that is apparently ok, but the=20 > metadata is corrupt (and note: as I had stopped writing to the drive=20 > when it started resilvering the data - all of it - should be intact...=20 > even if a mess.) > > Michelle > > --=20 > Michelle Sullivan > http://www.mhix.org/ > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Tue Apr 30 16:13:22 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BCB2815969AF for ; Tue, 30 Apr 2019 16:13:21 +0000 (UTC) (envelope-from cindy.allan@webbasedata.com) Received: from n1nlsmtp03.shr.prod.ams1.secureserver.net (n1nlsmtp03.shr.prod.ams1.secureserver.net [188.121.43.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay-hosting.secureserver.net", Issuer "Starfield Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A735881500 for ; Tue, 30 Apr 2019 16:13:20 +0000 (UTC) (envelope-from cindy.allan@webbasedata.com) Received: from n3plcpnl0275.prod.ams3.secureserver.net ([160.153.155.31]) by : HOSTING RELAY : with ESMTP id LV4IhPWnzRzlGLV4IhFv4D; Tue, 30 Apr 2019 08:53:18 -0700 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=webbasedata.com; s=default; h=Content-Type:MIME-Version:Message-ID:Date: Subject:To:From:Reply-To:Sender:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=9nJhgVjHEX9Ovot1f30bCkj+WZl+Mt7N8eH0nSNTUOg=; b=xg8Q61Jc4ExHXYlnAScgrEf2Wr hR6i4sI+RLvq6EeI1z5+bCq+kqFzrkm/Blqq/uNSYd+4y9aHET04+wYid/PWLdcbrXPKM4BR87B5Q dT/RR/ZXBk8ZPqO9OFYvcQdo8iW6r7+g0JQwqCV4xSqqz7bx7tTxUqSLMQ4pT1KPskcQRlaLATaG9 h29DVT39ZUjYZk233HZpKz7BnHGI+5/d5c62q19+wfuewCuSrvbDnsKXveanWnAFaSLEJbEuM9Aoe hGnoIcSnrEocg0GyF2Z8xAMKTOxX7tOmmQl8HER0m108MsJ5xMPKbX+0ZHO+6BzWVy+MIW73sNhV8 Xcmynm3w==; Received: from [139.99.99.151] (port=50205 helo=WS40) by n3plcpnl0275.prod.ams3.secureserver.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-SHA384:256) (Exim 4.91) (envelope-from ) id 1hLTyT-00GXrI-A5 for freebsd-stable@freebsd.org; Tue, 30 Apr 2019 07:43:14 -0700 Reply-To: From: "Cindy Allan" To: Subject: Nginx Users List. Date: Tue, 30 Apr 2019 20:12:08 +0530 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdT/YuKTCdF7Wfz5R1uOQGEbfY4MSg== Content-Language: en-us X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - n3plcpnl0275.prod.ams3.secureserver.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - webbasedata.com X-Get-Message-Sender-Via: n3plcpnl0275.prod.ams3.secureserver.net: authenticated_id: cindy.allan@webbasedata.com X-Authenticated-Sender: n3plcpnl0275.prod.ams3.secureserver.net: cindy.allan@webbasedata.com X-Source: X-Source-Args: X-Source-Dir: X-CMAE-Envelope: MS4wfPf4yWhLhGF3dyQ1D2nD8MjRCdYfI3ZRAjmse0YDRvpDFqfNLDNJpLIeLXhhr1bKdw6NxZf1lCEwjS6t3jn6WoTT55WcIpO9mlAu2KsFdlir36LnQO2q TTOz6/zY78c6VMV9PMdt1KvPTxCWzd6uI3lXfH6yf52BfZDQYurX/FCiCSL/7hPSPrCcmdk62iHZAyUvkFY/6mvgkMZz++l5x+usK4y1PcCoelFdGdR23UlG X-Rspamd-Queue-Id: A735881500 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none (invalid DKIM record) header.d=webbasedata.com header.s=default header.b=xg8Q61Jc X-Spamd-Result: default: False [2.20 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[cindy.allan@webbasedata.com]; MX_INVALID(0.50)[cached]; HAS_X_SOURCE(0.00)[]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[webbasedata.com:~]; HAS_X_ANTIABUSE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; IP_SCORE(0.37)[ip: (1.21), ipnet: 188.121.40.0/22(0.49), asn: 26496(0.22), country: US(-0.06)]; ASN(0.00)[asn:26496, ipnet:188.121.40.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[]; HAS_X_AS(0.00)[cindy.allan@webbasedata.com]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.17)[0.171,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[webbasedata.com]; NEURAL_SPAM_MEDIUM(0.35)[0.352,0]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_LONG(0.91)[0.907,0]; RCVD_IN_DNSWL_NONE(0.00)[193.43.121.188.list.dnswl.org : 127.0.5.0]; R_DKIM_PERMFAIL(0.00)[webbasedata.com:s=default]; R_SPF_NA(0.00)[]; HAS_X_GMSV(0.00)[cindy.allan@webbasedata.com]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 16:13:22 -0000 Hi, I would like to know if you would be interested in acquiring Nginx Users List. Information fields: First Name, Last Name, Title, Email Address, Phone Number, Company Name, Company URL, Company physical address, SIC Code, Industry, Company Size (Revenue and Employee). We can also help you with: Apache HTTP Server, Docker, Gunicorn, HAProxy, Redis, Squid (software), Apache Tomcat, Internet Information Services, uWSGI, lighttpd, Node.js, PHP, PostgreSQL, MySQL, Jenkins (software), CentOS, Ubuntu, Hypertext Transfer Protocol, Django, OpenSSL, MariaDB, WordPress, Laravel and many more. For further information on Cost and Counts let me know your Target Geography: ___________. Note: We would provide any industry, any job title of contact lists as per your requirements. Kind Regards, Cindy Allan Sr. Marketing Manager To quit accepting future messages, it would be ideal if you answer with "Withdraw" in the Subject Line. From owner-freebsd-stable@freebsd.org Tue Apr 30 19:44:41 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 967E6159C788 for ; Tue, 30 Apr 2019 19:44:41 +0000 (UTC) (envelope-from m.wipp@BlueBird-AG.com) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50105.outbound.protection.outlook.com [40.107.5.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 325718BE7B for ; Tue, 30 Apr 2019 19:44:39 +0000 (UTC) (envelope-from m.wipp@BlueBird-AG.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Teilen.onmicrosoft.com; s=selector1-BlueBirdAG-com02b; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PY8t+J5CPmGwj2pQsHNW5Q2eOc+yWTgP12YhcnO1GeA=; b=UqbA+ggWgVW5UW0g7DiH5+tGNBCrp07y6JBNZFmWl8HLPcvtIQ5XsPXfo0BfeybjzLhYes65fstZdSyLxVR4vCf0JIGfCTZT40BJIOAjMsGf46M5bfUdRyi/O2QCgxn7Vw2UZ3+OMKP50tivihFyzQjcoC3zIyZuMR2DUkLCMrk= Received: from AM2PR09MB0210.eurprd09.prod.outlook.com (10.161.236.19) by AM2PR09MB0451.eurprd09.prod.outlook.com (10.162.36.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1835.15; Tue, 30 Apr 2019 19:44:36 +0000 Received: from AM2PR09MB0210.eurprd09.prod.outlook.com ([fe80::5cb4:eb81:a28c:6f78]) by AM2PR09MB0210.eurprd09.prod.outlook.com ([fe80::5cb4:eb81:a28c:6f78%3]) with mapi id 15.20.1856.008; Tue, 30 Apr 2019 19:44:36 +0000 From: Markus Wipp To: "freebsd-stable@freebsd.org" Subject: Issues starting unbound on boot Thread-Topic: Issues starting unbound on boot Thread-Index: AQHU/40kmoBoQnez2E+XVWTCph3usA== Date: Tue, 30 Apr 2019 19:44:36 +0000 Message-ID: <9E319697-B182-49ED-AB44-F3BBC87EEE1B@bluebird-ag.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.134.72.73] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bec6b9e0-ae92-4559-d4b2-08d6cda446c6 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(7021145)(8989299)(4534185)(7022145)(4603075)(4627221)(201702281549075)(8990200)(7048125)(7024125)(7027125)(7023125)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:AM2PR09MB0451; x-ms-traffictypediagnostic: AM2PR09MB0451: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 00235A1EEF x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(366004)(376002)(39830400003)(346002)(136003)(396003)(189003)(199004)(53754006)(3846002)(66066001)(5640700003)(476003)(2616005)(36756003)(508600001)(486006)(102836004)(6116002)(25786009)(26005)(6506007)(186003)(5660300002)(86362001)(2501003)(71200400001)(83716004)(14454004)(2906002)(71190400001)(33656002)(305945005)(2351001)(66446008)(76116006)(64756008)(82746002)(7736002)(316002)(99286004)(66556008)(6486002)(66476007)(68736007)(8936002)(6916009)(81166006)(73956011)(53936002)(81156014)(256004)(66946007)(14444005)(8676002)(97736004)(6512007)(6436002); DIR:OUT; SFP:1102; SCL:1; SRVR:AM2PR09MB0451; H:AM2PR09MB0210.eurprd09.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: BlueBird-AG.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Li2Maz/rjo2Y51c1kLXrUnxnK4sc7Nl9oCcI30fVbVnNXrN48kWO+xbfCJH9ttC4D07pU81d3QwqEd7AayuRSJi9T6dVAgWZRiOINRO+m9Wgt1JqIT2HTSVugth62lMnDZCE5+szJoKPsWLeHenMMKHqDAzgulVeIRzweWvD0X8on2/WiB6tbUUWn6YCVEH7Tx5rf9QMsRbIKLvvZSPIfceHQhVhg6Qh3ZQg9c9qiaiWIB26S+NNvhs+s/SwCaM1O4E6OlKqmJwTOjw6nrgBh9Biig1rcrQYYlg5VY8DWIkjGm76EGhJkt3gs3dEq33Kah8VQKzkv88W9HFxB2Ls4aJtFeE3GaMFPXr4KgLnb+hcvn5cf/lHGgVPHoG4Xwa2c0u5w6bQdKmtg5dXYvb8m+mnARv6fUAiLZimCQSUqV0= Content-Type: text/plain; charset="utf-8" Content-ID: <35251D9D6C20F04A911EE2F23F02CA4B@eurprd09.prod.outlook.com> Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: BlueBird-AG.com X-MS-Exchange-CrossTenant-Network-Message-Id: bec6b9e0-ae92-4559-d4b2-08d6cda446c6 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Apr 2019 19:44:36.3849 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: fd291ad9-1693-4666-b93b-21139d09fa2a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR09MB0451 X-Rspamd-Queue-Id: 325718BE7B X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=Teilen.onmicrosoft.com header.s=selector1-BlueBirdAG-com02b header.b=UqbA+ggW; spf=pass (mx1.freebsd.org: domain of m.wipp@BlueBird-AG.com designates 40.107.5.105 as permitted sender) smtp.mailfrom=m.wipp@BlueBird-AG.com X-Spamd-Result: default: False [-2.28 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.94)[-0.935,0]; R_DKIM_ALLOW(-0.20)[Teilen.onmicrosoft.com:s=selector1-BlueBirdAG-com02b]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[BlueBird-AG.com]; NEURAL_HAM_LONG(-1.00)[-0.996,0]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.06)[0.056,0]; DKIM_TRACE(0.00)[Teilen.onmicrosoft.com:+]; MIME_BASE64_TEXT(0.10)[]; RCVD_IN_DNSWL_NONE(0.00)[105.5.107.40.list.dnswl.org : 127.0.3.0]; MX_GOOD(-0.01)[re.mail.protection.outlook.com,re.mail.protection.outlook.com]; TO_DN_EQ_ADDR_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 19:44:41 -0000 SGkgYWxsLCANCg0KSSBjdXJyZW50bHkgZmFjZSBhbiBpc3N1ZSwgd2hlcmUgSSBkb27igJl0IGtu b3cgZnVydGhlciBvbiB3aHkgdGhpcyBoYXBwZW5zIGFuZCB3aGF0IEkgY291bGQgZG8gYWJvdXQg aXQuDQpJIGhvcGUgdGhhdCB0aGlzIGlzIHRoZSBjb3JyZWN0IGxpc3QgdG8gYXNrIG15IHF1ZXN0 aW9uLiBJZiBub3QgcGxlYXNlIGxldCBtZSBrbm93IHdoZXJlIGVsc2UgSSBtaWdodCB0cnkgbXkg bHVjay4NCkkgaW5zdGFsbGVkIHVuYm91bmQgZnJvbSBwb3J0cywgY29uZmlndXJlZCBpdCBhbmQg Y2FuIHN0YXJ0IC8gc3RvcCBpdCBmcm9tIGNvbW1hbmQgbGluZSB3aXRoIHNlcnZpY2UgdW5ib3Vu ZCBzdGFydCB3aXRob3V0IGFueSBwcm9ibGVtcy4NCkJ1dCB3aGVuZXZlciBJIHJlYm9vdCB0aGUg bWFjaGluZSBpdCBqdXN0IGRvZXNu4oCZdCBnZXQgc3RhcnRlZC4gVGhlIG9ubHkgaW5mb3JtYXRp b24gSSB3YXMgYWJsZSB0byBmaW5kIG91dCBzbyBmYXIgY2FuIGJlIGZvdW5kIGluIC92YXIvbG9n L21lc3NhZ2VzOg0Kcm9vdDogL2V0Yy9yYzogV0FSTklORzogZmFpbGVkIHRvIHN0YXJ0IHVuYm91 bmQNCg0KSSB0cmllZCB0byBydW4gd2l0aCByY19kZWJ1Zz3igJxZRVPigJ0gYW5kIGdvdCB0aGUg Zm9sbG93aW5nOg0Kcm9vdDogL2V0Yy9yYzogREVCVUc6IGNoZWNreWVzbm86IHVuYm91bmRfZW5h YmxlIGlzIHNldCB0byBZRVMuDQpyb290OiAvZXRjL3JjOiBERUJVRzogcnVuX3JjX2NvbW1hbmQ6 IHN0YXJ0X3ByZWNtZDogc3RhcnRfcHJlY21kDQpyb290OiAvZXRjL3JjOiBERUJVRzogY2hlY2t5 ZXNubzogcmNfc3RhcnRtc2dzIGlzIHNldCB0byBZRVMuDQpyb290OiAvZXRjL3JjOiBERUJVRzog cnVuX3JjX2NvbW1hbmQ6IGRvaXQ6ICBsaW1pdHMgLUMgZGFlbW9uIC91c3IvbG9jYWwvc2Jpbi91 bmJvdW5kICAtYyAvdXNyL2xvY2FsL2V0Yy91bmJvdW5kL3VuYm91bmQuY29uZiANCnJvb3Q6IC9l dGMvcmM6IFdBUk5JTkc6IGZhaWxlZCB0byBzdGFydCB1bmJvdW5kDQoNCnVuYW1lIC1hOg0KRnJl ZUJTRCBoMDFkYzAxIDExLjItUkVMRUFTRS1wOSBGcmVlQlNEIDExLjItUkVMRUFTRS1wOSAjMDog VHVlIEZlYiAgNSAxNTozMDozNiBVVEMgMjAxOSAgICAgcm9vdEBhbWQ2NC1idWlsZGVyLmRhZW1v bm9sb2d5Lm5ldDovdXNyL29iai91c3Ivc3JjL3N5cy9HRU5FUklDICBhbWQ2NA0KDQpUaGFua3Mg Zm9yIGFueSB1c2VmdWwgaGludHMgaW4gYWR2YW5jZQ0KTWFya3Vz From owner-freebsd-stable@freebsd.org Tue Apr 30 20:10:30 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7AB07159D0EA for ; Tue, 30 Apr 2019 20:10:30 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gromit.dlib.vt.edu", Issuer "Chumby Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8047E8CA7C for ; Tue, 30 Apr 2019 20:10:29 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from mather.gromit23.net (c-98-244-101-97.hsd1.va.comcast.net [98.244.101.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id 09941296; Tue, 30 Apr 2019 16:10:28 -0400 (EDT) Content-Type: text/plain; charset=utf-8; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: Issues starting unbound on boot From: Paul Mather In-Reply-To: <9E319697-B182-49ED-AB44-F3BBC87EEE1B@bluebird-ag.com> Date: Tue, 30 Apr 2019 16:10:27 -0400 Cc: "freebsd-stable@freebsd.org" Content-Transfer-Encoding: 8bit Message-Id: References: <9E319697-B182-49ED-AB44-F3BBC87EEE1B@bluebird-ag.com> To: Markus Wipp X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 8047E8CA7C X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dmarc=fail reason="" header.from=vt.edu (policy=none) X-Spamd-Result: default: False [-3.63 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; IP_SCORE(-1.13)[ip: (-2.91), ipnet: 128.173.0.0/16(-1.45), asn: 1312(-1.24), country: US(-0.06)]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[vt.edu : No valid SPF, No valid DKIM,none]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: chumby.dlib.vt.edu]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_SPF_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[97.101.244.98.zen.spamhaus.org : 127.0.0.10]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1312, ipnet:128.173.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 20:10:30 -0000 On Apr 30, 2019, at 3:44 PM, Markus Wipp wrote: > Hi all, > > I currently face an issue, where I don’t know further on why this happens > and what I could do about it. > I hope that this is the correct list to ask my question. If not please > let me know where else I might try my luck. > I installed unbound from ports, configured it and can start / stop it > from command line with service unbound start without any problems. > But whenever I reboot the machine it just doesn’t get started. The only > information I was able to find out so far can be found in > /var/log/messages: > root: /etc/rc: WARNING: failed to start unbound > > I tried to run with rc_debug=“YES” and got the following: > root: /etc/rc: DEBUG: checkyesno: unbound_enable is set to YES. > root: /etc/rc: DEBUG: run_rc_command: start_precmd: start_precmd > root: /etc/rc: DEBUG: checkyesno: rc_startmsgs is set to YES. > root: /etc/rc: DEBUG: run_rc_command: doit: limits -C daemon > /usr/local/sbin/unbound -c /usr/local/etc/unbound/unbound.conf > root: /etc/rc: WARNING: failed to start unbound > > uname -a: > FreeBSD h01dc01 11.2-RELEASE-p9 FreeBSD 11.2-RELEASE-p9 #0: Tue Feb 5 > 15:30:36 UTC 2019 > root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 > > Thanks for any useful hints in advance I don't know how useful this is, but depending upon how the unbound startup script is defined to start (e.g., the REQUIRE: and BEFORE: lines in the startup script), maybe it is set to start up too early, e.g., before /usr/local is mounted? This at least would be one reason why it doesn't start up at boot but will start up manually from the command line. Cheers, Paul. From owner-freebsd-stable@freebsd.org Tue Apr 30 20:24:33 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C843159D645 for ; Tue, 30 Apr 2019 20:24:33 +0000 (UTC) (envelope-from m.wipp@BlueBird-AG.com) Received: from EUR03-VE1-obe.outbound.protection.outlook.com (mail-eopbgr50116.outbound.protection.outlook.com [40.107.5.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5AC798D2A4 for ; Tue, 30 Apr 2019 20:24:31 +0000 (UTC) (envelope-from m.wipp@BlueBird-AG.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Teilen.onmicrosoft.com; s=selector1-BlueBirdAG-com02b; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Tp/dvA8e8MGhDeRiL/drS3YC0eteEVEEDXibmSYjnjA=; b=uPdD2SnZ9urR95jpWEc/5tkXYVxd3a5xSegzMjd/F6WFDRX5icH7W6sZvxoR0SokzhwhcnRAqU+yRXdPHMg2/drFDalehJKfoK06RLa//2nN/DCCYVtW0uB2hUagPDA382RnXSxXH9cKjb/WfE7Oe/YZ6XUdHeYFLa2savv37BA= Received: from AM2PR09MB0210.eurprd09.prod.outlook.com (10.161.236.19) by AM2PR09MB0563.eurprd09.prod.outlook.com (10.162.38.27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1835.16; Tue, 30 Apr 2019 20:24:28 +0000 Received: from AM2PR09MB0210.eurprd09.prod.outlook.com ([fe80::5cb4:eb81:a28c:6f78]) by AM2PR09MB0210.eurprd09.prod.outlook.com ([fe80::5cb4:eb81:a28c:6f78%3]) with mapi id 15.20.1856.008; Tue, 30 Apr 2019 20:24:28 +0000 From: Markus Wipp To: Paul Mather CC: "freebsd-stable@freebsd.org" Subject: Re: Issues starting unbound on boot Thread-Topic: Issues starting unbound on boot Thread-Index: AQHU/40kLzpkeeK+5ESbtg3UVy/4mKZVImaAgAAD6oA= Date: Tue, 30 Apr 2019 20:24:28 +0000 Message-ID: References: <9E319697-B182-49ED-AB44-F3BBC87EEE1B@bluebird-ag.com> In-Reply-To: Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.134.72.73] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 6a21ee23-5fa3-4e90-6136-08d6cda9d8b6 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(7021145)(8989299)(4534185)(7022145)(4603075)(4627221)(201702281549075)(8990200)(7048125)(7024125)(7027125)(7023125)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:AM2PR09MB0563; x-ms-traffictypediagnostic: AM2PR09MB0563: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-forefront-prvs: 00235A1EEF x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(366004)(136003)(396003)(376002)(39830400003)(346002)(53754006)(199004)(189003)(76116006)(73956011)(316002)(186003)(508600001)(26005)(86362001)(5660300002)(99286004)(6436002)(36756003)(102836004)(76176011)(6506007)(14454004)(256004)(14444005)(53546011)(2616005)(476003)(11346002)(2171002)(3846002)(8676002)(6116002)(97736004)(53936002)(66066001)(81166006)(6486002)(33656002)(8936002)(2906002)(486006)(91956017)(6916009)(64756008)(7736002)(54896002)(236005)(82746002)(25786009)(66476007)(66556008)(229853002)(68736007)(71200400001)(83716004)(66946007)(66446008)(6246003)(4326008)(71190400001)(446003)(6512007)(81156014); DIR:OUT; SFP:1102; SCL:1; SRVR:AM2PR09MB0563; H:AM2PR09MB0210.eurprd09.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: BlueBird-AG.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: 667z7gs2ECYsLUH64P2nCWcw07akV5RtTfH11UQ3emzq5grGVgxWX+fKl2VJ/DXDc8W+eh9PI0VH2UAdZ0IsH6j6/oJoyg5erNT3bN6HYysdhgQsWCpeC1YD+Pw9/wLTgFaabktmatgHPagelwy7fdt9uldJVdkn0UfgHfOF8I9lz364fQK60wVkHcOw0/800YhybD3vMRwd3hVI6Z5484xZs/EuSkoEtkv+/MDvwScOyb6d9u9JhhHs9FzKSuOOlHsxn/xvrTA3zfl2aT/vYiUDlH8SA85xiIsKFwXIb0wbVV5nVj5CPwVNok814MMKb8Dh+Tj5E0WKwtPxTQmej3fPbn7C75vi0AiO1SthOLfoPmfCZqebAhEo+M+qhFH3eRiX2gr+qpeHJWEYR7aP4F0zgLFFZ3gYKEEHnEQdy3U= MIME-Version: 1.0 X-OriginatorOrg: BlueBird-AG.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6a21ee23-5fa3-4e90-6136-08d6cda9d8b6 X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Apr 2019 20:24:28.6800 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: fd291ad9-1693-4666-b93b-21139d09fa2a X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM2PR09MB0563 X-Rspamd-Queue-Id: 5AC798D2A4 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=Teilen.onmicrosoft.com header.s=selector1-BlueBirdAG-com02b header.b=uPdD2SnZ; spf=fail (mx1.freebsd.org: domain of m.wipp@BlueBird-AG.com does not designate 40.107.5.116 as permitted sender) smtp.mailfrom=m.wipp@BlueBird-AG.com X-Spamd-Result: default: False [-1.59 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; HAS_XOIP(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[Teilen.onmicrosoft.com:+]; MX_GOOD(-0.01)[cached: re.mail.protection.outlook.com]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.56)[-0.563,0]; MIME_BASE64_TEXT(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_SPF_FAIL(1.00)[-all]; R_DKIM_ALLOW(-0.20)[Teilen.onmicrosoft.com:s=selector1-BlueBirdAG-com02b]; NEURAL_HAM_MEDIUM(-0.88)[-0.883,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.93)[-0.929,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[BlueBird-AG.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[116.5.107.40.list.dnswl.org : 127.0.3.0] Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 20:24:33 -0000 DQpPbiAzMC4gQXByIDIwMTksIGF0IDIyOjEwLCBQYXVsIE1hdGhlciA8cGF1bEBncm9taXQuZGxp Yi52dC5lZHU8bWFpbHRvOnBhdWxAZ3JvbWl0LmRsaWIudnQuZWR1Pj4gd3JvdGU6DQoNCk9uIEFw ciAzMCwgMjAxOSwgYXQgMzo0NCBQTSwgTWFya3VzIFdpcHAgPG0ud2lwcEBCbHVlQmlyZC1BRy5j b208bWFpbHRvOm0ud2lwcEBCbHVlQmlyZC1BRy5jb20+PiB3cm90ZToNCg0KSGkgYWxsLA0KDQpJ IGN1cnJlbnRseSBmYWNlIGFuIGlzc3VlLCB3aGVyZSBJIGRvbuKAmXQga25vdyBmdXJ0aGVyIG9u IHdoeSB0aGlzIGhhcHBlbnMgYW5kIHdoYXQgSSBjb3VsZCBkbyBhYm91dCBpdC4NCkkgaG9wZSB0 aGF0IHRoaXMgaXMgdGhlIGNvcnJlY3QgbGlzdCB0byBhc2sgbXkgcXVlc3Rpb24uIElmIG5vdCBw bGVhc2UgbGV0IG1lIGtub3cgd2hlcmUgZWxzZSBJIG1pZ2h0IHRyeSBteSBsdWNrLg0KSSBpbnN0 YWxsZWQgdW5ib3VuZCBmcm9tIHBvcnRzLCBjb25maWd1cmVkIGl0IGFuZCBjYW4gc3RhcnQgLyBz dG9wIGl0IGZyb20gY29tbWFuZCBsaW5lIHdpdGggc2VydmljZSB1bmJvdW5kIHN0YXJ0IHdpdGhv dXQgYW55IHByb2JsZW1zLg0KQnV0IHdoZW5ldmVyIEkgcmVib290IHRoZSBtYWNoaW5lIGl0IGp1 c3QgZG9lc27igJl0IGdldCBzdGFydGVkLiBUaGUgb25seSBpbmZvcm1hdGlvbiBJIHdhcyBhYmxl IHRvIGZpbmQgb3V0IHNvIGZhciBjYW4gYmUgZm91bmQgaW4gL3Zhci9sb2cvbWVzc2FnZXM6DQpy b290OiAvZXRjL3JjOiBXQVJOSU5HOiBmYWlsZWQgdG8gc3RhcnQgdW5ib3VuZA0KDQpJIHRyaWVk IHRvIHJ1biB3aXRoIHJjX2RlYnVnPeKAnFlFU+KAnSBhbmQgZ290IHRoZSBmb2xsb3dpbmc6DQpy b290OiAvZXRjL3JjOiBERUJVRzogY2hlY2t5ZXNubzogdW5ib3VuZF9lbmFibGUgaXMgc2V0IHRv IFlFUy4NCnJvb3Q6IC9ldGMvcmM6IERFQlVHOiBydW5fcmNfY29tbWFuZDogc3RhcnRfcHJlY21k OiBzdGFydF9wcmVjbWQNCnJvb3Q6IC9ldGMvcmM6IERFQlVHOiBjaGVja3llc25vOiByY19zdGFy dG1zZ3MgaXMgc2V0IHRvIFlFUy4NCnJvb3Q6IC9ldGMvcmM6IERFQlVHOiBydW5fcmNfY29tbWFu ZDogZG9pdDogIGxpbWl0cyAtQyBkYWVtb24gL3Vzci9sb2NhbC9zYmluL3VuYm91bmQgIC1jIC91 c3IvbG9jYWwvZXRjL3VuYm91bmQvdW5ib3VuZC5jb25mDQpyb290OiAvZXRjL3JjOiBXQVJOSU5H OiBmYWlsZWQgdG8gc3RhcnQgdW5ib3VuZA0KDQp1bmFtZSAtYToNCkZyZWVCU0QgaDAxZGMwMSAx MS4yLVJFTEVBU0UtcDkgRnJlZUJTRCAxMS4yLVJFTEVBU0UtcDkgIzA6IFR1ZSBGZWIgIDUgMTU6 MzA6MzYgVVRDIDIwMTkgICAgIHJvb3RAYW1kNjQtYnVpbGRlci5kYWVtb25vbG9neS5uZXQ8bWFp bHRvOnJvb3RAYW1kNjQtYnVpbGRlci5kYWVtb25vbG9neS5uZXQ+Oi91c3Ivb2JqL3Vzci9zcmMv c3lzL0dFTkVSSUMgIGFtZDY0DQoNClRoYW5rcyBmb3IgYW55IHVzZWZ1bCBoaW50cyBpbiBhZHZh bmNlDQoNCg0KSSBkb24ndCBrbm93IGhvdyB1c2VmdWwgdGhpcyBpcywgYnV0IGRlcGVuZGluZyB1 cG9uIGhvdyB0aGUgdW5ib3VuZCBzdGFydHVwIHNjcmlwdCBpcyBkZWZpbmVkIHRvIHN0YXJ0IChl LmcuLCB0aGUgUkVRVUlSRTogYW5kIEJFRk9SRTogbGluZXMgaW4gdGhlIHN0YXJ0dXAgc2NyaXB0 KSwgbWF5YmUgaXQgaXMgc2V0IHRvIHN0YXJ0IHVwIHRvbyBlYXJseSwgZS5nLiwgYmVmb3JlIC91 c3IvbG9jYWwgaXMgbW91bnRlZD8gIFRoaXMgYXQgbGVhc3Qgd291bGQgYmUgb25lIHJlYXNvbiB3 aHkgaXQgZG9lc24ndCBzdGFydCB1cCBhdCBib290IGJ1dCB3aWxsIHN0YXJ0IHVwIG1hbnVhbGx5 IGZyb20gdGhlIGNvbW1hbmQgbGluZS4NCg0KQ2hlZXJzLA0KDQpQYXVsLg0KDQpGaXJzdCBvZiBh bGwgdGhhbmtzIGZvciBsb29raW5nIGF0IG15IGlzc3VlIFBhdWwuDQpJIHRoaW5rIEkgbmV2ZXIg bW9kaWZpZWQgdGhlIHN0YXJ0dXAtc2NyaXB0cywgc28gdGhleSBzaG91bGQgYmUgYXMgaW5zdGFs bGVkIHdpdGggdGhlIHBvcnQuDQojIFBST1ZJREU6IHVuYm91bmQNCiMgUkVRVUlSRTogU0VSVkVS UyBjbGVhbnZhcg0KDQpJ4oCZbSBydW5uaW5nIHRoaXMgbWFjaGluZSBvbiBaRlMsIHNvIC91c3Iv bG9jYWwgc2hvdWxkIGJlIG1vdW50ZWQgdGhlbiwgaWYgSSB1bmRlcnN0YW5kIHRoaXMgY29ycmVj dGx5Lg0KVGhlIHJlYWwgYmFkIHRoaW5nIGlzIHRoYXQgSSBoYXZlIG5vIGlkZWEgd2hlcmUgdG8g c3RhcnQgbG9va2luZyBmb3IgdGhlIHJvb3QgY2F1c2Ugb2YgdGhlIGlzc3VlLg0KDQpUaGFua3MN Ck1hcmt1cw0K From owner-freebsd-stable@freebsd.org Tue Apr 30 22:02:47 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1FC9E159F9A0 for ; Tue, 30 Apr 2019 22:02:47 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vtr.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F0E169DF6 for ; Tue, 30 Apr 2019 22:02:45 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vtr.rulingia.com (8.15.2/8.15.2) with ESMTPS id x3UM2TlJ052915 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 1 May 2019 08:02:35 +1000 (AEST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id x3UM2KEo001207 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 1 May 2019 08:02:20 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id x3UM2KAY001206; Wed, 1 May 2019 08:02:20 +1000 (AEST) (envelope-from peter) Date: Wed, 1 May 2019 08:02:20 +1000 From: Peter Jeremy To: Markus Wipp Cc: "freebsd-stable@freebsd.org" Subject: Re: Issues starting unbound on boot Message-ID: <20190430220220.GC74417@server.rulingia.com> References: <9E319697-B182-49ED-AB44-F3BBC87EEE1B@bluebird-ag.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ZoaI/ZTpAVc4A5k6" Content-Disposition: inline In-Reply-To: <9E319697-B182-49ED-AB44-F3BBC87EEE1B@bluebird-ag.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.11.4 (2019-03-13) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 22:02:47 -0000 --ZoaI/ZTpAVc4A5k6 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2019-Apr-30 19:44:36 +0000, Markus Wipp wrote: >I currently face an issue, where I don=E2=80=99t know further on why this = happens and what I could do about it. >I hope that this is the correct list to ask my question. If not please let= me know where else I might try my luck. >I installed unbound from ports, configured it and can start / stop it from= command line with service unbound start without any problems. >But whenever I reboot the machine it just doesn=E2=80=99t get started. The= only information I was able to find out so far can be found in /var/log/me= ssages: >root: /etc/rc: WARNING: failed to start unbound I have seen unbound fail to start for a variety of reasons but in all cases= , it has written a useful hint to the console. Can you confirm that it's not writin= g anything to your console. Are you able to share your configuration? --=20 Peter Jeremy --ZoaI/ZTpAVc4A5k6 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAlzIxexfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzQ/zw//YqCvPeD1UHNExG7Uq9F0rHYYtXDIDHGHnPqlmfX3Ny0H1DNP/nvWUxGs ohfOCc8oMllwZo3yZ4aVhwLPOyAhRl4UMo0xO2f3S2t8ANz9bZS0ubcr4ScInWJx wfvwMBb5tzij7KipLKCEwt9C50uBLOYmnvM76voOjzeZXaQwcd/AmuN8xR0SknUP u5VMnx8qQrQtyoxxcuSzA3FpsOuRWcQL0HTnic8u6cY4u1VI7qZ+wVxHG0ePYC1u 4iDMKyFraZDDrZeNAKXkyDRz6UitAsIyt9CtlLW3d3OiY0myActoErpiIyiu+nwt YR4ZJHJbpjiO99yqPwYKRVDjAc+xcZMKwIICrt3dF2w2dpG8m14XRvcMwSr41B5T 50y7c2tYBLPB/LRQSYjgyH5HNfU/zItCCJd8dHAi7mXM/nHd/Qb4j6o0MYaflQWP Pc+iDuhX2UZNNvdPsEx2P1uTyeazLTfsLoTrWQlbKWJGL23lrNvVofHYHZCq2Z+V 2cTKULm0SHlgTTIgJx17dr86CzFnvxVm9WXUJuZ+9G+6LDz6ys2JA/s0/cSe9Goi /nEN8XfzQDitFvxxaug1WQmq/7S6X4V2lXeiDonUluwkIDayqqhnqqzQVveH9/j0 +5KvAzx4ncqjOQDgTR74Hn42JziVrh/sj4CodPVOPYy/6JN4LcI= =CJ/N -----END PGP SIGNATURE----- --ZoaI/ZTpAVc4A5k6-- From owner-freebsd-stable@freebsd.org Tue Apr 30 22:13:31 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA7A8159FF9D; Tue, 30 Apr 2019 22:13:31 +0000 (UTC) (envelope-from SRS0=7/jM=TA=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 11DA96A723; Tue, 30 Apr 2019 22:13:30 +0000 (UTC) (envelope-from SRS0=7/jM=TA=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id B397E28417; Wed, 1 May 2019 00:03:44 +0200 (CEST) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 51D3828411; Wed, 1 May 2019 00:03:43 +0200 (CEST) Subject: Re: CFT: FreeBSD Package Base To: David Chisnall Cc: Kris Moore , FreeBSD Stable , freebsd-pkg@freebsd.org, =?UTF-8?Q?Goran_Meki=c4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-ports@freebsd.org References: <201904291931.x3TJV73d079802@slippy.cwsent.com> <6EB65F49-47CD-48B2-8C58-BF387D8C7A58@ixsystems.com> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz> Date: Wed, 1 May 2019 00:03:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 11DA96A723 X-Spamd-Bar: ++++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [6.46 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.88)[0.883,0]; IP_SCORE(0.94)[ip: (0.49), ipnet: 94.124.104.0/21(0.25), asn: 42000(3.88), country: CZ(0.08)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.95)[0.952,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: elsa.codelab.cz]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCPT_COUNT_SEVEN(0.00)[9]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=7/jM=TA=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=7/jM=TA=quip.cz=000.fbsd@elsa.codelab.cz]; SUSPICIOUS_RECIPS(1.50)[] X-Spam: Yes X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 22:13:31 -0000 David Chisnall wrote on 2019/04/30 10:22: > On 29/04/2019 21:12, Joe Maloney wrote: >> With CFT version you chose to build, and package individual components >> such as sendmail with a port option.  That does entirely solve the >> problem of being able to reinstall sendmail after the fact without a >> rebuild of the userland (base) port but perhaps base flavors could >> solve that problem assuming flavors could extend beyond python. > > This sounds very much like local optimisation. It's now easy to create a > custom base image.  Great.  But how do I express dependencies in ports > on a specific base configuration? This is easy if I depend on a specific > base package, but how does this work in your model?  For example, if I > have a package that depends on a library that is an optional part of the > base system, how do I express that pkg needs to either refuse to install > it, or install a userland pkg that includes that library in place of my > existing version as part of the install process? > > More importantly for the container use case, if I want to take a > completely empty jail and do pkg ins nginx (for example), what does the > maintainer of the nginx port need to do to express the minimum set of > the base system that needs to be installed to allow nginx to work? > > One of the goals for the pkg base concept was to allow this kind of use > case, easily creating a minimal environment required to run a single > service. With a monolithic base package set, you're going to need some > mechanism other than packages to express the specific base subset > package that you need and I think that you need to justify why this > mechanism is better than using small individual packages. Will it not be maintainer's nightmare to take care of all the dependencies on the base packages for each port we have in the ports tree? Miroslav Lachman From owner-freebsd-stable@freebsd.org Tue Apr 30 23:18:55 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5365215A15A4 for ; Tue, 30 Apr 2019 23:18:55 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 8FBD26C83A for ; Tue, 30 Apr 2019 23:18:44 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-transfer-encoding: 8BIT Content-type: text/plain; charset=UTF-8; format=flowed Received: from isux.com (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQS002JWS29Y410@hades.sorbs.net> for freebsd-stable@freebsd.org; Tue, 30 Apr 2019 16:32:35 -0700 (PDT) Subject: Re: ZFS... To: Walter Cramer Cc: Karl Denninger , freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <20190430102024.E84286@mulder.mintsol.com> From: Michelle Sullivan Message-id: Date: Wed, 01 May 2019 09:18:35 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-reply-to: <20190430102024.E84286@mulder.mintsol.com> X-Rspamd-Queue-Id: 8FBD26C83A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.86 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sorbs.net]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[battlestar.sorbs.net,anaconda.sorbs.net,ninja.sorbs.net,catapilla.sorbs.net,scorpion.sorbs.net]; NEURAL_HAM_SHORT(-0.92)[-0.917,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.69)[ip: (-1.81), ipnet: 72.12.192.0/19(-0.88), asn: 11114(-0.68), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Apr 2019 23:18:55 -0000 Walter Cramer wrote: > Brief "Old Man" summary/perspective here... > > Computers and hard drives are complex, sensitive physical things. > They, or the data on them, can be lost to fire, flood, lightning > strikes, theft, transportation screw-ups, and more. Mass data > corruption by faulty hardware or software is mostly rare, but does > happen. Then there's the users - authorized or not - who are inept or > malicious. Yup > > You can spent a fortune to make loss of the "live" data in your home > server / server room / data center very unlikely. Is that worth the > time and money? Depends on the business case. At any scale, it's > best to have a manager - who understands both computers and the bottom > line - keep a close eye on this. > That would sorta be my point.. (and yet default FreeBSD install - can't remember wihch version, could be everything current - is to push everything on a filesystem that relies on perfect (or as near as it) hardware where you know, we all know, that the target hardware is consumer grade).. I have 2 machines - almost identical here.. the differences only being the motherboard, CPU and RAM.. the cases are both identical, the PSUs are to, even the drives and controllers are (for the zfs part at least).. Both Supermicro cases with dual psus, both with 16x ST30000VN* drives (Iron Wolf, NAS drives) both with LSI HBAs, both with kingston dual 128GB flash drives mirrored for the base OS. One with 32G non ECC RAM and an onboard RAID for the OS drives, the other with a supermicro board and 16GB ECC RAM and FreeBSD (Geom) mirroring for the OS drive. > "Real" protection from data loss means multiple off-site and generally > off-line backups. You could spend a fortune on that, too...but for > your use case (~21TB in an array that could hold ~39TB, and what > sounds like a "home power user" budget), I'd say to put together two > "backup servers" - cheap little (aka transportable) FreeBSD systems > with, say 7x6GB HD's, raidz1. At the time 6TB ("T" :) ) were not available - 3's and 4's where the top available... it's just rolled on with replacements since. > With even a 1Gbit ethernet connection to your main system, savvy use > of (say) rsync (net/rsync in Ports), and the sort of "know your data / > divide & conquer" tactics that Karl mentions, you should be able to > complete initial backups (on both backup servers) in <1 month. After > that - rsync can generally do incremental backups far, far faster. > How often you gently haul the backup servers to/from your off-site > location(s) depends on a bunch of factors - backup frequency, cost of > bandwidth, etc. 2xbonded Gig connections each server... offsite backup not possible (feasible) due to the Australian mess that they call broadband (12MBps max - running down a hill, on a good day with the wind behind you, and 30 hail Marys every 20 yards) > > Never skimp on power supplies. Hence, dual supermicro, with dual 6 kva HP UPSs with batteries replaced every 36 months and a generator. > > -Walter > > [Credits: Nothing above is original. Others have already made most > of my points in this thread. It's pretty much all decades-old > computer wisdom in any case.] Yup, I know the drill. Michelle > > > On Tue, 30 Apr 2019, Michelle Sullivan wrote: > >> Karl Denninger wrote: >>> On 4/30/2019 05:14, Michelle Sullivan wrote: >>>>> On 30 Apr 2019, at 19:50, Xin LI wrote: >>>>>> On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan >>>>>> >> wrote: >>>>>> but in my recent experience 2 issues colliding at the same time >>>>>> results >> in disaster >>>>> Do we know exactly what kind of corruption happen to your pool? >>>>> If you >> see it twice in a row, it might suggest a software bug that should be >> investigated. >>>>> >>>>> All I know is it’s a checksum error on a meta slab (122) and from >>>>> what >> I can gather it’s the spacemap that is corrupt... but I am no >> expert. I don’t believe it’s a software fault as such, because this >> was cause by a hard outage (damaged UPSes) whilst resilvering a >> single (but completely failed) drive. ...and after the first outage >> a second occurred (same as the first but more damaging to the power >> hardware)... the host itself was not damaged nor were the drives or >> controller. >>> ..... >>>>> Note that ZFS stores multiple copies of its essential metadata, >>>>> and in my >> experience with my old, consumer grade crappy hardware (non-ECC RAM, >> with several faulty, single hard drive pool: bad enough to crash >> almost monthly and damages my data from time to time), >>>> This was a top end consumer grade mb with non ecc ram that had been >> running for 8+ years without fault (except for hard drive platter >> failures.). Uptime would have been years if it wasn’t for patching. >>> Yuck. >>> >>> I'm sorry, but that may well be what nailed you. >>> >>> ECC is not just about the random cosmic ray. It also saves your bacon >>> when there are power glitches. >> >> No. Sorry no. If the data is only half to disk, ECC isn't going to >> save you at all... it's all about power on the drives to complete the >> write. >>> >>> Unfortunately however there is also cache memory on most modern hard >>> drives, most of the time (unless you explicitly shut it off) it's on >>> for >>> write caching, and it'll nail you too. Oh, and it's never, in my >>> experience, ECC. >> >> No comment on that - you're right in the first part, I can't comment >> if there are drives with ECC. >> >>> >>> In addition, however, and this is something I learned a LONG time ago >>> (think Z-80 processors!) is that as in so many very important things >>> "two is one and one is none." >>> >>> In other words without a backup you WILL lose data eventually, and it >>> WILL be important. >>> >>> Raidz2 is very nice, but as the name implies it you have two >>> redundancies. If you take three errors, or if, God forbid, you *write* >>> a block that has a bad checksum in it because it got scrambled while in >>> RAM, you're dead if that happens in the wrong place. >> >> Or in my case you write part data therefore invalidating the checksum... >>> >>>> Yeah.. unlike UFS that has to get really really hosed to restore from >> backup with nothing recoverable it seems ZFS can get hosed where >> issues occur in just the wrong bit... but mostly it is recoverable >> (and my experience has been some nasty shit that always ended up >> being recoverable.) >>>> >>>> Michelle >>> Oh that is definitely NOT true.... again, from hard experience, >>> including (but not limited to) on FreeBSD. >>> >>> My experience is that ZFS is materially more-resilient but there is no >>> such thing as "can never be corrupted by any set of events." >> >> The latter part is true - and my blog and my current situation is not >> limited to or aimed at FreeBSD specifically, FreeBSD is my >> experience. The former part... it has been very resilient, but I >> think (based on this certain set of events) it is easily corruptible >> and I have just been lucky. You just have to hit a certain write to >> activate the issue, and whilst that write and issue might be very >> very difficult (read: hit and miss) to hit in normal every day >> scenarios it can and will eventually happen. >> >>> Backup >>> strategies for moderately large (e.g. many Terabytes) to very large >>> (e.g. Petabytes and beyond) get quite complex but they're also very >>> necessary. >>> >> and there in lies the problem. If you don't have a many 10's of >> thousands of dollars backup solutions, you're either: >> >> 1/ down for a looooong time. >> 2/ losing all data and starting again... >> >> ..and that's the problem... ufs you can recover most (in most >> situations) and providing the *data* is there uncorrupted by the >> fault you can get it all off with various tools even if it is a >> complete mess.... here I am with the data that is apparently ok, but >> the metadata is corrupt (and note: as I had stopped writing to the >> drive when it started resilvering the data - all of it - should be >> intact... even if a mess.) >> >> Michelle >> >> -- >> Michelle Sullivan >> http://www.mhix.org/ >> >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to >> "freebsd-stable-unsubscribe@freebsd.org" >> -- Michelle Sullivan http://www.mhix.org/ From owner-freebsd-stable@freebsd.org Wed May 1 00:14:08 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58AFF15A25DC for ; Wed, 1 May 2019 00:14:08 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id D914E6E1B2 for ; Wed, 1 May 2019 00:14:06 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQS002MEUMOY410@hades.sorbs.net> for freebsd-stable@freebsd.org; Tue, 30 Apr 2019 17:28:02 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> Date: Wed, 01 May 2019 10:14:02 +1000 Cc: freebsd-stable@freebsd.org Content-transfer-encoding: quoted-printable Message-id: References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> To: Karl Denninger X-Rspamd-Queue-Id: D914E6E1B2 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.37 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.66)[ip: (-1.74), ipnet: 72.12.192.0/19(-0.85), asn: 11114(-0.66), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.95)[-0.946,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 00:14:08 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 01 May 2019, at 01:15, Karl Denninger wrote: >=20 >=20 > IMHO non-ECC memory systems are ok for personal desktop and laptop > machines where loss of stored data requiring a restore is acceptable > (assuming you have a reasonable backup paradigm for same) but not for > servers and *especially* not for ZFS storage. I don't like the price of > ECC memory and I really don't like Intel's practices when it comes to > only enabling ECC RAM on their "server" class line of CPUs either but it > is what it is. Pay up for the machines where it matters. And the irony is the FreeBSD policy to default to zfs on new installs using t= he complete drive.. even when there is only one disk available and regardles= s of the cpu or ram class... with one usb stick I have around here it attem= pted to use zfs on one of my laptops. Damned if you do, damned if you don=E2=80=99t comes to mind. Michelle= From owner-freebsd-stable@freebsd.org Wed May 1 01:33:55 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB33F15A3A3D for ; Wed, 1 May 2019 01:33:54 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EFE1E703AD for ; Wed, 1 May 2019 01:33:53 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id BDF91211126 for ; Tue, 30 Apr 2019 21:33:22 -0400 (EDT) Received: from [192.168.10.25] (D15.Denninger.Net [192.168.10.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 1B820C8261 for ; Tue, 30 Apr 2019 20:33:22 -0500 (CDT) Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Tue, 30 Apr 2019 20:33:21 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms030405040800090701020109" X-Rspamd-Queue-Id: EFE1E703AD X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.12 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.84)[-0.839,0]; SUBJ_ALL_CAPS(0.45)[6]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.52)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.15), asn: 14061(1.47), country: US(-0.06)]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 01:33:55 -0000 This is a cryptographically signed message in MIME format. --------------ms030405040800090701020109 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 4/30/2019 19:14, Michelle Sullivan wrote: > > Michelle Sullivan > http://www.mhix.org/ > Sent from my iPad > >> On 01 May 2019, at 01:15, Karl Denninger wrote: >> >> >> IMHO non-ECC memory systems are ok for personal desktop and laptop >> machines where loss of stored data requiring a restore is acceptable >> (assuming you have a reasonable backup paradigm for same) but not for >> servers and *especially* not for ZFS storage. I don't like the price = of >> ECC memory and I really don't like Intel's practices when it comes to >> only enabling ECC RAM on their "server" class line of CPUs either but = it >> is what it is. Pay up for the machines where it matters. > And the irony is the FreeBSD policy to default to zfs on new installs u= sing the complete drive.. even when there is only one disk available and = regardless of the cpu or ram class... with one usb stick I have around h= ere it attempted to use zfs on one of my laptops. > > Damned if you do, damned if you don=E2=80=99t comes to mind. > Nope.=C2=A0 I'd much rather *know* the data is corrupt and be forced to restore from backups than to have SILENT corruption occur and perhaps screw me 10 years down the road when the odds are my backups have long-since been recycled. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms030405040800090701020109 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNTAxMDEzMzIx WjBPBgkqhkiG9w0BCQQxQgRAj0tKNVViAhkHlWMavxp6QAenY/xUuzwjiL3JBqMHjFbCS4SE kRtb7ppQaBCQvfaB9DQii950cmjO9dpm6iMCCzBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgAkf2697jwOdOF979ma6fTjJht//GFSyy7BBB7rTTE6f5wddLpyQvgTkjt2Hljyz6Ck QrEU7Hbjja65k4gLlHsKb55K7mwN/eol6FnXWJglh8EpOovx+jxpsNEUcNKHOpRH5QNoxpOV O8IzpYBDYUMrTHQoIOxHdqJQFufrD9Bqpm0xooTRTpuvtpE4JIbwKg/dbXPrB+H7nCnOwWh4 vDeIVOrbNeK3Pcxu5ojJOFihWpqjPsVd3eNk/m+eIAeztvF6Wh4uWtyUj60tRN70HrFAOmu5 wbJuAEVFqS7CiU5fe+07adoFBhMjbsPp0izl7xRQrI0f3dpZDKEBKVCQNaoBfS25IiCIORZl HqDvUNHhdevgzxqhyWztD1mTm43pcvQ8TW2nypR2lQhLnT/M+s0u3mxUbbgKdA91gTSKohhq 3QfA3mWb2/O/yN3fdppgUADpAAyqsDUptIZbxJ0froNeOl+D/wcwQ0o1kVdPIxEBLip3GGOu 56/IhZOyvtj4pyfWbjNw1oJCMcAL3iW+6+eMdODDevfrHX+QiPjBBP8AmWz/9ErmzwlySKlh 69fEFDGLnVg+apjgoWBU03AjAZ6PiadtOtarFfxC7v38M2YU4jVKPkpOdviuvYRH7k/olHN8 tOsN1kxvxJNAhSTDW0RWSZxpT+BgaHa7FBRiMWPvFgAAAAAAAA== --------------ms030405040800090701020109-- From owner-freebsd-stable@freebsd.org Wed May 1 01:59:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05BF115804F6 for ; Wed, 1 May 2019 01:59:44 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 9142B7147F for ; Wed, 1 May 2019 01:59:42 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQS00M3XZINZ500@hades.sorbs.net> for freebsd-stable@freebsd.org; Tue, 30 Apr 2019 19:13:39 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Wed, 01 May 2019 11:59:38 +1000 Cc: freebsd-stable@freebsd.org Content-transfer-encoding: quoted-printable Message-id: References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> To: Karl Denninger X-Rspamd-Queue-Id: 9142B7147F X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.93 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.64)[ip: (-1.67), ipnet: 72.12.192.0/19(-0.82), asn: 11114(-0.64), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.54)[-0.538,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 01:59:44 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 01 May 2019, at 11:33, Karl Denninger wrote: >=20 >=20 >> On 4/30/2019 19:14, Michelle Sullivan wrote: >>=20 >> Michelle Sullivan >> http://www.mhix.org/ >> Sent from my iPad >>=20 >>> On 01 May 2019, at 01:15, Karl Denninger wrote: >>>=20 >>>=20 >>> IMHO non-ECC memory systems are ok for personal desktop and laptop >>> machines where loss of stored data requiring a restore is acceptable >>> (assuming you have a reasonable backup paradigm for same) but not for >>> servers and *especially* not for ZFS storage. I don't like the price of= >>> ECC memory and I really don't like Intel's practices when it comes to >>> only enabling ECC RAM on their "server" class line of CPUs either but it= >>> is what it is. Pay up for the machines where it matters. >> And the irony is the FreeBSD policy to default to zfs on new installs usi= ng the complete drive.. even when there is only one disk available and regar= dless of the cpu or ram class... with one usb stick I have around here it a= ttempted to use zfs on one of my laptops. >>=20 >> Damned if you do, damned if you don=E2=80=99t comes to mind. >>=20 > Nope. I'd much rather *know* the data is corrupt and be forced to > restore from backups than to have SILENT corruption occur and perhaps > screw me 10 years down the road when the odds are my backups have > long-since been recycled. Ahh yes the be all and end all of ZFS.. stops the silent corruption of data.= . but don=E2=80=99t install it on anything unless it=E2=80=99s server grade w= ith backups and ECC RAM, but it=E2=80=99s good on laptops because it protect= s you from silent corruption of your data when 10 years later the backups ha= ve long-since been recycled... umm is that not a circular argument? Don=E2=80=99t get me wrong here.. and I know you (and some others are) zfs i= n the DC with 10s of thousands in redundant servers and/or backups to keep y= our critical data corruption free =3D good thing. ZFS on everything is what some say (because it prevents silent corruption) b= ut then you have default policies to install it everywhere .. including hard= ware not equipped to function safely with it (in your own arguments) and yet= it=E2=80=99s still good because it will still prevent silent corruption eve= n though it relies on hardware that you can trust... umm say what? Anyhow veered way way off (the original) topic... Modest (part consumer grade, part commercial) suffered irreversible data los= s because of a (very unusual, but not impossible) double power outage.. and n= o tools to recover the data (or part data) unless you have some form of back= up because the file system deems the corruption to be too dangerous to let y= ou access any of it (even the known good bits) ... =20 Michelle > Karl Denninger > karl@denninger.net > /The Market Ticker/ > /[S/MIME encrypted email preferred]/ From owner-freebsd-stable@freebsd.org Wed May 1 02:26:21 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 441811580FA6; Wed, 1 May 2019 02:26:21 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 2CC0071F82; Wed, 1 May 2019 02:26:19 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from isux.com (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQT00M540R1Z500@hades.sorbs.net>; Tue, 30 Apr 2019 19:40:16 -0700 (PDT) Subject: Re: ZFS... To: Xin LI Cc: rainer@ultra-secure.de, owner-freebsd-stable@freebsd.org, freebsd-stable , Andrea Venturoli References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> From: Michelle Sullivan Message-id: Date: Wed, 01 May 2019 12:26:15 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-reply-to: X-Rspamd-Queue-Id: 2CC0071F82 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.68 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sorbs.net]; RCPT_COUNT_FIVE(0.00)[5]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; NEURAL_HAM_SHORT(-0.81)[-0.810,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.62)[ip: (-1.60), ipnet: 72.12.192.0/19(-0.79), asn: 11114(-0.61), country: US(-0.06)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 02:26:21 -0000 Xin LI wrote: > > On Tue, Apr 30, 2019 at 5:08 PM Michelle Sullivan > wrote: > > but in my recent experience 2 issues colliding at the same time > results in disaster > > > Do we know exactly what kind of corruption happen to your pool? If > you see it twice in a row, it might suggest a software bug that should > be investigated. > Oh I did spot one interesting bug... though it is benign... Check out the following (note the difference between 'zpool status' and 'zpool status -v'): root@colossus:/mnt # zpool status pool: storage state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Apr 29 20:22:03 2019 6.54T scanned at 0/s, 6.54T issued at 0/s, 28.8T total 445G resilvered, 22.66% done, no estimated completion time config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 5 raidz2-0 ONLINE 0 0 20 mfid11 ONLINE 0 0 0 mfid10 ONLINE 0 0 0 mfid8 ONLINE 0 0 0 mfid7 ONLINE 0 0 0 mfid0 ONLINE 0 0 0 mfid5 ONLINE 0 0 0 mfid4 ONLINE 0 0 0 mfid3 ONLINE 0 0 0 mfid2 ONLINE 0 0 0 mfid14 ONLINE 0 0 0 mfid15 ONLINE 0 0 0 mfid6 ONLINE 0 0 0 mfid9 ONLINE 0 0 0 mfid13 ONLINE 0 0 0 mfid1 ONLINE 0 0 0 errors: 4 data errors, use '-v' for a list root@colossus:/mnt # zpool status pool: storage state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Apr 29 20:22:03 2019 6.54T scanned at 0/s, 6.54T issued at 0/s, 28.8T total 445G resilvered, 22.66% done, no estimated completion time config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 5 raidz2-0 ONLINE 0 0 20 mfid11 ONLINE 0 0 0 mfid10 ONLINE 0 0 0 mfid8 ONLINE 0 0 0 mfid7 ONLINE 0 0 0 mfid0 ONLINE 0 0 0 mfid5 ONLINE 0 0 0 mfid4 ONLINE 0 0 0 mfid3 ONLINE 0 0 0 mfid2 ONLINE 0 0 0 mfid14 ONLINE 0 0 0 mfid15 ONLINE 0 0 0 mfid6 ONLINE 0 0 0 mfid9 ONLINE 0 0 0 mfid13 ONLINE 0 0 0 mfid1 ONLINE 0 0 0 errors: 4 data errors, use '-v' for a list root@colossus:/mnt # zpool status -v pool: storage state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Apr 29 20:22:03 2019 6.54T scanned at 0/s, 6.54T issued at 0/s, 28.8T total 445G resilvered, 22.66% done, no estimated completion time config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 5 raidz2-0 ONLINE 0 0 20 mfid11 ONLINE 0 0 0 mfid10 ONLINE 0 0 0 mfid8 ONLINE 0 0 0 mfid7 ONLINE 0 0 0 mfid0 ONLINE 0 0 0 mfid5 ONLINE 0 0 0 mfid4 ONLINE 0 0 0 mfid3 ONLINE 0 0 0 mfid2 ONLINE 0 0 0 mfid14 ONLINE 0 0 0 mfid15 ONLINE 0 0 0 mfid6 ONLINE 0 0 0 mfid9 ONLINE 0 0 0 mfid13 ONLINE 0 0 0 mfid1 ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: :<0x3e> :<0x5d> storage:<0x0> storage@now:<0x0> root@colossus:/mnt # zpool status -v pool: storage state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Apr 29 20:22:03 2019 6.54T scanned at 0/s, 6.54T issued at 0/s, 28.8T total 445G resilvered, 22.66% done, no estimated completion time config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 7 raidz2-0 ONLINE 0 0 28 mfid11 ONLINE 0 0 0 mfid10 ONLINE 0 0 0 mfid8 ONLINE 0 0 0 mfid7 ONLINE 0 0 0 mfid0 ONLINE 0 0 0 mfid5 ONLINE 0 0 0 mfid4 ONLINE 0 0 0 mfid3 ONLINE 0 0 0 mfid2 ONLINE 0 0 0 mfid14 ONLINE 0 0 0 mfid15 ONLINE 0 0 0 mfid6 ONLINE 0 0 0 mfid9 ONLINE 0 0 0 mfid13 ONLINE 0 0 0 mfid1 ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: :<0x3e> :<0x5d> storage:<0x0> storage@now:<0x0> root@colossus:/mnt # zpool status -v pool: storage state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Apr 29 20:22:03 2019 6.54T scanned at 0/s, 6.54T issued at 0/s, 28.8T total 445G resilvered, 22.66% done, no estimated completion time config: NAME STATE READ WRITE CKSUM mfid10 ONLINE 0 0 0 mfid8 ONLINE 0 0 0 mfid7 ONLINE 0 0 0 mfid0 ONLINE 0 0 0 mfid5 ONLINE 0 0 0 mfid4 ONLINE 0 0 0 mfid3 ONLINE 0 0 0 mfid2 ONLINE 0 0 0 mfid14 ONLINE 0 0 0 mfid15 ONLINE 0 0 0 mfid6 ONLINE 0 0 0 mfid9 ONLINE 0 0 0 mfid13 ONLINE 0 0 0 mfid1 ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: :<0x3e> :<0x5d> storage:<0x0> storage@now:<0x0> root@colossus:/mnt # zpool status -v pool: storage state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Mon Apr 29 20:22:03 2019 6.54T scanned at 0/s, 6.54T issued at 0/s, 28.8T total 445G resilvered, 22.66% done, no estimated completion time config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 11 raidz2-0 ONLINE 0 0 44 mfid11 ONLINE 0 0 0 mfid10 ONLINE 0 0 0 mfid8 ONLINE 0 0 0 mfid7 ONLINE 0 0 0 mfid0 ONLINE 0 0 0 mfid5 ONLINE 0 0 0 mfid4 ONLINE 0 0 0 mfid3 ONLINE 0 0 0 mfid2 ONLINE 0 0 0 mfid14 ONLINE 0 0 0 mfid15 ONLINE 0 0 0 mfid6 ONLINE 0 0 0 mfid9 ONLINE 0 0 0 mfid13 ONLINE 0 0 0 mfid1 ONLINE 0 0 0 errors: Permanent errors have been detected in the following files: :<0x3e> :<0x5d> storage:<0x0> storage@now:<0x0> root@colossus:/mnt # -- Michelle Sullivan http://www.mhix.org/ From owner-freebsd-stable@freebsd.org Wed May 1 02:37:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22B6015813A0 for ; Wed, 1 May 2019 02:37:44 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 01DE3723B8 for ; Wed, 1 May 2019 02:37:42 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id B3CE1211125 for ; Tue, 30 Apr 2019 22:37:41 -0400 (EDT) Received: from [192.168.10.25] (D15.Denninger.Net [192.168.10.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id B508AC845D for ; Tue, 30 Apr 2019 21:37:40 -0500 (CDT) Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Tue, 30 Apr 2019 21:37:40 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms080702000005000509070200" X-Rspamd-Queue-Id: 01DE3723B8 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.26 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: px.denninger.net]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; SUBJ_ALL_CAPS(0.45)[6]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[197.57.1.68.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.52)[ip: (-9.88), ipnet: 104.236.64.0/18(-4.16), asn: 14061(1.47), country: US(-0.06)]; DMARC_NA(0.00)[denninger.net]; R_SPF_NA(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 02:37:44 -0000 This is a cryptographically signed message in MIME format. --------------ms080702000005000509070200 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 4/30/2019 20:59, Michelle Sullivan wrote >> On 01 May 2019, at 11:33, Karl Denninger wrote: >> >>> On 4/30/2019 19:14, Michelle Sullivan wrote: >>> >>> Michelle Sullivan >>> http://www.mhix.org/ >>> Sent from my iPad >>> >> Nope. I'd much rather *know* the data is corrupt and be forced to >> restore from backups than to have SILENT corruption occur and perhaps >> screw me 10 years down the road when the odds are my backups have >> long-since been recycled. > Ahh yes the be all and end all of ZFS.. stops the silent corruption of = data.. but don=E2=80=99t install it on anything unless it=E2=80=99s serve= r grade with backups and ECC RAM, but it=E2=80=99s good on laptops becaus= e it protects you from silent corruption of your data when 10 years later= the backups have long-since been recycled... umm is that not a circular= argument? > > Don=E2=80=99t get me wrong here.. and I know you (and some others are) = zfs in the DC with 10s of thousands in redundant servers and/or backups t= o keep your critical data corruption free =3D good thing. > > ZFS on everything is what some say (because it prevents silent corrupti= on) but then you have default policies to install it everywhere .. includ= ing hardware not equipped to function safely with it (in your own argumen= ts) and yet it=E2=80=99s still good because it will still prevent silent = corruption even though it relies on hardware that you can trust... umm s= ay what? > > Anyhow veered way way off (the original) topic... > > Modest (part consumer grade, part commercial) suffered irreversible dat= a loss because of a (very unusual, but not impossible) double power outag= e.. and no tools to recover the data (or part data) unless you have some = form of backup because the file system deems the corruption to be too dan= gerous to let you access any of it (even the known good bits) ... =20 > > Michelle IMHO you're dead wrong Michelle.=C2=A0 I respect your opinion but disagre= e vehemently. I run ZFS on both of my laptops under FreeBSD.=C2=A0 Both have non-power-protected SSDs in them.=C2=A0 Neither is mirrored or Raidz-anyt= hing. So why run ZFS instead of UFS? Because a scrub will detect data corruption that UFS cannot detect *at al= l.* It is a balance-of-harms test and you choose.=C2=A0 I can make a very cle= an argument that *greater information always wins*; that is, I prefer in every case to *know* I'm screwed rather than not.=C2=A0 I can defend agai= nst being screwed with some amount of diligence but in order for that diligence to be reasonable I have to know about the screwing in a reasonable amount of time after it happens. You may have never had silent corruption bite you.=C2=A0 I have had it ha= ppen several times over my IT career.=C2=A0 If that happens to you the odds ar= e that it's absolutely unrecoverable and whatever gets corrupted is *gone.*=C2=A0 The defensive measures against silent corruption require retention of backup data *literally forever* for the entire useful life of the information because from the point of corruption forward *the backups are typically going to be complete and correct copies of the corrupt data and thus equally worthless to what's on the disk itself.*=C2= =A0 With non-ZFS filesystems quite a lot of thought and care has to go into defending against that, and said defense usually requires the active cooperation of whatever software wrote said file in the first place (e.g. a database, etc.)=C2=A0 If said software has no tools to "walk" sai= d data or if it's impractical to have it do so you're at severe risk of being hosed.=C2=A0 Prior to ZFS there really wasn't any comprehensive def= ense against this sort of event.=C2=A0 There are a whole host of applications = that manipulate data that are absolutely reliant on that sort of thing not happening (e.g. anything using a btree data structure) and recovery if it *does* happen is a five-alarm nightmare if it's possible at all.=C2=A0= In the worst-case scenario you don't detect the corruption and the data that has the pointer to it that gets corrupted is overwritten and=C2=A0 destroyed. A ZFS scrub on a volume that has no redundancy cannot *fix* that corruption but it can and will detect it.=C2=A0 This puts a boundary on t= he backups that I must keep in order to *not* have that happen.=C2=A0 This i= s of very high value to me and is why, even on systems without ECC memory and without redundant disks, provided there is enough RAM to make it reasonable (e.g. not on embedded systems I do development on with are severely RAM-constrained) I run ZFS. BTW if you've never had a UFS volume unlink all the blocks within a file on an fsck and then recover them back into the free list after a crash you're a rare bird indeed.=C2=A0 If you think a corrupt ZFS volume is fun= try to get your data back from said file after that happens. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms080702000005000509070200 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNTAxMDIzNzQw WjBPBgkqhkiG9w0BCQQxQgRAs8kflupnuXWZUuWfr89pyjPUzEF9eEq7vtl3PwY6Yc7rXbp7 4aiX23eSJsxq1dU6FCLjF6a1Tzsowd/sXLjwZDBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCaDoKkS16QpnUKDf7FGNKnYXBKoVdP6bZqflqnztz/SntaJtME90A9JkNoEEtaSyPm JtGEaBsJjdU0QuiXgWF7fHzgFunWLNdlA9RKCocLAEBCN792NWzE+IyN/AzgyAT4yZsj04En 37we426s0XGR9YLNM6Ikfq6LiFOXuHZL1LClvzmpLbXa+m32Oc3quHaRk/VKKg0kQKYSBjF6 cjjGIWP4yTxwAtud0MyS6AgP44ZKWRuM5eTzR0dDppgEXi/PC+DO3lE0PVihzAOp9Huk94XX 18kRBicJTOjUoBJiXOOnKyUgh88NB1gGno+HpsofTo9QUHExr/g+tqpnP8jVBBnwME1RbvFp vfWMobIYIcHawny9jvvWkEI0Bi6rJ3Gyy3CluyiAhcfNnQ2Cm3QGyfNvuTzdWMJtjS5TH+x3 jkWi2ue6mnkF0MMoXmuOiT+tX57juYuodDA9GxekzpbqrH/BM3nYR/IUdaQX7ILoRxBNPzlh Dru4A6OCc1qwIA6zeQt4AOx+EY3HriVmJRytWfYlWuuY0024g+XR6JeH6DfsLaYQFF1D6UPi jfUKnTbY5FpjtWu96xBbge5Hajns7dNKukaol+Y2M1EFPN2DLVuEixHiTqVjDDW0P511orgX fWHFrwyk7BgMVvyFoX0Xc6Xd71lC45EQOqjM1oEDjgAAAAAAAA== --------------ms080702000005000509070200-- From owner-freebsd-stable@freebsd.org Wed May 1 03:17:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D4F61582553 for ; Wed, 1 May 2019 03:17:43 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id C8D3C7384E for ; Wed, 1 May 2019 03:17:41 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQT00M7I34MZ500@hades.sorbs.net> for freebsd-stable@freebsd.org; Tue, 30 Apr 2019 20:31:38 -0700 (PDT) Sun-Java-System-SMTP-Warning: Lines longer than SMTP allows found and truncated. Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Wed, 01 May 2019 13:17:36 +1000 Cc: freebsd-stable@freebsd.org Content-transfer-encoding: quoted-printable Message-id: References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> To: Karl Denninger X-Rspamd-Queue-Id: C8D3C7384E X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.21 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.59)[ip: (-1.55), ipnet: 72.12.192.0/19(-0.77), asn: 11114(-0.60), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.86)[-0.861,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 03:17:43 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 01 May 2019, at 12:37, Karl Denninger wrote: >=20 > On 4/30/2019 20:59, Michelle Sullivan wrote >>> On 01 May 2019, at 11:33, Karl Denninger wrote: >>>=20 >>>> On 4/30/2019 19:14, Michelle Sullivan wrote: >>>>=20 >>>> Michelle Sullivan >>>> http://www.mhix.org/ >>>> Sent from my iPad >>>>=20 >>> Nope. I'd much rather *know* the data is corrupt and be forced to >>> restore from backups than to have SILENT corruption occur and perhaps >>> screw me 10 years down the road when the odds are my backups have >>> long-since been recycled. >> Ahh yes the be all and end all of ZFS.. stops the silent corruption of da= ta.. but don=E2=80=99t install it on anything unless it=E2=80=99s server gra= de with backups and ECC RAM, but it=E2=80=99s good on laptops because it pro= tects you from silent corruption of your data when 10 years later the backup= s have long-since been recycled... umm is that not a circular argument? >>=20 >> Don=E2=80=99t get me wrong here.. and I know you (and some others are) zf= s in the DC with 10s of thousands in redundant servers and/or backups to kee= p your critical data corruption free =3D good thing. >>=20 >> ZFS on everything is what some say (because it prevents silent corruption= ) but then you have default policies to install it everywhere .. including h= ardware not equipped to function safely with it (in your own arguments) and y= et it=E2=80=99s still good because it will still prevent silent corruption e= ven though it relies on hardware that you can trust... umm say what? >>=20 >> Anyhow veered way way off (the original) topic... >>=20 >> Modest (part consumer grade, part commercial) suffered irreversible data l= oss because of a (very unusual, but not impossible) double power outage.. an= d no tools to recover the data (or part data) unless you have some form of b= ackup because the file system deems the corruption to be too dangerous to le= t you access any of it (even the known good bits) ... =20 >>=20 >> Michelle >=20 > IMHO you're dead wrong Michelle. I respect your opinion but disagree > vehemently. I guess we=E2=80=99ll have to agree to disagree then, but I think your attit= ude to pronounce me =E2=80=9Cdead wrong=E2=80=9D is short sighted, because i= t strikes of =E2=80=9CI=E2=80=99m right because ZFS is the answer to all pro= blems.=E2=80=9D .. I=E2=80=99ve been around in the industry long enough to s= ee a variety of issues... some disasters, some not so... I also should know better than to run without backups but financial constrai= nts precluded me.... as will for many non commercial people. >=20 > I run ZFS on both of my laptops under FreeBSD. Both have > non-power-protected SSDs in them. Neither is mirrored or Raidz-anything. >=20 > So why run ZFS instead of UFS? >=20 > Because a scrub will detect data corruption that UFS cannot detect *at all= .* I get it, I really do, but that balances out against, if you can=E2=80=99t r= ebuild it make sure you have (tested and working) backups and be prepared fo= r downtime when such corruption does occur. >=20 > It is a balance-of-harms test and you choose. I can make a very clean > argument that *greater information always wins*; that is, I prefer in > every case to *know* I'm screwed rather than not. I can defend against > being screwed with some amount of diligence but in order for that > diligence to be reasonable I have to know about the screwing in a > reasonable amount of time after it happens. Not disagreeing (and have not been.) >=20 > You may have never had silent corruption bite you. I have... but not with data on disks.. most of my silent corruption issues h= ave been with a layer or two above the hardware... like subversion commits o= verwriting previous commits without notification (damn I wish I could reliab= ly replicate it!) > I have had it happen > several times over my IT career. If that happens to you the odds are > that it's absolutely unrecoverable and whatever gets corrupted is > *gone.* Every drive corruption I have suffered in my career I have been able to reco= ver, all or partial data except where the hardware itself was totally hosed (= Ie clean room options only available)... even with brtfs.. yuk.. puck.. yuk.= . oh what a mess that was... still get nightmares on that one... but I sti= ll managed to get most of the data off... in fact I put it onto this machine= I currently have problems with.. so after the nightmare of brtfs looks like= zfs eventually nailed me. > The defensive measures against silent corruption require > retention of backup data *literally forever* for the entire useful life > of the information because from the point of corruption forward *the > backups are typically going to be complete and correct copies of the > corrupt data and thus equally worthless to what's on the disk itself.*=20 > With non-ZFS filesystems quite a lot of thought and care has to go into > defending against that, and said defense usually requires the active > cooperation of whatever software wrote said file in the first place Say what? =20 > (e.g. a database, etc.) So dbs (any?) talk actively to the file systems (any?) to actively prevent s= ilent corruption? Lol... I=E2=80=99m guessing you are actually talking about internal checks and bala= nces of data in the DB to ensure that dat retrieved from disk is not corrupt= /altered... you know like writing sha256 checksums of files you might downl= oad from the internet to ensure you got what you asked for and it wasn=E2=80= =99t changed/altered in transit. > If said software has no tools to "walk" said > data or if it's impractical to have it do so you're at severe risk of > being hosed. Umm what? I=E2=80=99m talking about a userland (libzfs) tool (Ie doesn=E2=80= =99t need the pool imported) such as zfs send (which requires the pool to be= imported - hence me not calling it a userland tool) to allow a sending of d= ata that can be found to other places where it can be either blindly recover= ed (corruption might be present) or can be used to locate files/paths etc th= at are known to be good (checksums match etc).. walk the structures, feed th= e data elsewhere where it can be examined/recovered... don=E2=80=99t alter i= t.... it=E2=80=99s a last resort tool when you don=E2=80=99t have working ba= ckups.. > Prior to ZFS there really wasn't any comprehensive defense > against this sort of event. There are a whole host of applications that > manipulate data that are absolutely reliant on that sort of thing not > happening (e.g. anything using a btree data structure) and recovery if > it *does* happen is a five-alarm nightmare if it's possible at all. In > the worst-case scenario you don't detect the corruption and the data > that has the pointer to it that gets corrupted is overwritten and=20 > destroyed. >=20 > A ZFS scrub on a volume that has no redundancy cannot *fix* that > corruption but it can and will detect it. So you=E2=80=99re advocating restore from backup for every corruption ... ok= ... > This puts a boundary on the > backups that I must keep in order to *not* have that happen. This is of > very high value to me and is why, even on systems without ECC memory and > without redundant disks, provided there is enough RAM to make it > reasonable (e.g. not on embedded systems I do development on with are > severely RAM-constrained) I run ZFS. >=20 > BTW if you've never had a UFS volume unlink all the blocks within a file > on an fsck and then recover them back into the free list after a crash > you're a rare bird indeed. If you think a corrupt ZFS volume is fun try > to get your data back from said file after that happens. Been there done that though with ext2 rather than UFS.. still got all my da= ta back... even though it was a nightmare.. >=20 > --=20 > Karl Denninger > karl@denninger.net > /The Market Ticker/ > /[S/MIME encrypted email preferred]/ From owner-freebsd-stable@freebsd.org Wed May 1 03:56:46 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04312158331C; Wed, 1 May 2019 03:56:46 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E73A974C1C; Wed, 1 May 2019 03:56:43 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id LgMEhBJ2TldkPLgMFhnFVa; Tue, 30 Apr 2019 21:56:36 -0600 X-Authority-Analysis: v=2.3 cv=Ko4zJleN c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=8nJEP1OIZ-IA:10 a=oexKYjalfGEA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=q2XNxBO46EK3obBdrt4A:9 a=wPNLvfGTeEIA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id 8BF6514E; Tue, 30 Apr 2019 20:56:33 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x413uX67074032; Tue, 30 Apr 2019 20:56:33 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x413uUnV068470; Tue, 30 Apr 2019 20:56:32 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201905010356.x413uUnV068470@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Miroslav Lachman <000.fbsd@quip.cz> cc: David Chisnall , FreeBSD Stable , freebsd-ports@freebsd.org, =?UTF-8?Q?Goran_Meki=c4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org Subject: Re: CFT: FreeBSD Package Base In-Reply-To: Message from Miroslav Lachman <000.fbsd@quip.cz> of "Wed, 01 May 2019 00:03:44 +0200." <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Date: Tue, 30 Apr 2019 20:56:30 -0700 X-CMAE-Envelope: MS4wfN/xmWRaWQf4f3J4LUJQu3pZlq2R+heRrUgbSIStI3jW7zX/5fMm4b0wrONS30PxYTM3L4AEceAbaUlH0j4Ssk2hgu0ilk7uSIb2ZQm3B34wLKJrGhxU HwWqFVGWz0FbohtGB8vZe+ZhlwMlOl+nV4bTEhDv1vj7LNpbO3NZmQOnMb1uHsrEYnimp8U3o5Ldv1Btob7Oz5spUCDh01syPNS8efzuut3lZbEWglh78Iul sPyzs8ESB8rbpCG1Etz3uwemvekBrPSWvbdp4F0pPhs+i985u/KunSKaEPyPP+EnNKeIq6a2IowFuvaH5ee+dvx8eKz/lEv74EXMf7LyIodLisYvO6SVjh3b 05bnRfDMx0Uroy8xz142FesMv0fY2sueoDnS5Afs/XbfU78s8tk0bOhtge4PbZSjTvqk5TyKv8onRizm9SpnzbpATzmeHLo6AdBYkyy7AWk5lCjRClOg0mbc 1/pTZMJvQHzSrCWq X-Rspamd-Queue-Id: E73A974C1C X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-3.63 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; HAS_XAW(0.00)[]; MX_GOOD(-0.01)[cached: spqr.komquats.com]; NEURAL_HAM_SHORT(-0.94)[-0.937,0]; RCPT_COUNT_SEVEN(0.00)[9]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.zen.spamhaus.org : 127.0.0.11]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_LOW(-0.10)[12.134.59.64.list.dnswl.org : 127.0.5.1]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; RCVD_COUNT_FIVE(0.00)[5]; REPLYTO_EQ_FROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-2.49)[ip: (-6.68), ipnet: 64.59.128.0/20(-3.21), asn: 6327(-2.46), country: CA(-0.09)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_NA(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 03:56:46 -0000 In message <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz>, Miroslav Lachman wri tes: > David Chisnall wrote on 2019/04/30 10:22: > > On 29/04/2019 21:12, Joe Maloney wrote: > >> With CFT version you chose to build, and package individual components > >> such as sendmail with a port option.  That does entirely solve the > >> problem of being able to reinstall sendmail after the fact without a > >> rebuild of the userland (base) port but perhaps base flavors could > >> solve that problem assuming flavors could extend beyond python. > > > > This sounds very much like local optimisation. It's now easy to create a > > custom base image.  Great.  But how do I express dependencies in ports > > on a specific base configuration? This is easy if I depend on a specific > > base package, but how does this work in your model?  For example, if I > > have a package that depends on a library that is an optional part of the > > base system, how do I express that pkg needs to either refuse to install > > it, or install a userland pkg that includes that library in place of my > > existing version as part of the install process? > > > > More importantly for the container use case, if I want to take a > > completely empty jail and do pkg ins nginx (for example), what does the > > maintainer of the nginx port need to do to express the minimum set of > > the base system that needs to be installed to allow nginx to work? > > > > One of the goals for the pkg base concept was to allow this kind of use > > case, easily creating a minimal environment required to run a single > > service. With a monolithic base package set, you're going to need some > > mechanism other than packages to express the specific base subset > > package that you need and I think that you need to justify why this > > mechanism is better than using small individual packages. > > Will it not be maintainer's nightmare to take care of all the > dependencies on the base packages for each port we have in the ports tree? No more than it is today. Remember, people have been doing this sort of thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and IBM can do it, I'm sure we can too. The dependency lists will be longer. We may require dependency lists that allow the choice of one of many prereqs or coreqs. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-stable@freebsd.org Wed May 1 04:16:30 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C8BE1584313 for ; Wed, 1 May 2019 04:16:30 +0000 (UTC) (envelope-from walterp@gmail.com) Received: from mail-it1-x142.google.com (mail-it1-x142.google.com [IPv6:2607:f8b0:4864:20::142]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DBD9375ACC for ; Wed, 1 May 2019 04:16:28 +0000 (UTC) (envelope-from walterp@gmail.com) Received: by mail-it1-x142.google.com with SMTP id l140so7112686itb.0 for ; Tue, 30 Apr 2019 21:16:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=IWke8/OuX9QTrnbHJiD1nMJDXKuDWxEIyLIzXJvb6cU=; b=UnvXw1f1vG1wSI4J7NjWaJ0LYnq1zRYe9cxHAOPSmvo0PFalN4wHkCbMWemWb3A1xM 5pYthQB6NoJylbJgd7L9trB6oROH+LV6yYwoThs5SWuAfhHiO3bc3vRn9W9ZJtnRRWKI l+7lynqgIu5FHBc2M1kA2nFE5XbUta35iU2EGRUtIu6sDmIDKysohSqSdxGhTM2OmQ7n wDmcNxUqgaw9TCbRphzisPru5CbjgGeFpn25o3BFNDGHEduD1jlGd25nKfNSMiD/7rNe TmlckBWhs7ItJujvEtFY/YPslaf1Swl40DgFUcksSm3X/O2QPwPq+EJZV5DFQgfRcg2K HgyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=IWke8/OuX9QTrnbHJiD1nMJDXKuDWxEIyLIzXJvb6cU=; b=HrJHSN4BDjqqc8OJgoYtzl/8IjPqL+v/lU5qKEnFeu/Wi8d2RC2hFY46xrUCfyNJXR XLhLrALJymUglI4jjqhBmWN9YIR5Rx58enMxMvy1bk0Ara/U7U7a0XJGP8Oy3mtgfRyX Kxu/O6w4KP0dH0E55gqABTa69fA6cnEjoPsSzs9CNe1QQvZPOv9e9cCgJKtmMKMWFDna wmS0eYdxDBKZX35LZIM+MdQPTQOOnw5aj3/YSmHn1gpxXrm8p0HUmBlYaCnhbzsFjTlZ 1+QFfh6x3TiTIQi466MaEOlHA2sKNaLlZEDZsoYg0fCJyhWONQabhKHPHIiwB6wWgtuD T8NQ== X-Gm-Message-State: APjAAAXomMDC/pkOzuFKA9fBmZIt9wBsiHRhU9YKZvsAOln06X9cA8ES AtHrzAUbbR1oHUa2OVZhiA6MU7JfIhrNJP/ibr4rzg== X-Google-Smtp-Source: APXvYqz+d56yEByn1sIg6sH2u/ie8qgwBnWrkgd3mAaPeyYBKy4KgtBCCZc+/KXJby+6ath/346aVy17lzad2Snze5U= X-Received: by 2002:a24:7b41:: with SMTP id q62mr678826itc.22.1556684187706; Tue, 30 Apr 2019 21:16:27 -0700 (PDT) MIME-Version: 1.0 References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> In-Reply-To: From: Walter Parker Date: Tue, 30 Apr 2019 21:16:13 -0700 Message-ID: Subject: Re: ZFS... To: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: DBD9375ACC X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=UnvXw1f1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of walterp@gmail.com designates 2607:f8b0:4864:20::142 as permitted sender) smtp.mailfrom=walterp@gmail.com X-Spamd-Result: default: False [-4.31 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.94)[-0.935,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.82)[ip: (1.39), ipnet: 2607:f8b0::/32(-3.18), asn: 15169(-2.24), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[2.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 04:16:30 -0000 On Tue, Apr 30, 2019 at 8:19 PM Michelle Sullivan wrote: > > > Michelle Sullivan > http://www.mhix.org/ > Sent from my iPad > > > On 01 May 2019, at 12:37, Karl Denninger wrote: > > > > On 4/30/2019 20:59, Michelle Sullivan wrote > >>> On 01 May 2019, at 11:33, Karl Denninger wrote: > >>> > >>>> On 4/30/2019 19:14, Michelle Sullivan wrote: > >>>> > >>>> Michelle Sullivan > >>>> http://www.mhix.org/ > >>>> Sent from my iPad > >>>> > >>> Nope. I'd much rather *know* the data is corrupt and be forced to > >>> restore from backups than to have SILENT corruption occur and perhaps > >>> screw me 10 years down the road when the odds are my backups have > >>> long-since been recycled. > >> Ahh yes the be all and end all of ZFS.. stops the silent corruption of > data.. but don=E2=80=99t install it on anything unless it=E2=80=99s serve= r grade with > backups and ECC RAM, but it=E2=80=99s good on laptops because it protects= you from > silent corruption of your data when 10 years later the backups have > long-since been recycled... umm is that not a circular argument? > >> > ZFS works fine on systems with ECC. According to one of the original Sun authors of ZFS, the scrub of death is a myth. A non-ECC running ZFS is not more risky than one running UFS. As far as backups go, you should have backups for your important data. This is true for any filesystem. Home users have been using backups for decades. For anybody that has important data, not having a backup is a false economy. Odds are that most people you don't have backup can afford them if they plan and allocate money for this task (as 8TB USB drives are now at Costco for ~$140, getting one of.should not be much of an issue). Tarsnap is great and cheap online backup system. Silent data corruption is a thing. CERN tested it 10-15 years on brand new, high end production hardware. They run a test running on 3000 new servers run for 1 week. They found 147 silent data corruption errors on the server farm (found due to ZFS error checking). > >> Don=E2=80=99t get me wrong here.. and I know you (and some others are)= zfs in > the DC with 10s of thousands in redundant servers and/or backups to keep > your critical data corruption free =3D good thing. > >> > >> ZFS on everything is what some say (because it prevents silent > corruption) but then you have default policies to install it everywhere .= . > including hardware not equipped to function safely with it (in your own > arguments) and yet it=E2=80=99s still good because it will still prevent = silent > corruption even though it relies on hardware that you can trust... umm s= ay > what? > >> I run ZFS on embedded firewalls, it has worked fine for years. ZFS by default is a good idea on systems that ZFS built in (such as FreeBSD and SmartOS). There are great things you do with boot environments and volume management is much nicer. Don't let the blowhards that say ZFS should only be run on server grade (ie ECC) hardware spook you. It works as well as UFS would on any system with at least 1GB of RAM (but I'd suggest getting at least 2GB if you can't get 4GB), you just need to adjust a few memory parameters. > >> Anyhow veered way way off (the original) topic... > >> > >> Modest (part consumer grade, part commercial) suffered irreversible > data loss because of a (very unusual, but not impossible) double power > outage.. and no tools to recover the data (or part data) unless you have > some form of backup because the file system deems the corruption to be to= o > dangerous to let you access any of it (even the known good bits) ... > >> > >> Michelle > > > > IMHO you're dead wrong Michelle. I respect your opinion but disagree > > vehemently. > > I guess we=E2=80=99ll have to agree to disagree then, but I think your at= titude to > pronounce me =E2=80=9Cdead wrong=E2=80=9D is short sighted, because it st= rikes of =E2=80=9CI=E2=80=99m > right because ZFS is the answer to all problems.=E2=80=9D .. I=E2=80=99ve= been around in > the industry long enough to see a variety of issues... some disasters, so= me > not so... > > I also should know better than to run without backups but financial > constraints precluded me.... as will for many non commercial people. > > > > > I run ZFS on both of my laptops under FreeBSD. Both have > > non-power-protected SSDs in them. Neither is mirrored or Raidz-anythin= g. > > > > So why run ZFS instead of UFS? > > > > Because a scrub will detect data corruption that UFS cannot detect *at > all.* > > I get it, I really do, but that balances out against, if you can=E2=80=99= t rebuild > it make sure you have (tested and working) backups and be prepared for > downtime when such corruption does occur. > > > > > It is a balance-of-harms test and you choose. I can make a very clean > > argument that *greater information always wins*; that is, I prefer in > > every case to *know* I'm screwed rather than not. I can defend against > > being screwed with some amount of diligence but in order for that > > diligence to be reasonable I have to know about the screwing in a > > reasonable amount of time after it happens. > > Not disagreeing (and have not been.) > > > > > You may have never had silent corruption bite you. > > I have... but not with data on disks.. most of my silent corruption > issues have been with a layer or two above the hardware... like subversio= n > commits overwriting previous commits without notification (damn I wish I > could reliably replicate it!) > > > > I have had it happen > > several times over my IT career. If that happens to you the odds are > > that it's absolutely unrecoverable and whatever gets corrupted is > > *gone.* > > Every drive corruption I have suffered in my career I have been able to > recover, all or partial data except where the hardware itself was totally > hosed (Ie clean room options only available)... even with brtfs.. yuk.. > puck.. yuk.. oh what a mess that was... still get nightmares on that > one... but I still managed to get most of the data off... in fact I put = it > onto this machine I currently have problems with.. so after the nightmare > of brtfs looks like zfs eventually nailed me. > > > > The defensive measures against silent corruption require > > retention of backup data *literally forever* for the entire useful life > > of the information because from the point of corruption forward *the > > backups are typically going to be complete and correct copies of the > > corrupt data and thus equally worthless to what's on the disk itself.* > > With non-ZFS filesystems quite a lot of thought and care has to go into > > defending against that, and said defense usually requires the active > > cooperation of whatever software wrote said file in the first place > > Say what? > > > (e.g. a database, etc.) > > So dbs (any?) talk actively to the file systems (any?) to actively preven= t > silent corruption? > > Lol... > > I=E2=80=99m guessing you are actually talking about internal checks and b= alances > of data in the DB to ensure that dat retrieved from disk is not > corrupt/altered... you know like writing sha256 checksums of files you > might download from the internet to ensure you got what you asked for and > it wasn=E2=80=99t changed/altered in transit. > > > If said software has no tools to "walk" said > > data or if it's impractical to have it do so you're at severe risk of > > being hosed. > > Umm what? I=E2=80=99m talking about a userland (libzfs) tool (Ie doesn= =E2=80=99t need the > pool imported) such as zfs send (which requires the pool to be imported - > hence me not calling it a userland tool) to allow a sending of data that > can be found to other places where it can be either blindly recovered > (corruption might be present) or can be used to locate files/paths etc th= at > are known to be good (checksums match etc).. walk the structures, feed th= e > data elsewhere where it can be examined/recovered... don=E2=80=99t alter = it.... > it=E2=80=99s a last resort tool when you don=E2=80=99t have working backu= ps.. > > > Prior to ZFS there really wasn't any comprehensive defense > > against this sort of event. There are a whole host of applications tha= t > > manipulate data that are absolutely reliant on that sort of thing not > > happening (e.g. anything using a btree data structure) and recovery if > > it *does* happen is a five-alarm nightmare if it's possible at all. In > > the worst-case scenario you don't detect the corruption and the data > > that has the pointer to it that gets corrupted is overwritten and > > destroyed. > > > > A ZFS scrub on a volume that has no redundancy cannot *fix* that > > corruption but it can and will detect it. > > So you=E2=80=99re advocating restore from backup for every corruption ...= ok... > > > > This puts a boundary on the > > backups that I must keep in order to *not* have that happen. This is o= f > > very high value to me and is why, even on systems without ECC memory an= d > > without redundant disks, provided there is enough RAM to make it > > reasonable (e.g. not on embedded systems I do development on with are > > severely RAM-constrained) I run ZFS. > > > > BTW if you've never had a UFS volume unlink all the blocks within a fil= e > > on an fsck and then recover them back into the free list after a crash > > you're a rare bird indeed. If you think a corrupt ZFS volume is fun tr= y > > to get your data back from said file after that happens. > > Been there done that though with ext2 rather than UFS.. still got all my > data back... even though it was a nightmare.. > > > > > > -- > > Karl Denninger > > karl@denninger.net > > /The Market Ticker/ > > /[S/MIME encrypted email preferred]/ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > --=20 The greatest dangers to liberty lurk in insidious encroachment by men of zeal, well-meaning but without understanding. -- Justice Louis D. Brandei= s From owner-freebsd-stable@freebsd.org Wed May 1 08:09:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4AC821599430 for ; Wed, 1 May 2019 08:09:40 +0000 (UTC) (envelope-from mw@wipp.bayern) Received: from virgo.wipp.bayern (virgo.wipp.bayern [IPv6:2a01:4f8:141:5249::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD9FF6AA45 for ; Wed, 1 May 2019 08:09:39 +0000 (UTC) (envelope-from mw@wipp.bayern) Received: from [IPv6:2a02:8106:209:e43c:e9c1:f8c5:a63f:21eb] (unknown [IPv6:2a02:8106:209:e43c:e9c1:f8c5:a63f:21eb]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: mw@wipp.bayern) by virgo.wipp.bayern (Postfix) with ESMTPSA id 0A9DCAEBAA; Wed, 1 May 2019 10:09:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wipp.bayern; s=dkim201411; t=1556698170; bh=Wk91GJQBsUFXI0aJ7CIHKjPSRoFArvru+FQK6XBUmnI=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=MfJ/I+dFpcV4+DxEiR22GYkZSiouKusimaZ5BzKDERg+4crS6NiwSs6y1/u0hEzY5 YdPbi/mc4vGpxDjZNlEyGIfyfCUBYGXxFn9kpafZ2TyrbxjEmikuYgEWo5gHfzev2O DY5gFyeS8012Q6M8Kw6V7bwSqNHtTDjwtTeQuQ+E= X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.101.2 at gemini.wipp.bayern Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: Issues starting unbound on boot From: Markus Wipp X-Mailer: iPhone Mail (16E227) In-Reply-To: <20190430220220.GC74417@server.rulingia.com> Date: Wed, 1 May 2019 10:09:27 +0200 Cc: Markus Wipp , "freebsd-stable@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <9108FAC6-6F7F-42AE-8F1E-829EBFF04397@wipp.bayern> References: <9E319697-B182-49ED-AB44-F3BBC87EEE1B@bluebird-ag.com> <20190430220220.GC74417@server.rulingia.com> To: Peter Jeremy X-Rspamd-Queue-Id: CD9FF6AA45 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.95 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.95)[-0.945,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 08:09:40 -0000 > On 1. May 2019, at 00:02, Peter Jeremy wrote: >=20 >> On 2019-Apr-30 19:44:36 +0000, Markus Wipp wrote= : >> I currently face an issue, where I don=E2=80=99t know further on why this= happens and what I could do about it. >> I hope that this is the correct list to ask my question. If not please le= t me know where else I might try my luck. >> I installed unbound from ports, configured it and can start / stop it fro= m command line with service unbound start without any problems. >> But whenever I reboot the machine it just doesn=E2=80=99t get started. Th= e only information I was able to find out so far can be found in /var/log/me= ssages: >> root: /etc/rc: WARNING: failed to start unbound >=20 > I have seen unbound fail to start for a variety of reasons but in all case= s, it has > written a useful hint to the console. Can you confirm that it's not writi= ng anything > to your console. Are you able to share your configuration? >=20 > --=20 > Peter Jeremy Hi Peter, many thanks to you! By looking at the console output I was able to determine= that the interface/up I bound unbound to was not yet available during the b= oot process (it was tun0 for my VPN). After finding this cause it was just a= matter of ensuring tun0 gets created and configured as cloned interface. So now everything works as it should! Thanks again to all! Have a nice day! Markus=20= From owner-freebsd-stable@freebsd.org Wed May 1 08:31:47 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62FE4159C6F5; Wed, 1 May 2019 08:31:47 +0000 (UTC) (envelope-from SRS0=6TW8=TB=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1B5F76B63C; Wed, 1 May 2019 08:31:45 +0000 (UTC) (envelope-from SRS0=6TW8=TB=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 655DD28423; Wed, 1 May 2019 10:31:43 +0200 (CEST) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 5C08428422; Wed, 1 May 2019 10:31:40 +0200 (CEST) Subject: Re: CFT: FreeBSD Package Base To: Cy Schubert Cc: FreeBSD Stable , freebsd-pkg@freebsd.org, =?UTF-8?Q?Goran_Meki=c4=87?= , freebsd-hackers@freebsd.org, David Chisnall , FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-ports@freebsd.org References: <201905010356.x413uUnV068470@slippy.cwsent.com> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <56b15cc3-4605-f693-4ad6-188ea865820d@quip.cz> Date: Wed, 1 May 2019 10:31:41 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: <201905010356.x413uUnV068470@slippy.cwsent.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 1B5F76B63C X-Spamd-Bar: ++++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [6.61 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.98)[0.983,0]; IP_SCORE(0.94)[ip: (0.49), ipnet: 94.124.104.0/21(0.24), asn: 42000(3.88), country: CZ(0.08)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(1.00)[0.996,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: elsa.codelab.cz]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; RCPT_COUNT_SEVEN(0.00)[9]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=6TW8=TB=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=6TW8=TB=quip.cz=000.fbsd@elsa.codelab.cz]; SUSPICIOUS_RECIPS(1.50)[] X-Spam: Yes X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 08:31:47 -0000 Cy Schubert wrote on 2019/05/01 05:56: > In message <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz>, Miroslav > Lachman wri > tes: >> David Chisnall wrote on 2019/04/30 10:22: >>> On 29/04/2019 21:12, Joe Maloney wrote: >>>> With CFT version you chose to build, and package individual components >>>> such as sendmail with a port option.  That does entirely solve the >>>> problem of being able to reinstall sendmail after the fact without a >>>> rebuild of the userland (base) port but perhaps base flavors could >>>> solve that problem assuming flavors could extend beyond python. >>> >>> This sounds very much like local optimisation. It's now easy to create a >>> custom base image.  Great.  But how do I express dependencies in ports >>> on a specific base configuration? This is easy if I depend on a specific >>> base package, but how does this work in your model?  For example, if I >>> have a package that depends on a library that is an optional part of the >>> base system, how do I express that pkg needs to either refuse to install >>> it, or install a userland pkg that includes that library in place of my >>> existing version as part of the install process? >>> >>> More importantly for the container use case, if I want to take a >>> completely empty jail and do pkg ins nginx (for example), what does the >>> maintainer of the nginx port need to do to express the minimum set of >>> the base system that needs to be installed to allow nginx to work? >>> >>> One of the goals for the pkg base concept was to allow this kind of use >>> case, easily creating a minimal environment required to run a single >>> service. With a monolithic base package set, you're going to need some >>> mechanism other than packages to express the specific base subset >>> package that you need and I think that you need to justify why this >>> mechanism is better than using small individual packages. >> >> Will it not be maintainer's nightmare to take care of all the >> dependencies on the base packages for each port we have in the ports tree? > > No more than it is today. Remember, people have been doing this sort of > thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and > IBM can do it, I'm sure we can too. The dependency lists will be > longer. We may require dependency lists that allow the choice of one of > many prereqs or coreqs. They are experts and they are paid for their work. I am not. I am maintaining a few packages and the reality is I don't know what they need in base. Till these days I don't care about this kind of dependency. I am not system developer or programmer and I think there are more than just me who see this as a kind of problem. So in this case, pkg base gives me nothing but more work on those packages. Miroslav Lachman From owner-freebsd-stable@freebsd.org Wed May 1 13:55:58 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F15115979B9 for ; Wed, 1 May 2019 13:55:58 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gromit.dlib.vt.edu", Issuer "Chumby Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 24F5B76CE8 for ; Wed, 1 May 2019 13:55:57 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from mather.gromit23.net (c-98-244-101-97.hsd1.va.comcast.net [98.244.101.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id 7B3C32EF; Wed, 1 May 2019 09:55:55 -0400 (EDT) Content-Type: text/plain; charset=utf-8; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Paul Mather In-Reply-To: Date: Wed, 1 May 2019 09:55:54 -0400 Cc: freebsd-stable Content-Transfer-Encoding: 8bit Message-Id: <713FB681-94F6-4F3A-AC69-417362D29B6E@gromit.dlib.vt.edu> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> To: Michelle Sullivan X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 24F5B76CE8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dmarc=fail reason="" header.from=vt.edu (policy=none) X-Spamd-Result: default: False [-3.12 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: chumby.dlib.vt.edu]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.98)[-0.979,0]; RECEIVED_SPAMHAUS_PBL(0.00)[97.101.244.98.zen.spamhaus.org : 127.0.0.10]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1312, ipnet:128.173.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-1.08)[ip: (-2.76), ipnet: 128.173.0.0/16(-1.38), asn: 1312(-1.19), country: US(-0.06)]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[vt.edu : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 13:55:58 -0000 On Apr 30, 2019, at 8:14 PM, Michelle Sullivan wrote: > > > Michelle Sullivan > http://www.mhix.org/ > Sent from my iPad > >> On 01 May 2019, at 01:15, Karl Denninger wrote: >> >> >> IMHO non-ECC memory systems are ok for personal desktop and laptop >> machines where loss of stored data requiring a restore is acceptable >> (assuming you have a reasonable backup paradigm for same) but not for >> servers and *especially* not for ZFS storage. I don't like the price of >> ECC memory and I really don't like Intel's practices when it comes to >> only enabling ECC RAM on their "server" class line of CPUs either but it >> is what it is. Pay up for the machines where it matters. > > And the irony is the FreeBSD policy to default to zfs on new installs > using the complete drive.. even when there is only one disk available and > regardless of the cpu or ram class... with one usb stick I have around > here it attempted to use zfs on one of my laptops. ZFS has MUCH more to recommend it than just the "self-healing" properties discussed in this thread. Its pooled storage model, good administration and snapshot/clone support (enabling features such as boot environments) make it preferable over UFS as a default file system. You can even gain the benfits of self-healing (for silent data corruption) for single-drive systems via "copies=2" or "copies=3" on file sets. > Damned if you do, damned if you don’t comes to mind. Not really. Nobody is forcing anyone only to use ZFS as a choice of file system. As you say above, it is a default (a very sensible one, IMHO, but even then, it's not really a default). If you believe ZFS is not right for you, do a UFS installation instead. BTW, I disagree that you need top-notch server-grade hardware to use ZFS. Its design embodies the notion of being distrustful of the hardware on which it is running, and it is targeted to be able to survive consumer hardware (as has been pointed out elsewhere in this thread), e.g., HBAs without BBUs. I am using ZFS on a Raspberry Pi with an external USB drive. How's that for server-grade hardware? :-) Cheers, Paul. From owner-freebsd-stable@freebsd.org Wed May 1 14:06:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51CA81597D6C for ; Wed, 1 May 2019 14:06:40 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id DFFC777446 for ; Wed, 1 May 2019 14:06:38 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-transfer-encoding: 8BIT Content-type: text/plain; charset=UTF-8; format=flowed Received: from isux.com (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQT00MNUX67Z500@hades.sorbs.net> for freebsd-stable@freebsd.org; Wed, 01 May 2019 07:20:33 -0700 (PDT) Subject: Re: ZFS... To: Paul Mather Cc: freebsd-stable References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> <713FB681-94F6-4F3A-AC69-417362D29B6E@gromit.dlib.vt.edu> From: Michelle Sullivan Message-id: <165446da-7b9a-26a3-e92f-41095382232e@sorbs.net> Date: Thu, 02 May 2019 00:06:33 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-reply-to: <713FB681-94F6-4F3A-AC69-417362D29B6E@gromit.dlib.vt.edu> X-Rspamd-Queue-Id: DFFC777446 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.76 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.94)[-0.945,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; IP_SCORE(-0.56)[ip: (-1.44), ipnet: 72.12.192.0/19(-0.72), asn: 11114(-0.56), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 14:06:40 -0000 Paul Mather wrote: > On Apr 30, 2019, at 8:14 PM, Michelle Sullivan > wrote: > >> >> >> Michelle Sullivan >> http://www.mhix.org/ >> Sent from my iPad >> >>> On 01 May 2019, at 01:15, Karl Denninger wrote: >>> >>> >>> IMHO non-ECC memory systems are ok for personal desktop and laptop >>> machines where loss of stored data requiring a restore is acceptable >>> (assuming you have a reasonable backup paradigm for same) but not for >>> servers and *especially* not for ZFS storage. I don't like the >>> price of >>> ECC memory and I really don't like Intel's practices when it comes to >>> only enabling ECC RAM on their "server" class line of CPUs either >>> but it >>> is what it is. Pay up for the machines where it matters. >> >> And the irony is the FreeBSD policy to default to zfs on new installs >> using the complete drive.. even when there is only one disk available >> and regardless of the cpu or ram class... with one usb stick I have >> around here it attempted to use zfs on one of my laptops. > > > ZFS has MUCH more to recommend it than just the "self-healing" > properties discussed in this thread. Its pooled storage model, good > administration and snapshot/clone support (enabling features such as > boot environments) make it preferable over UFS as a default file > system. You can even gain the benfits of self-healing (for silent > data corruption) for single-drive systems via "copies=2" or "copies=3" > on file sets. > > >> Damned if you do, damned if you don’t comes to mind. > > > Not really. Nobody is forcing anyone only to use ZFS as a choice of > file system. As you say above, it is a default (a very sensible one, > IMHO, but even then, it's not really a default). If you believe ZFS > is not right for you, do a UFS installation instead. > > BTW, I disagree that you need top-notch server-grade hardware to use > ZFS. Its design embodies the notion of being distrustful of the > hardware on which it is running, and it is targeted to be able to > survive consumer hardware (as has been pointed out elsewhere in this > thread), e.g., HBAs without BBUs. > > I am using ZFS on a Raspberry Pi with an external USB drive. How's > that for server-grade hardware? :-) > Was I drunk posting again? I thought others were advocating that server grade hardware was suitable for ZFS and if you are using consumer grade, you get what you pay for and dont blame ZFS etc.. This is an interesting issue... 2 thoughts of mind... ZFS safe to use on COnsumer hardware or not? ECC necessary or not? "The data on disk is always right" or not? .. it can't be all of the above by the very nature of the arguments that all seem to be against me, but not against each other... even though they are directly in conflict (and I have seen this on other ZFS lists... usually just before or after the justification for no 'FSCK for ZFS' (which after looking deeply into how ZFS works, I mostly agree with - which I stated earlier) - though a 'ZFS walk' tool may be the compromise that satisfies those who believe that an FSCK should be available and usually have no idea why it probably can never happen.... I will point out that someone from this thread messaged me this: https://www.klennet.com/zfs-recovery/default.aspx - which seems to be exactly what I'm talking about - a 'zfs walk' sorta told however... it's winblows only ... but if it does what it says on the packet.. this would probably be the missing link that would appease most ZFS detractors and people like me - who think ZFS is good for those with server grade hardware, but really not a good idea for the general linux user... :) (*waits for the flames*) -- Michelle Sullivan http://www.mhix.org/ From owner-freebsd-stable@freebsd.org Wed May 1 14:28:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31A631598425 for ; Wed, 1 May 2019 14:28:54 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.49.70]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "gromit.dlib.vt.edu", Issuer "Chumby Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 785AE80158 for ; Wed, 1 May 2019 14:28:53 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from mather.gromit23.net (c-98-244-101-97.hsd1.va.comcast.net [98.244.101.97]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id 794112F3; Wed, 1 May 2019 10:28:46 -0400 (EDT) Content-Type: text/plain; charset=utf-8; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Paul Mather In-Reply-To: Date: Wed, 1 May 2019 10:28:45 -0400 Cc: freebsd-stable Content-Transfer-Encoding: 8bit Message-Id: <7DBA7907-BE8F-4944-9A71-86E5AC1B85CA@gromit.dlib.vt.edu> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> To: Michelle Sullivan X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 785AE80158 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dmarc=fail reason="" header.from=vt.edu (policy=none) X-Spamd-Result: default: False [-3.07 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[vt.edu : No valid SPF, No valid DKIM,none]; RECEIVED_SPAMHAUS_PBL(0.00)[97.101.244.98.zen.spamhaus.org : 127.0.0.10]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-1.03)[ip: (-2.63), ipnet: 128.173.0.0/16(-1.32), asn: 1312(-1.14), country: US(-0.06)]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[chumby.dlib.vt.edu,gromit.dlib.vt.edu]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; R_SPF_NA(0.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.982,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1312, ipnet:128.173.0.0/16, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 14:28:54 -0000 On Apr 30, 2019, at 11:17 PM, Michelle Sullivan wrote: >> I have had it happen >> several times over my IT career. If that happens to you the odds are >> that it's absolutely unrecoverable and whatever gets corrupted is >> *gone.* > > Every drive corruption I have suffered in my career I have been able to > recover, all or partial data except where the hardware itself was totally > hosed (Ie clean room options only available)... even with brtfs.. yuk.. > puck.. yuk.. oh what a mess that was... still get nightmares on that > one... but I still managed to get most of the data off... in fact I put > it onto this machine I currently have problems with.. so after the > nightmare of brtfs looks like zfs eventually nailed me. It sounds from reading this thread that FreeBSD's built-in tools for ZFS recovery were insufficient for the corruption your pool suffered. Have you looked at the digital forensics realm to see whether those tools might help you? This article claims to extend The Sleuth Kit to support pooled storage such as ZFS, and they even describe recovering the bulk of an image file from a pool that has a disk missing (Evaluation Section, "Scenario C: reconstructing an incomplete pool"): "Extending The Sleuth Kit and its underlying model for pooled storage file system forensic analysis" https://www.sciencedirect.com/science/article/pii/S1742287617301901 >> If said software has no tools to "walk" said >> data or if it's impractical to have it do so you're at severe risk of >> being hosed. > > Umm what? I’m talking about a userland (libzfs) tool (Ie doesn’t need > the pool imported) such as zfs send (which requires the pool to be > imported - hence me not calling it a userland tool) to allow a sending of > data that can be found to other places where it can be either blindly > recovered (corruption might be present) or can be used to locate > files/paths etc that are known to be good (checksums match etc).. walk > the structures, feed the data elsewhere where it can be > examined/recovered... don’t alter it.... it’s a last resort tool when you > don’t have working backups.. See above. >> BTW if you've never had a UFS volume unlink all the blocks within a file >> on an fsck and then recover them back into the free list after a crash >> you're a rare bird indeed. If you think a corrupt ZFS volume is fun try >> to get your data back from said file after that happens. > > Been there done that though with ext2 rather than UFS.. still got all my > data back... even though it was a nightmare.. Is that an implication that had all your data been on UFS (or ext2:) this time around you would have got it all back? (I've got that impression through this thread from things you've written.) That sort of makes it sound like UFS is bulletproof to me. There are levels of corruption. Maybe what you suffered would have taken down UFS, too? I guess there's no way to know unless there's some way you can recreate exactly the circumstances that took down your original system (but this time your data on UFS). ;-) Cheers, Paul. From owner-freebsd-stable@freebsd.org Wed May 1 14:53:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 769D71598E53 for ; Wed, 1 May 2019 14:53:28 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 43F9E810EF for ; Wed, 1 May 2019 14:53:27 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII; format=flowed Received: from isux.com (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQT00MOBZC9Z500@hades.sorbs.net> for freebsd-stable@freebsd.org; Wed, 01 May 2019 08:07:22 -0700 (PDT) Subject: Re: ZFS... To: Paul Mather Cc: freebsd-stable References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> <7DBA7907-BE8F-4944-9A71-86E5AC1B85CA@gromit.dlib.vt.edu> From: Michelle Sullivan Message-id: <5c458075-351f-6eb6-44aa-1bd268398343@sorbs.net> Date: Thu, 02 May 2019 00:53:23 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-reply-to: <7DBA7907-BE8F-4944-9A71-86E5AC1B85CA@gromit.dlib.vt.edu> X-Rspamd-Queue-Id: 43F9E810EF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.77 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.54)[ip: (-1.39), ipnet: 72.12.192.0/19(-0.70), asn: 11114(-0.54), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.97)[-0.973,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 14:53:28 -0000 Paul Mather wrote: > On Apr 30, 2019, at 11:17 PM, Michelle Sullivan > wrote: > >> Been there done that though with ext2 rather than UFS.. still got >> all my data back... even though it was a nightmare.. > > > Is that an implication that had all your data been on UFS (or ext2:) > this time around you would have got it all back? (I've got that > impression through this thread from things you've written.) That sort > of makes it sound like UFS is bulletproof to me. Its definitely not (and far from it) bullet proof - however when the data on disk is not corrupt I have managed to recover it - even if it has been a nightmare - no structure - all files in lost+found etc... or even resorting to r-studio in the even of lost raid information etc.. > > There are levels of corruption. Maybe what you suffered would have > taken down UFS, too? Pretty sure not - and even if it would have - with the files intact I have always been able to recover them... r-studio being the last resort. > I guess there's no way to know unless there's some way you can > recreate exactly the circumstances that took down your original system > (but this time your data on UFS). ;-) True. This case - from what my limited knowledge has managed to fathom is a spacemap has become corrupt due to partial write during the hard power failure. This was the second hard outage during the resilver process following a drive platter failure (on a ZRAID2 - so single platter failure should be completely recoverable all cases - except hba failure or other corruption which does not appear to be the case).. the spacemap fails checksum (no surprises there being that it was part written) however it cannot be repaired (for what ever reason))... how I get that this is an interesting case... one cannot just assume anything about the corrupt spacemap... it could be complete and just the checksum is wrong, it could be completely corrupt and ignorable.. but what I understand of ZFS (and please watchers chime in if I'm wrong) the spacemap is just the freespace map.. if corrupt or missing one cannot just 'fix it' because there is a very good chance that the fix would corrupt something that is actually allocated and therefore the best solution would be (to "fix it") would be consider it 100% full and therefore 'dead space' .. but zfs doesn't do that - probably a good thing - the result being that a drive that is supposed to be good (and zdb reports some +36m objects there) becomes completely unreadable ... my thought (desire/want) on a 'walk' tool would be a last resort tool that could walk the datasets and send them elsewhere (like zfs send) so that I could create a new pool elsewhere and send the data it knows about to another pool and then blow away the original - if there are corruptions or data missing, thats my problem it's a last resort.. but in the case the critical structures become corrupt it means a local recovery option is enabled.. it means that if the data is all there and the corruption is just a spacemap one can transfer the entire drive/data to a new pool whilst the original host is rebuilt... this would *significantly* help most people with large pools that have to blow them away and re-create the pools because of errors/corruptions etc... and with the addition of 'rsync' (the checksumming of files) it would be trivial to just 'fix' the data corrupted or missing from a mirror host rather than transferring the entire pool from (possibly) offsite.... Regards, -- Michelle Sullivan http://www.mhix.org/ From owner-freebsd-stable@freebsd.org Wed May 1 16:16:18 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C28ED159AD78 for ; Wed, 1 May 2019 16:16:18 +0000 (UTC) (envelope-from chrcoluk@gmail.com) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4548883E95 for ; Wed, 1 May 2019 16:16:17 +0000 (UTC) (envelope-from chrcoluk@gmail.com) Received: by mail-wr1-x431.google.com with SMTP id l2so9475378wrb.9 for ; Wed, 01 May 2019 09:16:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=20TjVBDZVykQib9kia6+b1RIkj1ggvLaCI1r7fu0MqA=; b=TMHNz9J4mPRLjh4QXjeMy6i+djDZG7lTtEiSpGGcoXzh0AgvBIEXTDtzWuySYNR7S/ Oom29Q8ROXb6q7od7lsPNA1jS3Sma4jE9OW+6G7ef5KvaysBNSx2CRqMCSzLNh6s+YqJ 2h68HZC42Ko2KPqBH7fNYkn64wpl9xp13uVz2RPTxtPhaUQ+uLxsnqmUWuDSGYAFi5c5 G0Wk45zyngOQvxL1SGLMwE2iKjcc7MhdyublDaPzEXm27A8jioHfpiAeQMQWAsFgTIoH gti94HKIe4Pk5hiCuuP1NmYsUF0GjotzAduM7BrmFV29WzoFkEq9RTv4OFw/N3YpP91F O9ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=20TjVBDZVykQib9kia6+b1RIkj1ggvLaCI1r7fu0MqA=; b=lwlW1Z2FRxmKXgsAjlWxeZJAN6qen4KFRpzjcBmAOv9Wy6WJdUpXO5n24/cbSAOfl/ m6mZEGg5QmYBuFPaveULH62yMpSH89U4dBd+NX60zNDdqdksHiRXMQu4XCE1InrTlpTS z3CIyjZYa08U7iIBO6h5ckHs6rzvP0MOwmv5C+pzMph45W6lOyILMJhniC93H1k4FEgj JdLv8SXiSSjFlt1SlFZd5F54XAsGuqTmGnevRIB0Vw3mcLoqJfJUzthnsplfHyTSIfVK QEGzmaRhLj1SCQOjEOXOMXZ8ahdjUe8n6Jq1XvEKE1pHl80hc65yL07wnVPToH1o/JwS DwCg== X-Gm-Message-State: APjAAAVOfR3+99EJHWVpHw8fNPCBM8akrkp4aCHAz2hXAQaS/fy8lWyq fVqpD7g/Iipwc9uEbc5PEqibyMTxSp2Zs5IVWfN9Ab2f X-Google-Smtp-Source: APXvYqzpF5SF1I28mIl9oFS4M7UtkV34Oxc5Fshj0VXHtZKOHYo61ygqGqEMEG91/0iFjvuTkHwm64A93wNvVmWLQ5U= X-Received: by 2002:adf:f70a:: with SMTP id r10mr32478710wrp.96.1556727375625; Wed, 01 May 2019 09:16:15 -0700 (PDT) MIME-Version: 1.0 References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> In-Reply-To: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> From: Chris Date: Wed, 1 May 2019 16:16:04 +0000 Message-ID: Subject: Re: ZFS... To: Michelle Sullivan Cc: freebsd-stable Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4548883E95 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=TMHNz9J4; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of chrcoluk@gmail.com designates 2a00:1450:4864:20::431 as permitted sender) smtp.mailfrom=chrcoluk@gmail.com X-Spamd-Result: default: False [-6.30 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.97)[-0.970,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[1.3.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.77)[ip: (-9.31), ipnet: 2a00:1450::/32(-2.25), asn: 15169(-2.24), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 16:16:19 -0000 Your story is so unusual I am wondering if its not fiction, I mean all sorts of power cuts where it just so happens the UPS fails every time, then you decide to ship a server halfway round the world, and on top of that you get a way above average rate of hard drive failures. But aside from all this you managed to recover multiple times. ZFS is never claimed to be a get out of jail free card, but it did survive in your case multiple times, I suggest tho if you value redundancy, do not use RAIDZ but use Mirror instead. I dont know why people keep persisting with raid 5/6 now days with drives as large as they are. I have used ZFS since the days of FreeBSD 8.x and its resilience compared to the likes of ext is astounding and especially compared to UFS. Before marking it down think how would UFS or ext have managed the scenarios you presented in your blog. Also think about where you hosting your data with all your power failures and the UPS equipment you utilise as well. On Mon, 29 Apr 2019 at 16:26, Michelle Sullivan wrote: > > I know I'm not going to be popular for this, but I'll just drop it here > anyhow. > > http://www.michellesullivan.org/blog/1726 > > Perhaps one should reconsider either: > > 1. Looking at tools that may be able to recover corrupt ZFS metadata, or > 2. Defaulting to non ZFS filesystems on install. > > -- > Michelle Sullivan > http://www.mhix.org/ > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed May 1 17:39:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4C17159C731 for ; Wed, 1 May 2019 17:39:44 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B745486943 for ; Wed, 1 May 2019 17:39:42 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-ed1-x542.google.com with SMTP id w37so15509586edw.4 for ; Wed, 01 May 2019 10:39:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=bT+2JE66EV5C+360NnFnIEPIRC8yh648yuvET2KW1ls=; b=1yJwBFgbiJIovR0UiWsNS0t1jPGy5udPzQ4zk2Se/vgq+jwLkTjbpgVwvUo0IqS/iA tHs3/1izMxRu8w3iQWA2McvgekKbbvF2cwFLzrPKqeZmLWL/htmQCoL+RIrcFbj5eENO /aRz/UwnApPAdCiJwv/TUU6bIen+toNdsCORaQIiCebnz59yjbDBODZILpCazLymbON3 epXVBj4ebNq8NRE5X5Zfvybr8A17jcB7Sd95OeF5gG6bbwilQTP1xd905+KDbxR0FAZp CreOLlvYaYKFUsr0I6BM9d89YIycrDfy+4HXgYGlOV5aq54iTPuQ8IQbp233bk1ijeiC YGGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=bT+2JE66EV5C+360NnFnIEPIRC8yh648yuvET2KW1ls=; b=ZYu2AzxYDq8PdT/AXswvcaLin8HSTV2ML1/txP8nruiPh7K+EtFWVaPL3UHw0S57dK I2/zDZiVJ3+v/Ow1YpoTR6E1tyuYR1KXR6W7YQqqkgNSnBgLgqQXD5LE2J25DdKFVjTC jGpRSzaK8ayG7keSEWW7fhu93o9uRTQzKF/x8q6KdV2zlSL8PKQaK4m7SxYISVOlaBJL esdHi7xc3R4Gr6hzvX48wbmQbs7lwhfVNgo1xL9t6eZcGNK2N45sDeB+RHMECcR+JNIM +bglsyjbKFlfsDAMw/VrpYMU3TwOmh0MewQ4axznPnRewt+F8vygzcpqkfTk8vpT8Uue a2IA== X-Gm-Message-State: APjAAAU3UGthhZu6ByxW6fCss1XYvsrD0KHMMy4GzbukIj/YpGd0LOFn 2qMfFq2fcwDveSsrZpDK5IcL+ZCU8fQ= X-Google-Smtp-Source: APXvYqwOjb4eJeu6JFd6CVfAxn8MTRePs747cPWuKrvUcwaRF4PN2xo+q+Qs4r6MfyiUEL+auB5ZDg== X-Received: by 2002:a17:906:2447:: with SMTP id a7mr37261429ejb.235.1556732380611; Wed, 01 May 2019 10:39:40 -0700 (PDT) Received: from [10.44.128.75] ([161.12.40.153]) by smtp.gmail.com with ESMTPSA id l3sm11110382edl.63.2019.05.01.10.39.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 May 2019 10:39:39 -0700 (PDT) Subject: Re: ZFS... To: Michelle Sullivan , Paul Mather Cc: freebsd-stable References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> <7DBA7907-BE8F-4944-9A71-86E5AC1B85CA@gromit.dlib.vt.edu> <5c458075-351f-6eb6-44aa-1bd268398343@sorbs.net> From: Steven Hartland Message-ID: <47137ea9-1ab2-1271-c15f-c0c05a17b92f@multiplay.co.uk> Date: Wed, 1 May 2019 18:39:39 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <5c458075-351f-6eb6-44aa-1bd268398343@sorbs.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: B745486943 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=multiplay-co-uk.20150623.gappssmtp.com header.s=20150623 header.b=1yJwBFgb; spf=pass (mx1.freebsd.org: domain of killing@multiplay.co.uk designates 2a00:1450:4864:20::542 as permitted sender) smtp.mailfrom=killing@multiplay.co.uk X-Spamd-Result: default: False [-3.87 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[multiplay-co-uk.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[multiplay.co.uk]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[multiplay-co-uk.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[ASPMX.L.GOOGLE.COM,ALT2.ASPMX.L.GOOGLE.COM,ALT1.ASPMX.L.GOOGLE.COM,ASPMX3.GOOGLEMAIL.COM,ASPMX2.GOOGLEMAIL.COM]; RCVD_IN_DNSWL_NONE(0.00)[2.4.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.97)[ip: (-0.28), ipnet: 2a00:1450::/32(-2.25), asn: 15169(-2.24), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.84)[-0.842,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 17:39:45 -0000 On 01/05/2019 15:53, Michelle Sullivan wrote: > Paul Mather wrote: >> On Apr 30, 2019, at 11:17 PM, Michelle Sullivan >> wrote: >> >>> Been there done that though with ext2 rather than UFS..  still got >>> all my data back... even though it was a nightmare.. >> >> >> Is that an implication that had all your data been on UFS (or ext2:) >> this time around you would have got it all back?  (I've got that >> impression through this thread from things you've written.) That sort >> of makes it sound like UFS is bulletproof to me. > > Its definitely not (and far from it) bullet proof - however when the > data on disk is not corrupt I have managed to recover it - even if it > has been a nightmare - no structure - all files in lost+found etc... > or even resorting to r-studio in the even of lost raid information etc.. Yes but you seem to have done this with ZFS too, just not in this particularly bad case. If you imagine that the in memory update for the metadata was corrupted and then written out to disk, which is what you seem to have experienced with your ZFS pool, then you'd be in much the same position. > > This case - from what my limited knowledge has managed to fathom is a > spacemap has become corrupt due to partial write during the hard power > failure. This was the second hard outage during the resilver process > following a drive platter failure (on a ZRAID2 - so single platter > failure should be completely recoverable all cases - except hba > failure or other corruption which does not appear to be the case).. > the spacemap fails checksum (no surprises there being that it was part > written) however it cannot be repaired (for what ever reason))... how > I get that this is an interesting case... one cannot just assume > anything about the corrupt spacemap... it could be complete and just > the checksum is wrong, it could be completely corrupt and ignorable.. > but what I understand of ZFS (and please watchers chime in if I'm > wrong) the spacemap is just the freespace map.. if corrupt or missing > one cannot just 'fix it' because there is a very good chance that the > fix would corrupt something that is actually allocated and therefore > the best solution would be (to "fix it") would be consider it 100% > full and therefore 'dead space' .. but zfs doesn't do that - probably > a good thing - the result being that a drive that is supposed to be > good (and zdb reports some +36m objects there) becomes completely > unreadable ...  my thought (desire/want) on a 'walk' tool would be a > last resort tool that could walk the datasets and send them elsewhere > (like zfs send) so that I could create a new pool elsewhere and send > the data it knows about to another pool and then blow away the > original - if there are corruptions or data missing, thats my problem > it's a last resort.. but in the case the critical structures become > corrupt it means a local recovery option is enabled.. it means that if > the data is all there and the corruption is just a spacemap one can > transfer the entire drive/data to a new pool whilst the original host > is rebuilt... this would *significantly* help most people with large > pools that have to blow them away and re-create the pools because of > errors/corruptions etc... and with the addition of 'rsync' (the > checksumming of files) it would be trivial to just 'fix' the data > corrupted or missing from a mirror host rather than transferring the > entire pool from (possibly) offsite.... From what I've read that's not a partial write issue, as in that case the pool would have just rolled back. It sounds more like the write was successful but the data in that write was trashed due to your power incident and that was replicated across ALL drives. To be clear this may or may not be what your seeing as you don't see to have covered any of the details of the issues your seeing and what in detail steps you have tried to recover with? I'm not saying this is the case but all may not be lost depending on the exact nature of the corruption. For more information on space maps see: https://www.delphix.com/blog/delphix-engineering/openzfs-code-walk-metaslabs-and-space-maps https://sdimitro.github.io/post/zfs-lsm-flushing/ A similar behavior resulted in being a bug: https://www.reddit.com/r/zfs/comments/97czae/zfs_zdb_space_map_errors_on_unmountable_zpool/     Regards     Steve From owner-freebsd-stable@freebsd.org Wed May 1 23:21:50 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D3A5158084B for ; Wed, 1 May 2019 23:21:50 +0000 (UTC) (envelope-from ggm@algebras.org) Received: from mail-it1-x143.google.com (mail-it1-x143.google.com [IPv6:2607:f8b0:4864:20::143]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BCC8D6AF6D for ; Wed, 1 May 2019 23:21:46 +0000 (UTC) (envelope-from ggm@algebras.org) Received: by mail-it1-x143.google.com with SMTP id x132so281407itf.2 for ; Wed, 01 May 2019 16:21:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=algebras-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=5WmwH5NkVUI2HIdbMNumu7MIyCRRYcmkr6a8TGMsRjs=; b=C4iGPZnrJqpvqi5rUDPOGh9fvmiuugSLG+2XOkDeO70DbvEwSI85UbStp6SOmvJr2g t/cjrwUVjMVLJeeZ3eeXev0h/4IBTGKaZgpSk8Fq273j+CYS+R/ksAVfJQeFvhoXlv4x bL0/bgkeCEooiVd2DVB3ecK+5Iij4r/45xrnhd0UlKQpMmRzWUKnCBRj7nV5DNrKQujA 5M3IYgQrifNgbs/eHCFwIIMbKie1SFfscrFShVK7CYWfgpjfqtOGZCWeP/aPN3NZD87A 65JN5E1WaI4KcrDXx8CtbL4uVgPtAexxetSq+0VqxpA9uXfErjAuAAQ5Qlh101nEEj0M ZV9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=5WmwH5NkVUI2HIdbMNumu7MIyCRRYcmkr6a8TGMsRjs=; b=Z9hvp3H6FJAWuTNE7r5LL0JGyY7NzSymvkPsG4HpAeodl5oRfS14qV80cwWs6TTCbS Qna9TZCAQnTGcgXZns8uFcI8PRe7savmnC3pDH3WYHOqqSPHT21d6/qTUguE+SnCuYXD AdTlFQ3S21o7Aw3utb6MdS4ESE4xWMz0EZl1AtLvxkVVhlwgWEBYfoI/TLAObVU5eMQn +eT27rQ2edfZbNvaTibjHWkuHuGF9X1jswQJIdshJ+gW3g3CNLY+wOkK39f1FGGcmUI7 4ZExbGdMoC+EdG0+RbBFYlgDFfx7t9HATKzn0kZRk1gEjMVdQQi9iZmwm2K2IMJRbxC/ EMQA== X-Gm-Message-State: APjAAAWE+sXlT0odx5a2Oy5lCPyRg6lmSdZum99cp/pLoRBhAooXBqh3 Qjc2jkxPhvOdgUBrWH7dA0NAiK2gYTs8c2iWzQzLrCufbtw= X-Google-Smtp-Source: APXvYqwiBOkB8a9fekOEhV9VM6R4WRvjf65VvbBEdpZc6dh33OISvmJanCg5eCDzD1hP6K2+8xbvaSrKGdeHWNPqjbs= X-Received: by 2002:a02:c619:: with SMTP id i25mr245071jan.141.1556752905936; Wed, 01 May 2019 16:21:45 -0700 (PDT) MIME-Version: 1.0 References: <201905010356.x413uUnV068470@slippy.cwsent.com> <56b15cc3-4605-f693-4ad6-188ea865820d@quip.cz> In-Reply-To: <56b15cc3-4605-f693-4ad6-188ea865820d@quip.cz> From: George Michaelson Date: Thu, 2 May 2019 09:21:34 +1000 Message-ID: Subject: Re: CFT: FreeBSD Package Base To: FreeBSD Stable Cc: freebsd-ports@freebsd.org, freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: BCC8D6AF6D X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=algebras-org.20150623.gappssmtp.com header.s=20150623 header.b=C4iGPZnr; spf=pass (mx1.freebsd.org: domain of ggm@algebras.org designates 2607:f8b0:4864:20::143 as permitted sender) smtp.mailfrom=ggm@algebras.org X-Spamd-Result: default: False [-4.23 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[algebras-org.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[algebras.org]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[algebras-org.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[ASPMX.L.GOOGLE.COM,ALT2.ASPMX.L.GOOGLE.COM,ASPMX4.GOOGLEMAIL.COM,ALT1.ASPMX.L.GOOGLE.COM,ASPMX2.GOOGLEMAIL.COM,ASPMX3.GOOGLEMAIL.COM,ASPMX5.GOOGLEMAIL.COM]; RCVD_IN_DNSWL_NONE(0.00)[3.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.96)[-0.959,0]; IP_SCORE(-0.77)[ip: (1.67), ipnet: 2607:f8b0::/32(-3.19), asn: 15169(-2.25), country: US(-0.06)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 23:21:50 -0000 The current state of this looks logical: its the bare minimum to represent the .tgz which we recognized from install menu choices, which minimises any real SAT solver dependency load, because these are not cross-dependant in any strong sense. The goal surely would be to prove this works and test. Then, we can move to the state I think I want, and I presume by extension everyone else wants: Sets which are small enough they expose risk of cross dependency, but once a SAT solver reconciles them, you have ONLY the minimum needed to do your job. So a "tiny" FreeBSD could be back under 4MB (hypothetically) because it was /rescue and the kernel, and if you needed /usr/share you loaded a pkg. -G On Wed, May 1, 2019 at 7:33 PM Miroslav Lachman <000.fbsd@quip.cz> wrote: > > Cy Schubert wrote on 2019/05/01 05:56: > > In message <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz>, Miroslav > > Lachman wri > > tes: > >> David Chisnall wrote on 2019/04/30 10:22: > >>> On 29/04/2019 21:12, Joe Maloney wrote: > >>>> With CFT version you chose to build, and package individual componen= ts > >>>> such as sendmail with a port option.=C3=82 That does entirely solve= the > >>>> problem of being able to reinstall sendmail after the fact without a > >>>> rebuild of the userland (base) port but perhaps base flavors could > >>>> solve that problem assuming flavors could extend beyond python. > >>> > >>> This sounds very much like local optimisation. It's now easy to creat= e a > >>> custom base image.=C3=82 Great.=C3=82 But how do I express dependen= cies in ports > >>> on a specific base configuration? This is easy if I depend on a speci= fic > >>> base package, but how does this work in your model?=C3=82 For exampl= e, if I > >>> have a package that depends on a library that is an optional part of = the > >>> base system, how do I express that pkg needs to either refuse to inst= all > >>> it, or install a userland pkg that includes that library in place of = my > >>> existing version as part of the install process? > >>> > >>> More importantly for the container use case, if I want to take a > >>> completely empty jail and do pkg ins nginx (for example), what does t= he > >>> maintainer of the nginx port need to do to express the minimum set of > >>> the base system that needs to be installed to allow nginx to work? > >>> > >>> One of the goals for the pkg base concept was to allow this kind of u= se > >>> case, easily creating a minimal environment required to run a single > >>> service. With a monolithic base package set, you're going to need som= e > >>> mechanism other than packages to express the specific base subset > >>> package that you need and I think that you need to justify why this > >>> mechanism is better than using small individual packages. > >> > >> Will it not be maintainer's nightmare to take care of all the > >> dependencies on the base packages for each port we have in the ports t= ree? > > > > No more than it is today. Remember, people have been doing this sort of > > thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and > > IBM can do it, I'm sure we can too. The dependency lists will be > > longer. We may require dependency lists that allow the choice of one of > > many prereqs or coreqs. > > They are experts and they are paid for their work. I am not. I am > maintaining a few packages and the reality is I don't know what they > need in base. Till these days I don't care about this kind of > dependency. I am not system developer or programmer and I think there > are more than just me who see this as a kind of problem. > So in this case, pkg base gives me nothing but more work on those package= s. > > Miroslav Lachman > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed May 1 23:27:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 255341580AD7 for ; Wed, 1 May 2019 23:27:53 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id D023B6B31A for ; Wed, 1 May 2019 23:27:47 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQU00MUQN5AZ500@hades.sorbs.net> for freebsd-stable@freebsd.org; Wed, 01 May 2019 16:41:37 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Thu, 02 May 2019 09:27:36 +1000 Cc: freebsd-stable Message-id: <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> To: Chris X-Rspamd-Queue-Id: D023B6B31A X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.09 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.983,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[sorbs.net]; NEURAL_HAM_SHORT(-0.83)[-0.825,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[battlestar.sorbs.net,anaconda.sorbs.net,ninja.sorbs.net,catapilla.sorbs.net,scorpion.sorbs.net,desperado.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.52)[ip: (-1.35), ipnet: 72.12.192.0/19(-0.68), asn: 11114(-0.53), country: US(-0.06)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 23:27:53 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 02 May 2019, at 02:16, Chris wrote: >=20 > Your story is so unusual I am wondering if its not fiction, I mean all > sorts of power cuts where it just so happens the UPS fails every time, The only =E2=80=9Cfiction=E2=80=9D is the date.. was the 10th not the 19th M= arch... https://www.southcoastregister.com.au/story/5945663/homes-left-witho= ut-power-after-electrical-pole-destroyed-in-sanctuary-point-accident/ UPSes do glitch out sometimes, but rarely.. they do have problems when 11kv= comes down a 240v line though... > then you decide to ship a server halfway round the world, and on top > of that you get a way above average rate of hard drive failures. But > aside from all this you managed to recover multiple times. >=20 Incorrect.. I shipped the server around the world 18 months ago.. (Oct 2017= ).. before that the power problems... well Malta is legendary for them... = =20 This was the last to hit me before I moved back home to Australia after bein= g there for 8 years... =20 https://www.timesofmalta.com/articles/view/20170826/local/hundreds-of-househ= olds-without-power-after-fire-in-distribution-centre.656490 > ZFS is never claimed to be a get out of jail free card, but it did > survive in your case multiple times, I suggest tho if you value > redundancy, do not use RAIDZ but use Mirror instead. I dont know why > people keep persisting with raid 5/6 now days with drives as large as > they are. Could that be because at the time of building it, the largest drives were 4T= ... had the 6T drives been available to me I would have mirrored 6T drives i= nstead of zRAID2 ing 16x3T drives. >=20 > I have used ZFS since the days of FreeBSD 8.x and its resilience > compared to the likes of ext is astounding and especially compared to > UFS. I=E2=80=99m not disputing its resilience to errors in the file data, it is r= ather good, but when it comes to the metadata that=E2=80=99s when I have alw= ays had problems.. it=E2=80=99s ok until it isn=E2=80=99t, then it=E2=80=99s= lucky if you can get it back... and until now I=E2=80=99ve had that luck. >=20 > Before marking it down think how would UFS or ext have managed the > scenarios you presented in your blog. Well I have 2 servers with zfs, the rest are UFS or HPFS... the only other I= ssue I had was a (mirrored drive) with HPFS... it got corrupted where it (t= he FSCK like tools) couldn=E2=80=99t fix it... but the drive was still acce= ssible and the backups were on the zfs drives... (timemachines in zvols over= iscsi)... I didn=E2=80=99t need to go to timemachine (though I did check th= e data for consistency after =E2=80=9Crestore=E2=80=9D) .. I got new drives,= replaced then, mirrored them, then copied over everything except the OS by m= ounting one of the drives in an external caddy... solved the underlying =E2= =80=9Cunfixable=E2=80=9D error in the HPFS structures... >=20 > Also think about where you hosting your data with all your power > failures and the UPS equipment you utilise as well. Well I have insurance quotes for new UPSes that I=E2=80=99m waiting on repla= cement so that=E2=80=99s sorta mute... I could post the images of them here= if you like or don=E2=80=99t believe me? Seriously it is unusual I get it, but all my ZFS problems have been due to f= ailures whilst resilvering ... it=E2=80=99s always (over 8 years of running t= hese servers) resilvering tHat does it... it=E2=80=99ll be happily progress= ing and another drive fails, power goes out, kernel panic, etc... then the p= roblems start, and if it does it twice you better start praying. This is my= experience. Michelle >=20 >> On Mon, 29 Apr 2019 at 16:26, Michelle Sullivan wrot= e: >>=20 >> I know I'm not going to be popular for this, but I'll just drop it here >> anyhow. >>=20 >> http://www.michellesullivan.org/blog/1726 >>=20 >> Perhaps one should reconsider either: >>=20 >> 1. Looking at tools that may be able to recover corrupt ZFS metadata, or >> 2. Defaulting to non ZFS filesystems on install. >>=20 >> -- >> Michelle Sullivan >> http://www.mhix.org/ >>=20 >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"= > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed May 1 23:46:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7E16158139F for ; Wed, 1 May 2019 23:46:27 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 3A2026BD54 for ; Wed, 1 May 2019 23:46:27 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQU00MV5O0JZ500@hades.sorbs.net> for freebsd-stable@freebsd.org; Wed, 01 May 2019 17:00:23 -0700 (PDT) Sun-Java-System-SMTP-Warning: Lines longer than SMTP allows found and truncated. Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <47137ea9-1ab2-1271-c15f-c0c05a17b92f@multiplay.co.uk> Date: Thu, 02 May 2019 09:46:22 +1000 Cc: Paul Mather , freebsd-stable Message-id: <289FE04E-1692-4763-96B3-91E8C1BBBBD6@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> <7DBA7907-BE8F-4944-9A71-86E5AC1B85CA@gromit.dlib.vt.edu> <5c458075-351f-6eb6-44aa-1bd268398343@sorbs.net> To: Steven Hartland X-Rspamd-Queue-Id: 3A2026BD54 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.14 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; NEURAL_HAM_SHORT(-0.88)[-0.883,0]; SUBJ_ALL_CAPS(0.45)[6]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.985,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; IP_SCORE(-0.51)[ip: (-1.31), ipnet: 72.12.192.0/19(-0.66), asn: 11114(-0.51), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 23:46:28 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 02 May 2019, at 03:39, Steven Hartland wrote:= >=20 >=20 >=20 >> On 01/05/2019 15:53, Michelle Sullivan wrote: >> Paul Mather wrote: >>>> On Apr 30, 2019, at 11:17 PM, Michelle Sullivan wr= ote: >>>>=20 >>>> Been there done that though with ext2 rather than UFS.. still got all m= y data back... even though it was a nightmare.. >>>=20 >>>=20 >>> Is that an implication that had all your data been on UFS (or ext2:) thi= s time around you would have got it all back? (I've got that impression thr= ough this thread from things you've written.) That sort of makes it sound li= ke UFS is bulletproof to me. >>=20 >> Its definitely not (and far from it) bullet proof - however when the data= on disk is not corrupt I have managed to recover it - even if it has been a= nightmare - no structure - all files in lost+found etc... or even resorting= to r-studio in the even of lost raid information etc.. > Yes but you seem to have done this with ZFS too, just not in this particul= arly bad case. >=20 There is no r-studio for zfs or I would have turned to it as soon as this is= sue hit. > If you imagine that the in memory update for the metadata was corrupted an= d then written out to disk, which is what you seem to have experienced with y= our ZFS pool, then you'd be in much the same position. >>=20 >> This case - from what my limited knowledge has managed to fathom is a spa= cemap has become corrupt due to partial write during the hard power failure.= This was the second hard outage during the resilver process following a dri= ve platter failure (on a ZRAID2 - so single platter failure should be comple= tely recoverable all cases - except hba failure or other corruption which do= es not appear to be the case).. the spacemap fails checksum (no surprises th= ere being that it was part written) however it cannot be repaired (for what e= ver reason))... how I get that this is an interesting case... one cannot jus= t assume anything about the corrupt spacemap... it could be complete and jus= t the checksum is wrong, it could be completely corrupt and ignorable.. but w= hat I understand of ZFS (and please watchers chime in if I'm wrong) the spac= emap is just the freespace map.. if corrupt or missing one cannot just 'fix i= t' because there is a very good chance that the fix would corrupt something t= hat is actually allocated and therefore the best solution would be (to "fix i= t") would be consider it 100% full and therefore 'dead space' .. but zfs doe= sn't do that - probably a good thing - the result being that a drive that is= supposed to be good (and zdb reports some +36m objects there) becomes compl= etely unreadable ... my thought (desire/want) on a 'walk' tool would be a l= ast resort tool that could walk the datasets and send them elsewhere (like z= fs send) so that I could create a new pool elsewhere and send the data it kn= ows about to another pool and then blow away the original - if there are cor= ruptions or data missing, thats my problem it's a last resort.. but in the c= ase the critical structures become corrupt it means a local recovery option i= s enabled.. it means that if the data is all there and the corruption is jus= t a spacemap one can transfer the entire drive/data to a new pool whilst the= original host is rebuilt... this would *significantly* help most people wit= h large pools that have to blow them away and re-create the pools because of= errors/corruptions etc... and with the addition of 'rsync' (the checksummin= g of files) it would be trivial to just 'fix' the data corrupted or missing f= rom a mirror host rather than transferring the entire pool from (possibly) o= ffsite.... >=20 > =46rom what I've read that's not a partial write issue, as in that case th= e pool would have just rolled back. It sounds more like the write was succes= sful but the data in that write was trashed due to your power incident and t= hat was replicated across ALL drives. >=20 I think this might be where the problem started.. it was already rolling bac= k from the first power issue (it did exactly what was expected and programme= d, it rolled back 5 seconds.. which as no-one had write access to it from th= e start of the resilver I really didn=E2=80=99t care as the only changes wer= e the resilver itself.). Now you assertion/musing maybe correct... all driv= es got trashed data.. I think not but unless we get into it and examine it I= think we won=E2=80=99t know. What I do know is in the second round -FfX wo= uldn=E2=80=99t work, I used zdb to locate a =E2=80=9CLOADED=E2=80=9D MOS and= used -t to import.. the txg number was 7 or 8 from current so just ou= tside of the -X limit (going off memory here, so could have been more, but I= remember it was just past the switch limit.) > To be clear this may or may not be what your seeing as you don't see to ha= ve covered any of the details of the issues your seeing and what in detail s= teps you have tried to recover with? There have been many steps over the last month.. and some of which I may hav= e made it from very difficult to recover to non recoverable now... though th= e only writes is what the kernel does as have not got it (the dataset) mount= ed at any time, even though it has imported. >=20 > I'm not saying this is the case but all may not be lost depending on the e= xact nature of the corruption. >=20 > For more information on space maps see: > https://www.delphix.com/blog/delphix-engineering/openzfs-code-walk-metasla= bs-and-space-maps This is something I read a month ago, along with multiple other articles on t= he same blog, including https://www.delphix.com/blog/openzfs-pool-import-rec= overy Which I might add got me from non importable to importable but not mountable= . I have *not* attempted to bypass the checksum line for spacemap load to date= as I see that as a possible way to make the problem worse. > https://sdimitro.github.io/post/zfs-lsm-flushing/ Not read this. >=20 > A similar behavior resulted in being a bug: > https://www.reddit.com/r/zfs/comments/97czae/zfs_zdb_space_map_errors_on_u= nmountable_zpool/ >=20 Or this.. will go there following =E2=80=9Cpressing send=E2=80=9D.. :) > Regards > Steve > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed May 1 23:52:25 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48CAB1581740 for ; Wed, 1 May 2019 23:52:25 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 892116C152 for ; Wed, 1 May 2019 23:52:24 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQU00MVAOAIZ500@hades.sorbs.net> for freebsd-stable@freebsd.org; Wed, 01 May 2019 17:06:20 -0700 (PDT) Sun-Java-System-SMTP-Warning: Lines longer than SMTP allows found and truncated. Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <289FE04E-1692-4763-96B3-91E8C1BBBBD6@sorbs.net> Date: Thu, 02 May 2019 09:52:20 +1000 Cc: freebsd-stable Content-transfer-encoding: quoted-printable Message-id: References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <34539589-162B-4891-A68F-88F879B59650@sorbs.net> <576857a5-a5ab-eeb8-2391-992159d9c4f2@denninger.net> <7DBA7907-BE8F-4944-9A71-86E5AC1B85CA@gromit.dlib.vt.edu> <5c458075-351f-6eb6-44aa-1bd268398343@sorbs.net> To: Steven Hartland X-Rspamd-Queue-Id: 892116C152 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.17 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.993,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.49)[ip: (-1.27), ipnet: 72.12.192.0/19(-0.64), asn: 11114(-0.50), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.92)[-0.923,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 May 2019 23:52:25 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 02 May 2019, at 09:46, Michelle Sullivan wrote: >=20 > What I do know is in the second round -FfX wouldn=E2=80=99t work, *after the second round From owner-freebsd-stable@freebsd.org Thu May 2 10:53:57 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB9D215906E9; Thu, 2 May 2019 10:53:56 +0000 (UTC) (envelope-from root@sfc.wide.ad.jp) Received: from mail1.sfc.wide.ad.jp (mail1.sfc.wide.ad.jp [203.178.142.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 82D5B882A6; Thu, 2 May 2019 10:53:55 +0000 (UTC) (envelope-from root@sfc.wide.ad.jp) Received: by mail1.sfc.wide.ad.jp (Postfix, from userid 0) id B7B971BCCF; Thu, 2 May 2019 19:40:08 +0900 (JST) X-Original-To: micchie@sfc.wide.ad.jp Received: from ironport1.sfc.wide.ad.jp (ironport1.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:150]) by mail1.sfc.wide.ad.jp (Postfix) with ESMTPS id 7161212691 for ; Wed, 1 May 2019 13:00:36 +0900 (JST) Received-SPF: None (ironport1.sfc.wide.ad.jp: no sender authenticity information available from domain of owner-freebsd-current@freebsd.org) identity=pra; client-ip=8.8.178.116; receiver=ironport1.sfc.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="owner-freebsd-current@freebsd.org"; x-conformance=sidf_compatible Received-SPF: Pass (ironport1.sfc.wide.ad.jp: domain of owner-freebsd-current@freebsd.org designates 8.8.178.116 as permitted sender) identity=mailfrom; client-ip=8.8.178.116; receiver=ironport1.sfc.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="owner-freebsd-current@freebsd.org"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:8.8.178.116 ip4:96.47.72.81 ip6:2001:1900:2254:206a::19:2 ip6:2610:1c1:1:606c::19:2 ~all" Received-SPF: None (ironport1.sfc.wide.ad.jp: no sender authenticity information available from domain of postmaster@mx2.freebsd.org) identity=helo; client-ip=8.8.178.116; receiver=ironport1.sfc.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="postmaster@mx2.freebsd.org"; x-conformance=sidf_compatible IronPort-SDR: +p/mLucn7TetiYTI/wCXSH4l5Y82ShzsEQYCS8CXqmPuihuS1zePafWHrYl5Ch2XhCWivRy9WW K0XWcqc7q31+2PhRFrjJjwaf683iA2We4= IronPort-PHdr: =?us-ascii?q?9a23=3ADlDfjxG8QZTbUO/Q4kPZhJ1GYnF86YWxBRYc79?= =?us-ascii?q?8ds5kLTJ76p8q5bnLW6fgltlLVR4KTs6sC17OP9f6+EjNQqb+681k6OKRWUB?= =?us-ascii?q?EEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i764jEdAA?= =?us-ascii?q?jwOhRoLerpBIHSk9631+ev8JHPfglEnjWwba5vIBmsrAjct8YajIlhJ60s1h?= =?us-ascii?q?bHv3xEdvhMy2h1P1yThRH85smx/J5n7Stdvu8q+tBDX6vnYak2VKRUAzs6PW?= =?us-ascii?q?874s3rrgTDQhCU5nQASGUWkwFHDBbD4RrnQ5r+qCr6tu562CmHIc37SK0/VD?= =?us-ascii?q?q+46t3ThLjlTwKPCAl/m7JlsNwjbpboBO/qBx5347Ue5yeOP5ncq/AYd8WWW?= =?us-ascii?q?9NU8BWWCJZBYOybIUBBPcFMepFs4Xxol0Dpga8CwaxHuPi0jhGhmX206w1ze?= =?us-ascii?q?ovDAPJ0wMuEdIWsnnartr4OLocXOyp0KbGwivMb+hN1Dfh8YjEbgwtrOuQUb?= =?us-ascii?q?5td8fa1EkhFxnCjlWVsYHoITOU3fkQvWeH7+pgSPmghHA9qwFtvzOiwdkhi4?= =?us-ascii?q?nTiowI1l/E9Th5z50rKt2iVE52esKrEIBMuCGaKot5Wd0tQ3t0uCog1rIGvp?= =?us-ascii?q?u7cTEMxZ86yRDfbPmHfJKJ4hLlTOuRLy14i2x7d72knBa97FKsyurmVsm71l?= =?us-ascii?q?tBsylLksHUu30OzRDe6dKLRuZ/80qiwzqDyh3f5+BcLU02iabXMZ8sz7wqmp?= =?us-ascii?q?YOr0jOHjX6lFj0gaKQa04q4PKn6/79bbXjvpKcN5F7igX5Mqk2n8y+Dvo0Ph?= =?us-ascii?q?IAX2iB5+uzyqbs/VbkT7lQif02iKbZvIjbJcQduKG5HxdY34U55xu8FTuqzd?= =?us-ascii?q?YVkHodIF5YZR6KjpDlO1TUL/D5Cfe/jU6skDBux/3eOb3gDJPNIWbZkLv4c7?= =?us-ascii?q?Z98VBTxxQowtBE+p1YE60OIO/pWk/rqtPYCAM5Phe6w+b9FtpyyJgRWXmSDa?= =?us-ascii?q?ODN6PSq1CI6vo1I+aQfI8VpCr9K/896v7rjH82hUMSfamo3ZQJc324BfpmLF?= =?us-ascii?q?mDbXvig9cODHkFshAkQ+P3klGCUyZTND6PWbkh7GQ7FJ6+FtWEAYSsm6Cami?= =?us-ascii?q?mhE5BcfWwADUqDVnLhdoGBUvFLbzqOI8hniXsKT7WsQZQtkBqzrhX3zaY0E+?= =?us-ascii?q?zP5ydNsJvi0MRytePJmBw47jAxBNmHyWyLUyRuk3oVSiQq9K5lrEF30VvF1r?= =?us-ascii?q?J31OdFH95e7O8cTwEhKJTHxPZ7Ad2hZgWUW9aTT02iS8vuJTg6Ttc4xZdaOx?= =?us-ascii?q?4lXduolRzrxjerG/keherYPpEs9rPg2C3aNo5GwHbb3qQ7kxFyadoJDmamm6?= =?us-ascii?q?Ry7RT7DJXCk1iTmry2M68b2Xie2n2EyD+lrQljWw5tW6jfQThLTVqQlt/5+0?= =?us-ascii?q?nPVaOGAq8sMhZLxNSebKBNb4u63h19WP7/NYGGMCqKkGCqCEPNl+vWNtjDZn?= =?us-ascii?q?kd0SPBCUMNjwEU+zO8OBMjAju6+jKEXj10ElT3ZFnwqK9lpnj9UU44wkeDYw?= =?us-ascii?q?tjz+n94QYb0NqbTf5bxbcYoGE5sTwhG0m70siTBcCNqgB6cOBXfMkn7VFb/W?= =?us-ascii?q?zDugF3JZDmKLpt1RYFawoih0r1zF1sD5lY180jrXck1g13fKWH1l5bMj2C0J?= =?us-ascii?q?36IbGSIXTo5h2pd4bYwFzT2s2avKAV56dwsE3t6SeuEEdq6HB7y59V3n+bs4?= =?us-ascii?q?3NFxYXWIntX1wf8ggg4b/AbyQ3/I+S0mdjaPPmiDLJ1tM3Cedg8S6OJIsEYo?= =?us-ascii?q?WDEgK6U8wAAM62L/Y2wh6yYBlCI+Fe+eg9M4Wkb6nO3qnjJ+tmkD+86AYPqI?= =?us-ascii?q?lgzkKB8TZ9Qe/Uzt4Exf+fxA6OSzb7ih+orMn2nYlOYTxaEHC4zGDoA4tYZ6?= =?us-ascii?q?s6eohuay/mJNKxy887iYTgVnRD+BuoGkkc1cm1URSIYlH3xgEW0l4Y4DSmlS?= =?us-ascii?q?a+0z1ogmQxtKPMuU6Gi+/mdRcBJitKXDwyyweqeNfkyYlDAhvgdQUimRq76F?= =?us-ascii?q?yvxrJcorx4NXWKB1xGdG7pLm1vFKK88LiaMKstoNsltztaVOOkbBWUULn49l?= =?us-ascii?q?EdzCXuBS5Q3j02fSqm/57khQZ3jn61Imx+oXDCdYd33xiVt7m+DbZBmyELQi?= =?us-ascii?q?V1k2ydAUe6MsPs88+Zmp3YvqW1Sni7W5tPWSfxwI6Kryf94ndlS07a/bj7ip?= =?us-ascii?q?jsFg401jX+3t9hWHDTrRryVYLs0ryzLeNtekQxXg3G5sF3G59zns4LvL9MiS?= =?us-ascii?q?hIvpyO5jJHkH30N81cw76nKmEDTngUwtLYpgHpnk97fDqFwIfwV3PVxcUEBZ?= =?us-ascii?q?HyZXwQ1zh44dpHBa2P4PlPgDFvr1ykhQzLbPV3hTtbzuEhoHIXmOAGvgMxwz?= =?us-ascii?q?7VUvZCWxAHbWq1zkrA5svbzu0ffGu1dLmsyEdy1cusCr2PuEAUWXr0fIsjAT?= =?us-ascii?q?4l68x+NFzW13igoorgedTWcZcSrkjE1UeG1rcFbstoyKlv52IvI2/2sHw7xv?= =?us-ascii?q?RujAZp24m9ppTebX5j9+SmAxJVcDT5IcELqVSPxe5TmNib24e3E9BvADIOCd?= =?us-ascii?q?HvUvSlCnccr/nqPR2CVTYhsG2WH6b3ExWS5U19ozTICZ/hZBT1bDEJiM5vQh?= =?us-ascii?q?WQPhkVhx8SUS9/kIQ0GwqxyInqbV1j6zYNzlDipxZG0e4uMAPwGDS6xk/gen?= =?us-ascii?q?I/T56RKwBT5wdJ6hLOMMCQ2el0GjlR4pyrqAHeYnzefQlDCnsFH1CVH122dK?= =?us-ascii?q?f7/sHOqqLLY4j2Z+uLe7iFrvZSEuuF1Y76mJUz5C6Cb42OJiUwX7trnBMcDT?= =?us-ascii?q?YgQpqRmi1TGX1J0X6SKZHB4kfln08/5sGnrqaxAFq2t9LeU+MAa48yp0Ls2f?= =?us-ascii?q?mKZbzJ1XQjYTdAiMFWmyaQmuEUhQxL2X82KTL/Q+1R53SFWqvUnuUOVUwhZi?= =?us-ascii?q?h+NdVF4+cH5icWYpaJrNTz2/Y4j+QyDU9CT0C43NqnbooVIm2wclrLQkSTZv?= =?us-ascii?q?yAInXQzsf7bLnZK/UYhfhItxC2pTeQElPydjWFmT7zUhmzMOZKxCiFNR1asY?= =?us-ascii?q?u5f15jE2/mBN7hbxS6Npdwg1hUifUsgWjWMGcHLTVmW0Zd9Pue9y5ViO5zXW?= =?us-ascii?q?tb4TstLOWJnTqY8/iNKpsStqgOYGw8nOZb7XImjrpNuXECFLosx3uU94cw8B?= =?us-ascii?q?m8n+KCyyRqSk9PsDFNnIWRoRwkIqvSsIZBVXuC9R5L733DbnZC79ZjFNDrvL?= =?us-ascii?q?hdj9bVk6emYjBd9d/OuMUGCsXeM87CO2c7KxfvBBbaFgYIRyWnc2bFiAYO9Z?= =?us-ascii?q?Pavm3QtZU8ppX2zdAWTaRHUVUuCv4AIkF1RpoMOpp6VCgo17mBg4RbgBj25A?= =?us-ascii?q?mUT8JcsJfdU/uUCvi6MzeVg45PYB4QyK/5J4Aeb9Gp6wlZclB/2b/yNQ/IR9?= =?us-ascii?q?kU+X9vdAI5ql5Btn9kQT9rghO3WkaW+HYWUMWMsFszgwp6b/4q8W63sVktK1?= =?us-ascii?q?fAvyd2l1M+y4y83GKhNQXpJaL1Zrl4Tir5s09obcH+U1wzYxe/kEZ/OHHDXb?= =?us-ascii?q?0D17Y=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0EXAQDuGMlch3SyCAhmHAEBAQQBAQc?= =?us-ascii?q?EAQGBVAQBAQsBgT1QWxFVIAQzjQuKJoIOmkEaGAsKAoN4RoY1GwcBBDMGDgE?= =?us-ascii?q?DAQEBAwEBAgEBAQEBAhABAQEKCwkIKSMMgjoFAgMfBw5NawEBAQEBASMCDWQ?= =?us-ascii?q?BAgMBAQE9AQEECAIDGwsBAgMBAgYCAwcOCgkdCAMBCxgBCQsbBggHBAEEGAS?= =?us-ascii?q?DAQGCCgECAgqidYosgiCCeQEBBYI3gzyBPAkTgR8Bii2BHVWBAj+DJX4+gmE?= =?us-ascii?q?BBIE+AQEehWKLOoc8PZJwZQcCE4F4VYVAhC4GgxyEVRuVL5JRjjaBZSKBV00?= =?us-ascii?q?jUIFNgR8JggaJA4VfUhB4HI98SIF7AQE?= X-IPAS-Result: =?us-ascii?q?A0EXAQDuGMlch3SyCAhmHAEBAQQBAQcEAQGBVAQBAQsBg?= =?us-ascii?q?T1QWxFVIAQzjQuKJoIOmkEaGAsKAoN4RoY1GwcBBDMGDgEDAQEBAwEBAgEBA?= =?us-ascii?q?QEBAhABAQEKCwkIKSMMgjoFAgMfBw5NawEBAQEBASMCDWQBAgMBAQE9AQEEC?= =?us-ascii?q?AIDGwsBAgMBAgYCAwcOCgkdCAMBCxgBCQsbBggHBAEEGASDAQGCCgECAgqid?= =?us-ascii?q?YosgiCCeQEBBYI3gzyBPAkTgR8Bii2BHVWBAj+DJX4+gmEBBIE+AQEehWKLO?= =?us-ascii?q?oc8PZJwZQcCE4F4VYVAhC4GgxyEVRuVL5JRjjaBZSKBV00jUIFNgR8JggaJA?= =?us-ascii?q?4VfUhB4HI98SIF7AQE?= X-IronPort-AV: E=Sophos;i="5.60,416,1549897200"; d="scan'208";a="7837155" Received: from mx2.freebsd.org ([8.8.178.116]) by ironport1.sfc.wide.ad.jp with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 01 May 2019 13:00:28 +0900 Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx2.freebsd.org (Postfix) with ESMTPS id 568B46AA55; Wed, 1 May 2019 04:00:23 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 82E9F75012; Wed, 1 May 2019 04:00:20 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1556683222; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:list-id:list-unsubscribe: list-subscribe:list-post; bh=4oCUGjfmMIx05OZmT8a3oiE4MzgrgGEEn4iBxbd66JM=; b=SIY97bvibJXMyKMdD8rXoCcV7pZGD4xGlXar3ucpH+93Cut8DdJz/Yu7JhMDF4P0chmtGH 2RGeufy4KhppQw9JMzNAY8Jibyr1BXm95cTN2yIIkMPfG9Wh4KoE4X3aUxDUOLAh+2yeGY usRSQqQ6MAZpUxsvRSxf4AJTeCTttCWeuTFP9S5lb8z5R6+SzV69jwvFEW5y658F0+Ee1r vd1rXc1beU1QsW2iHCgRVUWo3m+TGR1om0kNYlvx+OMNk+iiXcXmjNWIJGJ2VmOxFIRzaj Qt031PAXXp6cMt1NXOH/DADg3Y7SnU2MYoi0QVtRsoOmC0qpYjsRJ2Fo9R11NA== Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04312158331C; Wed, 1 May 2019 03:56:46 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E73A974C1C; Wed, 1 May 2019 03:56:43 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id LgMEhBJ2TldkPLgMFhnFVa; Tue, 30 Apr 2019 21:56:36 -0600 X-Authority-Analysis: v=2.3 cv=Ko4zJleN c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=8nJEP1OIZ-IA:10 a=oexKYjalfGEA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=q2XNxBO46EK3obBdrt4A:9 a=wPNLvfGTeEIA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTPS id 8BF6514E; Tue, 30 Apr 2019 20:56:33 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x413uX67074032; Tue, 30 Apr 2019 20:56:33 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x413uUnV068470; Tue, 30 Apr 2019 20:56:32 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201905010356.x413uUnV068470@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Miroslav Lachman <000.fbsd@quip.cz> cc: David Chisnall , FreeBSD Stable , freebsd-ports@freebsd.org, =?UTF-8?Q?Goran_Meki=c4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org Subject: Re: CFT: FreeBSD Package Base In-Reply-To: Message from Miroslav Lachman <000.fbsd@quip.cz> of "Wed, 01 May 2019 00:03:44 +0200." <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz> Mime-Version: 1.0 Date: Tue, 30 Apr 2019 20:56:30 -0700 X-CMAE-Envelope: MS4wfN/xmWRaWQf4f3J4LUJQu3pZlq2R+heRrUgbSIStI3jW7zX/5fMm4b0wrONS30PxYTM3L4AEceAbaUlH0j4Ssk2hgu0ilk7uSIb2ZQm3B34wLKJrGhxU HwWqFVGWz0FbohtGB8vZe+ZhlwMlOl+nV4bTEhDv1vj7LNpbO3NZmQOnMb1uHsrEYnimp8U3o5Ldv1Btob7Oz5spUCDh01syPNS8efzuut3lZbEWglh78Iul sPyzs8ESB8rbpCG1Etz3uwemvekBrPSWvbdp4F0pPhs+i985u/KunSKaEPyPP+EnNKeIq6a2IowFuvaH5ee+dvx8eKz/lEv74EXMf7LyIodLisYvO6SVjh3b 05bnRfDMx0Uroy8xz142FesMv0fY2sueoDnS5Afs/XbfU78s8tk0bOhtge4PbZSjTvqk5TyKv8onRizm9SpnzbpATzmeHLo6AdBYkyy7AWk5lCjRClOg0mbc 1/pTZMJvQHzSrCWq X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Errors-To: owner-freebsd-current@freebsd.org Sender: owner-freebsd-current@freebsd.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1556683222; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:list-id:list-unsubscribe: list-subscribe:list-post; bh=4oCUGjfmMIx05OZmT8a3oiE4MzgrgGEEn4iBxbd66JM=; b=LoGcPYhcJXW6FaDMAGD7HbNIB8L4hy8qocpGSeFhrIcy+W2VC95CQWIAo7p3/LboxcsQN2 aAnIZ2MukoKwf4sn0AA9tA1vOFvof1tVqQIxo/xpb16znmIykGUr5Kh63Bs4/7w1l1lLei CZJBmQqjtp0ASNthePrCM/VX1pY+0IXx+8P9Zt0OJBbBII0BGSHbessCLw7Z6wnFhG+RBY OstJtiidF0oLHsTtAtqte2UmDK2nn7wcifBsOFsdLY8loCWpG6mJAz/XII/mLWQZcKVktG JLSobBjolNpu4YMkvegoAwZOY/Sc8eXnWqdLNc8sQH2WnG4s3Mdm8bQHkcpyPA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1556683222; a=rsa-sha256; cv=none; b=YXcufkDh6yWTqrCU0OEi31kosNacLMWMRVrtnGYI8pr8yDZbsZz6D60ctNltkQQ7+DXg3R eSgMHlw+Es8sMl45ORan1G9sWMyPaEohJyeqhkseto39oMqGmufZ8jd40YWWeY2uFLyAo+ tmXQLy48Os/kMtr4offaUd7oBZ+ZeNW0yev5uH38YUXYaBXyA7y9u0CSMBK5mM140irfn8 HwojazB0yzBdsqLgniHhJ7gdETlihG9OXdeM9Shk3y1EwlaMb1GejNi7tspsoC4xeVHXky Bzvb8NuLd2W9cZ8LrvtPWNvQI43JKzkDzjWaVMOXKzi6R/u3Oywuo4W7l/N0Dg== ARC-Authentication-Results: i=1; mx1.freebsd.org X-Rspamd-Queue-Id: 82D5B882A6 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=freebsd.org header.s=dkim header.b=SIY97bvi; spf=pass (mx1.freebsd.org: domain of root@sfc.wide.ad.jp designates 203.178.142.133 as permitted sender) smtp.mailfrom=root@sfc.wide.ad.jp X-Spamd-Result: default: False [-3.53 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.sfc.wide.ad.jp]; MV_CASE(0.50)[]; RCVD_COUNT_TWELVE(0.00)[12]; HAS_XAW(0.00)[]; DKIM_TRACE(0.00)[freebsd.org:+]; MX_GOOD(-0.01)[cached: ironport1.otemachi.wide.ad.jp]; RCPT_COUNT_SEVEN(0.00)[9]; NEURAL_HAM_SHORT(-0.87)[-0.872,0]; MAILLIST(-0.20)[mailman]; FORGED_SENDER(0.00)[Cy.Schubert@cschubert.com,root@sfc.wide.ad.jp]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.zen.spamhaus.org : 127.0.0.11]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; DATE_IN_PAST(1.00)[]; ASN(0.00)[asn:2500, ipnet:203.178.128.0/17, country:JP]; FROM_NEQ_ENVFROM(0.00)[Cy.Schubert@cschubert.com,root@sfc.wide.ad.jp]; ARC_ALLOW(-1.00)[i=1]; IP_SCORE(-1.94)[ip: (-7.03), ipnet: 203.178.128.0/17(-1.67), asn: 2500(-0.94), country: JP(-0.06)]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[freebsd.org:s=dkim]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; HAS_LIST_UNSUB(-0.01)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FORGED_SENDER_MAILLIST(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 10:53:57 -0000 In message <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz>, Miroslav = Lachman wri tes: > David Chisnall wrote on 2019/04/30 10:22: > > On 29/04/2019 21:12, Joe Maloney wrote: > >> With CFT version you chose to build, and package individual components = > >> such as sendmail with a port option.=C2=A0 That does entirely solve th= e = > >> problem of being able to reinstall sendmail after the fact without a = > >> rebuild of the userland (base) port but perhaps base flavors could = > >> solve that problem assuming flavors could extend beyond python. > > = > > This sounds very much like local optimisation. It's now easy to create = a = > > custom base image.=C2=A0 Great.=C2=A0 But how do I express dependencies= in ports = > > on a specific base configuration? This is easy if I depend on a specifi= c = > > base package, but how does this work in your model?=C2=A0 For example, = if I = > > have a package that depends on a library that is an optional part of th= e = > > base system, how do I express that pkg needs to either refuse to instal= l = > > it, or install a userland pkg that includes that library in place of my = > > existing version as part of the install process? > > = > > More importantly for the container use case, if I want to take a = > > completely empty jail and do pkg ins nginx (for example), what does the = > > maintainer of the nginx port need to do to express the minimum set of = > > the base system that needs to be installed to allow nginx to work? > > = > > One of the goals for the pkg base concept was to allow this kind of use = > > case, easily creating a minimal environment required to run a single = > > service. With a monolithic base package set, you're going to need some = > > mechanism other than packages to express the specific base subset = > > package that you need and I think that you need to justify why this = > > mechanism is better than using small individual packages. > > Will it not be maintainer's nightmare to take care of all the = > dependencies on the base packages for each port we have in the ports tree? No more than it is today. Remember, people have been doing this sort of = thing for decades. If the folks at Red Hat, Oracle (formerly Sun), and = IBM can do it, I'm sure we can too. The dependency lists will be = longer. We may require dependency lists that allow the choice of one of = many prereqs or coreqs. -- = Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. _______________________________________________ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu May 2 10:52:42 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 770C115905CA; Thu, 2 May 2019 10:52:42 +0000 (UTC) (envelope-from root@sfc.wide.ad.jp) Received: from mail1.sfc.wide.ad.jp (mail1.sfc.wide.ad.jp [203.178.142.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B443E880DC; Thu, 2 May 2019 10:52:40 +0000 (UTC) (envelope-from root@sfc.wide.ad.jp) Received: by mail1.sfc.wide.ad.jp (Postfix, from userid 0) id E3C141B449; Thu, 2 May 2019 19:40:06 +0900 (JST) X-Original-To: micchie@sfc.wide.ad.jp Received: from ironport1.sfc.wide.ad.jp (ironport1.sfc.wide.ad.jp [IPv6:2001:200:0:8803:203:178:142:150]) by mail1.sfc.wide.ad.jp (Postfix) with ESMTPS id 66BBB12334 for ; Wed, 1 May 2019 07:17:20 +0900 (JST) Received-SPF: None (ironport1.sfc.wide.ad.jp: no sender authenticity information available from domain of owner-freebsd-current@freebsd.org) identity=pra; client-ip=8.8.178.116; receiver=ironport1.sfc.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="owner-freebsd-current@freebsd.org"; x-conformance=sidf_compatible Received-SPF: Pass (ironport1.sfc.wide.ad.jp: domain of owner-freebsd-current@freebsd.org designates 8.8.178.116 as permitted sender) identity=mailfrom; client-ip=8.8.178.116; receiver=ironport1.sfc.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="owner-freebsd-current@freebsd.org"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:8.8.178.116 ip4:96.47.72.81 ip6:2001:1900:2254:206a::19:2 ip6:2610:1c1:1:606c::19:2 ~all" Received-SPF: None (ironport1.sfc.wide.ad.jp: no sender authenticity information available from domain of postmaster@mx2.freebsd.org) identity=helo; client-ip=8.8.178.116; receiver=ironport1.sfc.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="postmaster@mx2.freebsd.org"; x-conformance=sidf_compatible IronPort-SDR: W157G+hmN4zhwFNoCTJIsNahQelwABGA2MBHcai90tMBGEDfT1J2zSv3SXrU5NF3j51ciXyI4j 2Fx1QTieIxwUuiHDe2dFkTFlUCvi5BK6w= IronPort-PHdr: =?us-ascii?q?9a23=3ASIwYZRxiUit1Q8bXCy+O+j09IxM/srCxBDY+r6?= =?us-ascii?q?Qd2uISIJqq85mqBkHD//Il1AaPAdyCrawawLKG+4nbGkU4qa6bt34DdJEeHz?= =?us-ascii?q?Qksu4x2zIaPcieFEfgJ+TrZSFpVO5LVVti4m3peRMNQJW2aFLduGC94iAPER?= =?us-ascii?q?vjKwV1Ov71GonPhMiryuy+4ZLebxhViDanYL5/LRq6oRnSu8ILnYZsN6E9xw?= =?us-ascii?q?fTrHBVYepW32RoJVySnxb4+Mi9+YNo/jpTtfw86cNOSL32cKskQ7NWCjQmKH?= =?us-ascii?q?0169bwtRbfVwuP52ATXXsQnxFVHgXK9hD6XpP2sivnqupw3TSRMMPqQbwoXz?= =?us-ascii?q?mp8rxmQwH0higZKzE58XnXis1ug6JdvBKhvAF0z4rNbI2IKPZyYqbRcNUHTm?= =?us-ascii?q?RDQ8lRTTRMDJ6iYYsBD+QPPuhWoIfyqFQMtRuzBwuiCezyxjJGmnP40rE13P?= =?us-ascii?q?o6EQzdwAwvBMgDvGjIoNj7NqofV/2+wqnSzTXEavNbwTDz6InSchAmuvGMWa?= =?us-ascii?q?h/cdTMwkcpDQzKklWQqYn/MDOTy+8AtHSX4/Z8We61hW4osBpxrSW0y8c1l4?= =?us-ascii?q?nJh5oVxkrY+iR42Ys4I8CzRkB8Yd6hCpRQtieaOpN4QsM4XW5npj01yqYbtp?= =?us-ascii?q?KheygK0psnyhjCYPKEa4iF+hHuWPySLDtimX5oerGyiwyv/US9y+DwTsi53E?= =?us-ascii?q?hQoiZbndTArHQA2wLJ5sSZRPZx4EGs0iuV2Q/J8OFLO0U0mLLbK5E/xr4wkY?= =?us-ascii?q?IesUHMHyPogkn2grWWeVs69eez8OTof6/qppiGN497kg3+M6IuldKjAekgMQ?= =?us-ascii?q?UCRWuW9f6h2LDh/kD1WqhGguA3n6XBtZDVP8Ubpqq3Aw9P1YYj7g6yDzCo0d?= =?us-ascii?q?Qeg3YHNktJeBydj4f3JV7BO+v4Auqmj1uwjTdr3erGMaPnApXRNXXDnq3hcq?= =?us-ascii?q?hj5EFH0wo80MpT55VOCrEOOP7zQFP+tMTEDh8lNAy52+PnCNRh2YwCWWKPA7?= =?us-ascii?q?GWPbjJsV+G+u0jOeeNZI4ItznjMfQq/ePhgWU7mV8FLuGU24AKYiW4Auh+OB?= =?us-ascii?q?feJn7tmcsaV2kQswc0V+Csj0eNFjtaZnK3VqR74SknCYWgFsDNXI2giqeNmS?= =?us-ascii?q?ymBIdbYHwVN1aXDH29coyFX+sLOiyIL8JriTBBXqW7UYInyVS0uRbnxqF7Bu?= =?us-ascii?q?TO9yASrp6l08J6sPbOnxM//iAhEsKGzmuWRHt1lG5bezhj8KlhoEA14VeCyq?= =?us-ascii?q?5xh/gQQcBT6ulATAo8HbDGwuFQMPy0XRjOKISnUlGjF/CnGzYqStsvi/sHaE?= =?us-ascii?q?Z4EtPq2kibh2ykCaQZv6WWBYRy+bCKjCu5HNp013uTjPpptFIhWMYacDT+3v?= =?us-ascii?q?QlpSHOG47ElVmYnK+2dKMamRTA73qH0XHX7BsKURR3V73CR2xGIFDYppLm50?= =?us-ascii?q?/PCbaiT7Y/YUNa0cDXDKxMZ5XyiEleAu/5MYHZfGS8gSK+HxuFzKiAKYH3YX?= =?us-ascii?q?8Q2jn1BlIJngoI8TCBLwdtTjy5rTfmBSd1XUnqf1uq9OB/rH2hSUphyB6MYl?= =?us-ascii?q?ck2aC49xQOibmXVukJ07QYkCk7pjh+B1r72MjZWJKbvwQ0WqJabJsm5Utfk2?= =?us-ascii?q?LUswsoJpu7M6VrnUITaSxyrxmo2Q96B49bnY4ttnxzllhIJKmV0U1Me3aj5b?= =?us-ascii?q?6rZuaFDG708VjvarTR3EnZysrPvLwB47Inqlnu+gilUEg6oT1h1JFO3n2Q64?= =?us-ascii?q?+vbkJaWI/tUksx6xlxpq3LKig76YTO0HRwMK6y+jbc0tMtDeEhx16uZdBaeK?= =?us-ascii?q?+DEQbzFYUdCa3MYKQthF6kdFcKJu1f/rU4esi7beGL3rKDMvxtkTi9gSJA+o?= =?us-ascii?q?88mkOA+ixgS/LZio4fyqL9vEPPXDP9gVG998HvzNACNXdLQTr5kHG6V8ZLa6?= =?us-ascii?q?Z/fJgGEzKvOcqx3tJkm8erQHBTskGiC15A0cPvewLBCj61lQBWy0kTpmSq3C?= =?us-ascii?q?WiyDkh2TY2qKeFmSLUzunobxZCMXNRXmRkkX/iO4W5hcsWGk+yYEJ68XntrV?= =?us-ascii?q?a/3KVdqKlleiPUXkFFZW34NWxvWLC887GfedRC7o8Avz9aXe6nbRadULG38H?= =?us-ascii?q?54m2vzWmBZwj48bTSjvJ70ygd7hGyqJ3F2tHPFeMt0yE6FtubRTvNQwDcKAR?= =?us-ascii?q?JAp2WHVgqENsKytZWVjZDFquekSzvnTZNYNzbqwY7GvSf953U4SRG4mvmynJ?= =?us-ascii?q?XgH21YmWf8yN9nT2DGthvxaJLnkq+nK/hqeFVACkXx5sBhFsd5iIR4iJwL2H?= =?us-ascii?q?ccj4mY5jJX1ze1aIoLn/unMjIEXltpi5bN7RLg2VF/I37B3I//WniHg4Nga9?= =?us-ascii?q?S8fmIKy3c45sFOBr2T6e8Mli90r1yk6APJNKg7znFElqtosSdE0IRr8EI3wy?= =?us-ascii?q?6QA64fBxxZJyvqjxOS84n4t6tcInmufbX23UY4n8jyadPK6gxaRnv9fY8vWC?= =?us-ascii?q?Fq6cAqelXT1nDpr4/+ddTadt5WvQCOiRrGk8BbMp88nOEGwy19Niiu2B9tg/?= =?us-ascii?q?5+lhFo0Zyg6cKANGNg5ue3GBNVPCb5IsgO4Cvgi7p2kNyb0oq0EtNmADpBD/?= =?us-ascii?q?6KBbq4VTkVs/rgLQOHFjYx/2yaFbToFgia8E56rnjLHsPjJzSNKXIe19knWA?= =?us-ascii?q?iFKRkVnlUPRDtj1M1cdEji1In7fUx+/DxU+lPotk4G1LdzLxemGmbH+FX4LG?= =?us-ascii?q?dyEcfPakMPsElD/xuHYJLYsr01QHsersTEzkTFK3TFNV4QVThbAx3cXwK5ZO?= =?us-ascii?q?Tyt4OYreHKVLPvfaGGYK3Q+7YADq/anJn2gNI0oW7SPZfdZygwVbUt005HFy?= =?us-ascii?q?kjSf7UkDgOVSEb0hn1QZXA/EWa/St65oC+6vPtQAX1+dLJELVZd8pu9x3wiK?= =?us-ascii?q?vFNfbCzCB+YS1V0J8B3xqqgPAWwUIShidydjKsDaVItCjDS7jVk7NWCBhTYj?= =?us-ascii?q?16Nc9B5aYxlgdXPsuThtTw379+xvk7bjUNHUTmgd2sbNcWLnuVMUOdQkiROb?= =?us-ascii?q?mMPjGNxNv4IOu9RbBWkORIpkiwtDKcQCqBdnyIkzjkUQzqMPkZ3GfLekUB6M?= =?us-ascii?q?fkK0g+QXLuR9/ndBChZd5tijsuxqco3zXRO2VaLjlwfgVEovuS9XA94L03Fm?= =?us-ascii?q?pf435iNeTBlTye6rySLYkQvOEuDD55muhB7DEx1qFP4SdYbPJvlSbYtdIopE?= =?us-ascii?q?up2LrqqHIvQF9VpzBHiZje911lIrnc/4JcVGzs+QJXq2uKBhkAvNcjDcfg8f?= =?us-ascii?q?M1qJCHhOf4LzFM9MjR9M0XCp3PKc6JB3EmNALgBD/eCAZcEWybcFrHjkkYq8?= =?us-ascii?q?m8s2WPp8Fh+Jb2lZcEUbMdU0Y6RKtDWxZVWecaKZIyZQsK1LuSjcoG/3279U?= =?us-ascii?q?KDSt9Xs5rcWrSVG/q9cW/E34kBXAMBxPbDFapWNoD/3BU9OFxqxsLEAUfdV8?= =?us-ascii?q?tB5Cp7YV1srQ=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0GfAAAVychch3SyCAhmGQEBAQEBAQE?= =?us-ascii?q?BAQEBAQcBAQEBAQGBZU5wUGxVIAQzhBCEFIRningBAQaBECWDXoVugTyPORo?= =?us-ascii?q?YCwoCg3hfhhsbBwEENBMBAwEBAQMBAQIBAQEBAQIQAQEBCgsJCCkjDII6BQI?= =?us-ascii?q?DHwEGDk1rAQEBAQEBIwINZAECAwEBASARDAEBBAgCHgsBAgMBAgYBAQoOCgI?= =?us-ascii?q?CGAoEAgICAQEjAQkWEAYIBQIEAQECARgEgwEBgXYUAQIBAQquWnGBL4J5AQE?= =?us-ascii?q?FgkeDE4FUAwaBCyeKLoEdgQ+BB4E4DIFhfj6CYQEEgT4BAYMogliNQ5hgZQc?= =?us-ascii?q?BAYILgXGEJIQuBoMchE8GG5UvklGONoFmIYFXMxpzgmyCD4kDhUFwEHgckFJ?= =?us-ascii?q?IgXsBAQ?= X-IPAS-Result: =?us-ascii?q?A0GfAAAVychch3SyCAhmGQEBAQEBAQEBAQEBAQcBAQEBA?= =?us-ascii?q?QGBZU5wUGxVIAQzhBCEFIRningBAQaBECWDXoVugTyPORoYCwoCg3hfhhsbB?= =?us-ascii?q?wEENBMBAwEBAQMBAQIBAQEBAQIQAQEBCgsJCCkjDII6BQIDHwEGDk1rAQEBA?= =?us-ascii?q?QEBIwINZAECAwEBASARDAEBBAgCHgsBAgMBAgYBAQoOCgICGAoEAgICAQEjA?= =?us-ascii?q?QkWEAYIBQIEAQECARgEgwEBgXYUAQIBAQquWnGBL4J5AQEFgkeDE4FUAwaBC?= =?us-ascii?q?yeKLoEdgQ+BB4E4DIFhfj6CYQEEgT4BAYMogliNQ5hgZQcBAYILgXGEJIQuB?= =?us-ascii?q?oMchE8GG5UvklGONoFmIYFXMxpzgmyCD4kDhUFwEHgckFJIgXsBAQ?= X-IronPort-AV: E=Sophos;i="5.60,415,1549897200"; d="scan'208";a="7835002" Received: from mx2.freebsd.org ([8.8.178.116]) by ironport1.sfc.wide.ad.jp with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 01 May 2019 07:17:16 +0900 Received: from mx1.freebsd.org (mx1.freebsd.org [96.47.72.80]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx2.freebsd.org (Postfix) with ESMTPS id D1F7E70BEE; Tue, 30 Apr 2019 22:17:12 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 36E946AAC3; Tue, 30 Apr 2019 22:17:10 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1556662632; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id: list-unsubscribe:list-subscribe:list-post; bh=cMbAoIsSAQXA1HL2+KazKaL69s2xJHlUgjRGtqZyy34=; b=YE5g4kT5dUBM3jjgJa1cPowNhddLCtPUJEMe+WHHn4o+10iGIB8nr4dE4WV9jukZnXEazG YoN18l5EHeSNScT2ts510mZgHJx6mHfSpuyS4XIXcpHKRgP27501wvIUC+RZZoA56lzom8 lusTaIsoBYN0rvwI16AhwYqsOWjuoNevZRlbxB5+jrCl+fST3v1ePuc7uP5nZUfk+lEMyy STIaRb036pdrrXpwwbW1uRmRgDyabf8J5sV9oaHtdFHT39EjoR/Phpd6UeHXFgKV0rDj7q 6tL0gD0DYn5T4kxORoIMEDJDmZ+LbzqUyTawh5Yj8VyEBGHlNUjfPU7LiymSyg== Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA7A8159FF9D; Tue, 30 Apr 2019 22:13:31 +0000 (UTC) (envelope-from SRS0=7/jM=TA=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 11DA96A723; Tue, 30 Apr 2019 22:13:30 +0000 (UTC) (envelope-from SRS0=7/jM=TA=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id B397E28417; Wed, 1 May 2019 00:03:44 +0200 (CEST) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 51D3828411; Wed, 1 May 2019 00:03:43 +0200 (CEST) Subject: Re: CFT: FreeBSD Package Base To: David Chisnall Cc: Kris Moore , FreeBSD Stable , freebsd-pkg@freebsd.org, =?UTF-8?Q?Goran_Meki=c4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-ports@freebsd.org References: <201904291931.x3TJV73d079802@slippy.cwsent.com> <6EB65F49-47CD-48B2-8C58-BF387D8C7A58@ixsystems.com> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz> Date: Wed, 1 May 2019 00:03:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: X-Spam: Yes X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: owner-freebsd-current@freebsd.org Sender: owner-freebsd-current@freebsd.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1556662632; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id: list-unsubscribe:list-subscribe:list-post; bh=cMbAoIsSAQXA1HL2+KazKaL69s2xJHlUgjRGtqZyy34=; b=qqTynqoOg6Me6PtXZicIUutDA6ISa7JmOIN+iEF2OpFmP53QhuoAY05wh2IDoUPOgHK2t0 9+08a+OgU5xm8Zgt4jYyrsv4YPfjAVAtll3CyabexVSl9TAk3JLuIUxo6juP+utWfp9IgW UDa9Hu97Ql1VTI0qf0KrJSURX5yN2G8yUGa89wN58BsKSQCzM7/SJXcUIpQC0z29P2eFZY 2TO/aq1+zVxDldjd3X5p4BWjpf3WtJkvkBWSi6cqEUcECJAqJmEh78Qjp9b+xHZMVnITAP dOBWglvxL8VMbt3UqIFeUuCQ3PGui8tntCOp4+ume1Ft+eokmk7FdLRGMnSZXQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1556662632; a=rsa-sha256; cv=none; b=JB3Qhye2cpoM9b+Q7MOs1J4IwZ7WIiTeAkRXZhYr/ahnc44YhbO8R4SY2Mm67U6LgJku36 62j2tRSm+DSya8352R73vkVMA1yiSJmI+x3Z4S8wA7fYzfGLR1xt9zzJGgwA55nxA+2eY8 4J35o6N7EvzCKGYjXdQs/PKHxa42x1UOHXWuKWawkD1EdWd2DiTcAjOd3jPXUtcJFMHqXG i/x3hLuEDIVaSdn8vQjVJmKfZg5NWPVGj5xpAGg1W1TSnGcPeugewf4gxz0hLPYZSuBr+0 9g8uFNkteNov5nwOajVqo/ZkEO795SbTCaMVSJy2JziSc62XjX1tIjowaPmi1A== ARC-Authentication-Results: i=1; mx1.freebsd.org X-Rspamd-Queue-Id: B443E880DC X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=freebsd.org header.s=dkim header.b=YE5g4kT5; spf=pass (mx1.freebsd.org: domain of root@sfc.wide.ad.jp designates 203.178.142.133 as permitted sender) smtp.mailfrom=root@sfc.wide.ad.jp X-Spamd-Result: default: False [4.95 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FORGED_SENDER_MAILLIST(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.sfc.wide.ad.jp]; GREYLIST(0.00)[pass,body]; DKIM_TRACE(0.00)[freebsd.org:+]; MX_GOOD(-0.01)[cached: ironport1.otemachi.wide.ad.jp]; RCPT_COUNT_SEVEN(0.00)[9]; MAILLIST(-0.20)[mailman]; FORGED_SENDER(0.00)[000.fbsd@quip.cz,root@sfc.wide.ad.jp]; MIME_BASE64_TEXT(0.10)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; DATE_IN_PAST(1.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,root@sfc.wide.ad.jp]; ASN(0.00)[asn:2500, ipnet:203.178.128.0/17, country:JP]; IP_SCORE(-1.78)[ip: (-6.60), ipnet: 203.178.128.0/17(-1.45), asn: 2500(-0.80), country: JP(-0.06)]; R_DKIM_ALLOW(-0.20)[freebsd.org:s=dkim]; SPAM_FLAG(5.00)[]; FROM_HAS_DN(0.00)[]; ARC_ALLOW(-1.00)[i=1]; NEURAL_HAM_LONG(-0.25)[-0.253,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[quip.cz]; HAS_LIST_UNSUB(-0.01)[]; NEURAL_SPAM_MEDIUM(0.73)[0.729,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.37)[0.371,0]; RCVD_COUNT_SEVEN(0.00)[10]; SUSPICIOUS_RECIPS(1.50)[] X-BeenThere: freebsd-stable@freebsd.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 10:52:42 -0000 RGF2aWQgQ2hpc25hbGwgd3JvdGUgb24gMjAxOS8wNC8zMCAxMDoyMjoKPiBPbiAyOS8wNC8yMDE5 IDIxOjEyLCBKb2UgTWFsb25leSB3cm90ZToKPj4gV2l0aCBDRlQgdmVyc2lvbiB5b3UgY2hvc2Ug dG8gYnVpbGQsIGFuZCBwYWNrYWdlIGluZGl2aWR1YWwgY29tcG9uZW50cyAKPj4gc3VjaCBhcyBz ZW5kbWFpbCB3aXRoIGEgcG9ydCBvcHRpb24uwqAgVGhhdCBkb2VzIGVudGlyZWx5IHNvbHZlIHRo ZSAKPj4gcHJvYmxlbSBvZiBiZWluZyBhYmxlIHRvIHJlaW5zdGFsbCBzZW5kbWFpbCBhZnRlciB0 aGUgZmFjdCB3aXRob3V0IGEgCj4+IHJlYnVpbGQgb2YgdGhlIHVzZXJsYW5kIChiYXNlKSBwb3J0 IGJ1dCBwZXJoYXBzIGJhc2UgZmxhdm9ycyBjb3VsZCAKPj4gc29sdmUgdGhhdCBwcm9ibGVtIGFz c3VtaW5nIGZsYXZvcnMgY291bGQgZXh0ZW5kIGJleW9uZCBweXRob24uCj4gCj4gVGhpcyBzb3Vu ZHMgdmVyeSBtdWNoIGxpa2UgbG9jYWwgb3B0aW1pc2F0aW9uLiBJdCdzIG5vdyBlYXN5IHRvIGNy ZWF0ZSBhIAo+IGN1c3RvbSBiYXNlIGltYWdlLsKgIEdyZWF0LsKgIEJ1dCBob3cgZG8gSSBleHBy ZXNzIGRlcGVuZGVuY2llcyBpbiBwb3J0cyAKPiBvbiBhIHNwZWNpZmljIGJhc2UgY29uZmlndXJh dGlvbj8gVGhpcyBpcyBlYXN5IGlmIEkgZGVwZW5kIG9uIGEgc3BlY2lmaWMgCj4gYmFzZSBwYWNr YWdlLCBidXQgaG93IGRvZXMgdGhpcyB3b3JrIGluIHlvdXIgbW9kZWw/wqAgRm9yIGV4YW1wbGUs IGlmIEkgCj4gaGF2ZSBhIHBhY2thZ2UgdGhhdCBkZXBlbmRzIG9uIGEgbGlicmFyeSB0aGF0IGlz IGFuIG9wdGlvbmFsIHBhcnQgb2YgdGhlIAo+IGJhc2Ugc3lzdGVtLCBob3cgZG8gSSBleHByZXNz IHRoYXQgcGtnIG5lZWRzIHRvIGVpdGhlciByZWZ1c2UgdG8gaW5zdGFsbCAKPiBpdCwgb3IgaW5z dGFsbCBhIHVzZXJsYW5kIHBrZyB0aGF0IGluY2x1ZGVzIHRoYXQgbGlicmFyeSBpbiBwbGFjZSBv ZiBteSAKPiBleGlzdGluZyB2ZXJzaW9uIGFzIHBhcnQgb2YgdGhlIGluc3RhbGwgcHJvY2Vzcz8K PiAKPiBNb3JlIGltcG9ydGFudGx5IGZvciB0aGUgY29udGFpbmVyIHVzZSBjYXNlLCBpZiBJIHdh bnQgdG8gdGFrZSBhIAo+IGNvbXBsZXRlbHkgZW1wdHkgamFpbCBhbmQgZG8gcGtnIGlucyBuZ2lu eCAoZm9yIGV4YW1wbGUpLCB3aGF0IGRvZXMgdGhlIAo+IG1haW50YWluZXIgb2YgdGhlIG5naW54 IHBvcnQgbmVlZCB0byBkbyB0byBleHByZXNzIHRoZSBtaW5pbXVtIHNldCBvZiAKPiB0aGUgYmFz ZSBzeXN0ZW0gdGhhdCBuZWVkcyB0byBiZSBpbnN0YWxsZWQgdG8gYWxsb3cgbmdpbnggdG8gd29y az8KPiAKPiBPbmUgb2YgdGhlIGdvYWxzIGZvciB0aGUgcGtnIGJhc2UgY29uY2VwdCB3YXMgdG8g YWxsb3cgdGhpcyBraW5kIG9mIHVzZSAKPiBjYXNlLCBlYXNpbHkgY3JlYXRpbmcgYSBtaW5pbWFs IGVudmlyb25tZW50IHJlcXVpcmVkIHRvIHJ1biBhIHNpbmdsZSAKPiBzZXJ2aWNlLiBXaXRoIGEg bW9ub2xpdGhpYyBiYXNlIHBhY2thZ2Ugc2V0LCB5b3UncmUgZ29pbmcgdG8gbmVlZCBzb21lIAo+ IG1lY2hhbmlzbSBvdGhlciB0aGFuIHBhY2thZ2VzIHRvIGV4cHJlc3MgdGhlIHNwZWNpZmljIGJh c2Ugc3Vic2V0IAo+IHBhY2thZ2UgdGhhdCB5b3UgbmVlZCBhbmQgSSB0aGluayB0aGF0IHlvdSBu ZWVkIHRvIGp1c3RpZnkgd2h5IHRoaXMgCj4gbWVjaGFuaXNtIGlzIGJldHRlciB0aGFuIHVzaW5n IHNtYWxsIGluZGl2aWR1YWwgcGFja2FnZXMuCgpXaWxsIGl0IG5vdCBiZSBtYWludGFpbmVyJ3Mg bmlnaHRtYXJlIHRvIHRha2UgY2FyZSBvZiBhbGwgdGhlIApkZXBlbmRlbmNpZXMgb24gdGhlIGJh c2UgcGFja2FnZXMgZm9yIGVhY2ggcG9ydCB3ZSBoYXZlIGluIHRoZSBwb3J0cyB0cmVlPwoKTWly b3NsYXYgTGFjaG1hbgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwpmcmVlYnNkLWN1cnJlbnRAZnJlZWJzZC5vcmcgbWFpbGluZyBsaXN0Cmh0dHBzOi8vbGlz dHMuZnJlZWJzZC5vcmcvbWFpbG1hbi9saXN0aW5mby9mcmVlYnNkLWN1cnJlbnQKVG8gdW5zdWJz Y3JpYmUsIHNlbmQgYW55IG1haWwgdG8gImZyZWVic2QtY3VycmVudC11bnN1YnNjcmliZUBmcmVl YnNkLm9yZyIK From owner-freebsd-stable@freebsd.org Thu May 2 10:47:53 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B12591590333; Thu, 2 May 2019 10:47:53 +0000 (UTC) (envelope-from root@sfc.wide.ad.jp) Received: from mail1.sfc.wide.ad.jp (mail1.sfc.wide.ad.jp [203.178.142.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 18D2687BEF; Thu, 2 May 2019 10:47:51 +0000 (UTC) (envelope-from root@sfc.wide.ad.jp) Received: by mail1.sfc.wide.ad.jp (Postfix, from userid 0) id 82A8A1AA95; Thu, 2 May 2019 19:40:02 +0900 (JST) X-Original-To: micchie@sfc.wide.ad.jp Received: from ironport1.otemachi.wide.ad.jp (ironport1.otemachi.wide.ad.jp [203.178.140.206]) by mail1.sfc.wide.ad.jp (Postfix) with ESMTP id 979FC1232C for ; Tue, 30 Apr 2019 20:26:14 +0900 (JST) Received-SPF: None (ironport1.otemachi.wide.ad.jp: no sender authenticity information available from domain of owner-freebsd-current@freebsd.org) identity=pra; client-ip=2001:1900:2254:206a::19:2; receiver=ironport1.otemachi.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="owner-freebsd-current@freebsd.org"; x-conformance=sidf_compatible Received-SPF: Pass (ironport1.otemachi.wide.ad.jp: domain of owner-freebsd-current@freebsd.org designates 2001:1900:2254:206a::19:2 as permitted sender) identity=mailfrom; client-ip=2001:1900:2254:206a::19:2; receiver=ironport1.otemachi.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="owner-freebsd-current@freebsd.org"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:8.8.178.116 ip4:96.47.72.81 ip6:2001:1900:2254:206a::19:2 ip6:2610:1c1:1:606c::19:2 ~all" Received-SPF: None (ironport1.otemachi.wide.ad.jp: no sender authenticity information available from domain of postmaster@mx2.freebsd.org) identity=helo; client-ip=2001:1900:2254:206a::19:2; receiver=ironport1.otemachi.wide.ad.jp; envelope-from="owner-freebsd-current@freebsd.org"; x-sender="postmaster@mx2.freebsd.org"; x-conformance=sidf_compatible X-Ironport-Dmarc-Check-Result: validskip IronPort-SDR: eIs08nZC354J2ci8qBcWPcCYgy5msBzdmXCPDsecZ+2MWJQWLONXFDMrLyjzSSbqi8N1eDRzVq HNP9f5EhKSRgpzqKDRnlxUpey1FfTt89Y= IronPort-PHdr: =?us-ascii?q?9a23=3A9U6zSxG0Egi5nWVtOlzQ0J1GYnF66ZnXHytQx4?= =?us-ascii?q?Quj69HbqWk+cS4bR7BsKw+xFacR57SrulEgrCL7fOoUjka7JKIoH0ON4ZBUx?= =?us-ascii?q?YVhMlF1wB1DtSIBEfgILjtdSNpVN8XWFxk+iLeUylJTZ+jOgOM+yztt2BBUh?= =?us-ascii?q?WqOxByI7b7AInX3d+60ey/4c/IbgNEiSDve75qKATzsQKDrdQcg4ZpNvQq1x?= =?us-ascii?q?HLr3BUKehf2W46KVuPkkSZhKb4+ZN46C1ftrcg+5xFXL7hdqs1CL1UXighYX?= =?us-ascii?q?wytsbg5xLIHw6O43ZZTGgMiRtPBVrA7RT3G4n8qDD7sewvvUv7dcbxS6w5Xy?= =?us-ascii?q?jn87ZzRU3jjyEbOjgltX3GksEi6cA66BmmoQByxZKRfJucMuFwc7mYcNURQm?= =?us-ascii?q?ZAWMkDXCtHHo66c84TFeMHJuhVtczwoVAJoRq4ClvJZoKnyzkNnHLu2bx/z+?= =?us-ascii?q?NzEAeDxgE4G8NLrHOGttj7Ob0fV+u41/vSwDvOdf5Q0jrmuozJaQxkoP3eON?= =?us-ascii?q?AROc7Q0lVqCgXek1+LopbkNT7H0uUWry2A8uB7XviugWN1rwBqvn23wdww2e?= =?us-ascii?q?yr54gVnwvZzQBg56ILBcGJcUFre861FrdUrHqgDttZfJglcjBrigoIib4mpJ?= =?us-ascii?q?ubYQFfm/FFphSKYNusV5O38zjfZu23GC5ggnN+fp6HoAey0GWakc/le5Pp8X?= =?us-ascii?q?QagzV8qYbTmU4H80Tx9MWiFrxy/UKl92eQ6xHoyMJtfWYIi8+5St043v0gvL?= =?us-ascii?q?wjrmTGQw31hXnWgaKvVnQV4Nqq8fqgXqvk/I3GE65KigPDEaEco9e/Av85HF?= =?us-ascii?q?dXGHK54e+t9rzt213ZZZNBqOxT8MiIrJzDff8xgv+rOANX2aI+1k+cJg/h/t?= =?us-ascii?q?AAvkMJCVRsdRG43o/bJXz3DPanKLSBrmydsSVo4fzKD6DHLZzIKkHPjbDTbJ?= =?us-ascii?q?N0jiw9zTJ0lt0cvpxGUOsoJfDae0nsidXUU0MZDyy+yMLiKM1xi74ESGGdWI?= =?us-ascii?q?STNfz57HiHz88zYM6nY45JsxmkMMAGxtfVlSMymTp/HPmK0atIQnG7WcY6G2?= =?us-ascii?q?CfZlHdhuoZLTshuyg5F+vRulK5fSxRSmmIB/xnuCM1ONmMI53xZo6Agp+cxw?= =?us-ascii?q?PqNcJJMzUVQkDJEG3vcZ2DQetJci+JP8t9xyQNTqPyEdVz/AyytAL817tsJ/?= =?us-ascii?q?bV/Stdj5/4ydxp/LePzEMz7ztyF8OHzzjLSH15m2oDRz5ww6FzrAp0xhGByf?= =?us-ascii?q?swmORWQPpU4f4BSQImLdjE1eUvDM/0VxiEcM2ERVy9TpCtHCsrQ9Qt69EUbk?= =?us-ascii?q?t3ANnkiQrMjGKxG7FAr7WQH9Qv977EmX34JsJz0XHDga4+iVQ8B8VVOGCsna?= =?us-ascii?q?I6/BXJHYPPj22Xjaunf78QminX+zTL1nKA6WdfVgM4SqDZRTYfa0/R+Mz+/V?= =?us-ascii?q?/HRqSyBK4PNxsbj8CYJ6ZAcdGvik9JF7/vO9XbNnq4gHz4RQ2JybWFcJfwdi?= =?us-ascii?q?0D0T/cBkkJn0Fb/XuPOQUkQCb0i2zEATFkDl6pZFnjqLMskn6wQ04qwg3PVH?= =?us-ascii?q?VPjOXtqDgSg/HUC/IJ2bQcvz098nNzAFu41tnWCpyfpgZsOqxQJ9Elsh9B0i?= =?us-ascii?q?rCugpxM4bFTegqj0MCcwlxo0Ll1glmQoRGn88wqXo2zQ10YauG2VJFfjmc0N?= =?us-ascii?q?j+ILrSYmX1+RmubebR1DS8mJ6R5aEC8rE2sVjuthqlU0E/6W9q1cV93WGR65?= =?us-ascii?q?/RAEwVS52wGkc7+h5mpq3LNzEn7tCxtzUkOq21vznentMxUbF9kk/yJoYHaP?= =?us-ascii?q?HeTlKuQZdCWcH8erV7yQL6C3BMdPpf/6M1Ic68Ivqc36izO/x8xXShkWlC6Y?= =?us-ascii?q?t90wST9iZxDOfEmZQdkJT6lkOKUSnxiFC5v4X5g4dBMHsVAWyy023qHoNcbL?= =?us-ascii?q?Z+OI0WE32jJdGfyM9zipT2Vzhf7lHpVDZkkIe5PAGfaVDwx1ga0lsRrGbilz?= =?us-ascii?q?G1wjZvn3Qis7CF0SrS6+r4eRcNIWINQ3NtxwSJQ8D8n5URW06maBIsnR2u6B?= =?us-ascii?q?PhxqRVk694KnHaXUZCeyWvdTNYX6C9t6SPb4t00L1163QFaO2nehjaS6T6pQ?= =?us-ascii?q?MdyT+6WWxCyTUyfDGr/Ir4kxE8gW7bLm4h5HbefMhxw1/Y6rm+DbZWwTwLXm?= =?us-ascii?q?x2kzTaDEOwetK058+ZkYvrvPqzWmi6WttUayajwY6btSS97HFnGnjd17jpwo?= =?us-ascii?q?W+Q1FriXepjYM3CXeS9Ej1MNuyhvbgeet8NlFlHlr999Z3Fslln408iYtRvB?= =?us-ascii?q?pSzpSZ8HwbkHviZNBS2Kbwdn0IFnYAx9/Y5hSg2VU2dCrRntCgBy3FkJE7Oo?= =?us-ascii?q?XoODtI03BvtpEYUvTxjvQMhyZ+r1umoBiEYOJ6mC8a0+d3rnkGjuQAswosiD?= =?us-ascii?q?qcDr1UFEUePzS/8nbAp924sqhTY36iNLaq008r19e+CryY5w9GVXvzYZtnFj?= =?us-ascii?q?Jq9Mh5LHrCynD+4JroPt7KYphA03/c2weFlOVTJJ8rw7ANmixqIyfzp3Aowf?= =?us-ascii?q?U3yBdzx4u3upSvJX9g8aiiDlhfLDK/NKZxsnn9yK1ZmMiRxYWmGJ5sTy4KUJ?= =?us-ascii?q?XfRvWtCDsOtP7jOlXGAHgmp3ycA7aaAR6H5RIssSfUC57yfSLyRjFR3ZB4SR?= =?us-ascii?q?KaPkAanA0EQGBwgMsiDg7zjM35LBUpoGBXvwGo7EIVlKppL0WtAzmZ+FjuNW?= =?us-ascii?q?1yGMH6TlIe7xkctR2NaZXMt7MrR3MCpMD9/lbVdGXJNV0ZXztBW1TaVQq9Z+?= =?us-ascii?q?H8vNKZobHKWLLgJKmcOOTc86RIXvOMj8v1gLFr9DuNKMiDe0JaIaZmhRhlWn?= =?us-ascii?q?Z0U4TYgTIJUysNi3uLaNKWoR63+iwxscGz/bLiVETt/dnHDbwaKthp9x2s5M?= =?us-ascii?q?XLf+eNmCZ0LypZ3ZIQ1DfJzrYYxlsblyBpcXGkD70BsSfHSK+YlLVQClYXbC?= =?us-ascii?q?Z6Nc0A6KxZvEEFIcnAltb8zaJ1lNYwGw0DW0bsl8uyaYoNOW78fFLLCUCXNa?= =?us-ascii?q?iXcD3Gx8akBMH0AbZUjehSq1ixoWPBSxe6bG3Sz2G7DUrybbMW1H+Sbk4H6d?= =?us-ascii?q?PnIXMPQSD5QdnrawO2Kop2lzow2boumiqMMnIRNDF3c0ILsbyc4GVRjrN9Az?= =?us-ascii?q?8kjDItIO+aliKe9+SdJIwRtK4hBTtymvgf7m43xrFO4GRPWeZvlS3OhtB0rl?= =?us-ascii?q?ysieXJzSBoGkkryH4DlMeQsENuNL+MvIFHQmrB9QkR4H+4DgRQ4dV/B9DloK?= =?us-ascii?q?wWzcLA3vGWSn8K45ff+s0SANLRIcSMPS86MBbnLzXTCRMMUT+hMWyG2x5t1c?= =?us-ascii?q?qK/3jQlaAU74D2kcNVGLFBWVE/Df9cDV5qToRbccVHGwg8mLvet/YmoH+3rR?= =?us-ascii?q?3fXsJf58uVWOifDP/0JHCel7YWPkJVk4O9FpwaM8jA42InalR+m92VSU/AB5?= =?us-ascii?q?ZDuCBsZBQ45kJX/yomQw=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A8G2AAAYIchcfQAZASCCktDAagDkgAJ?= =?us-ascii?q?mHAEBAQQBAQcEAQGBUgYBAQsBgT0kLFsRVSAEM5czUAEBBoEICCWDXoVujwS?= =?us-ascii?q?BbhsBARgLCgKDeIZ6GwcBBDEIDgEBAgEBAQMBAQIBAQECAhABAQsUCEwMgnh?= =?us-ascii?q?NawEBAQEBASMCDWQBAgMBAQE3BgEBBAgCHgsBAgMBAgYBAQoOCh4ICAMBIwE?= =?us-ascii?q?JJgYBBwUCBAEBAQEBGASDAQGBdhQBAwutLIIggnkBAQWCR4RoAwYTgR8Biiy?= =?us-ascii?q?BHRd4gQeBOAyBYX4+gmEBBIE+AQGGAI0ThWCTLGUJgguGFIQuBoMchE4hjAe?= =?us-ascii?q?JJowNhkKGPYdRAgQCBAUCFYFRATWBV00wCDuCbIIPiQOFQHEQdR+QU4JDAQE?= X-IPAS-Result: =?us-ascii?q?A8G2AAAYIchcfQAZASCCktDAagDkgAJmHAEBAQQBAQcEA?= =?us-ascii?q?QGBUgYBAQsBgT0kLFsRVSAEM5czUAEBBoEICCWDXoVujwSBbhsBARgLCgKDe?= =?us-ascii?q?IZ6GwcBBDEIDgEBAgEBAQMBAQIBAQECAhABAQsUCEwMgnhNawEBAQEBASMCD?= =?us-ascii?q?WQBAgMBAQE3BgEBBAgCHgsBAgMBAgYBAQoOCh4ICAMBIwEJJgYBBwUCBAEBA?= =?us-ascii?q?QEBGASDAQGBdhQBAwutLIIggnkBAQWCR4RoAwYTgR8BiiyBHRd4gQeBOAyBY?= =?us-ascii?q?X4+gmEBBIE+AQGGAI0ThWCTLGUJgguGFIQuBoMchE4hjAeJJowNhkKGPYdRA?= =?us-ascii?q?gQCBAUCFYFRATWBV00wCDuCbIIPiQOFQHEQdR+QU4JDAQE?= X-IronPort-AV: E=Sophos;i="5.60,413,1549897200"; d="scan'208";a="526421" Received: from mx2.freebsd.org ([IPv6:2001:1900:2254:206a::19:2]) by ironport1.otemachi.wide.ad.jp with ESMTP/TLS/ECDHE-RSA-AES128-GCM-SHA256; 30 Apr 2019 19:22:53 +0900 Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mx1.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx2.freebsd.org (Postfix) with ESMTPS id 752DF6CBDC; Tue, 30 Apr 2019 10:22:07 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BFA6C97B7C; Tue, 30 Apr 2019 10:22:04 +0000 (UTC) (envelope-from owner-freebsd-current@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1556619726; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id: list-unsubscribe:list-subscribe:list-post; bh=3uPMqOLVHtYQgxrxynDls3RQ8GV3oY9kZDZ+kHumHuA=; b=cOCLsWyBTZcNQqpnisnDPOyhIER3JqE25IJ8EwQS7wGVbL0JveM4/gkmlF9wTySJKA4NTv +qt+xBNXwHm5JkuTHeeQHYYrZjrv/Tvj2z2LIVhjUKmZQshksiF71/rJpcyJifNxHMNcNy 9tdq0SBL9zSdjcLuU1KAP/FlpKRfMnAekW9oZvEQDn7M+VEPYIzkNddYuNGamlQduoZyMa mP/3e+1fz22AmjAInuUad54HTHbcLmHpd5Urykz2Cbm7C5OgAEt+OCoi4cG1sPKIASp4ch 8KdX7Bik/Z4PMnoCYlUpT4CmJo3oSZgSDzjEwR0636zfX3BFrZHfEjKusL9C7w== Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8927158867E; Tue, 30 Apr 2019 08:23:12 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from theravensnest.org (theravensnest.org [46.226.110.62]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "theravensnest.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22C2A931AD; Tue, 30 Apr 2019 08:23:11 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from [10.164.186.150] ([167.220.197.150]) (authenticated bits=0) by theravensnest.org (8.15.2/8.15.2) with ESMTPSA id x3U8MAWr015104 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 30 Apr 2019 08:22:10 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: mail: Host [167.220.197.150] claimed to be [10.164.186.150] Subject: Re: CFT: FreeBSD Package Base To: Joe Maloney , Cy Schubert Cc: "Rodney W. Grimes" , Kris Moore , FreeBSD Stable , freebsd-ports@freebsd.org, =?UTF-8?Q?Goran_Meki=c4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, Emmanuel Vadot References: <201904291931.x3TJV73d079802@slippy.cwsent.com> <6EB65F49-47CD-48B2-8C58-BF387D8C7A58@ixsystems.com> From: David Chisnall Message-ID: Date: Tue, 30 Apr 2019 09:22:56 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <6EB65F49-47CD-48B2-8C58-BF387D8C7A58@ixsystems.com> Content-Language: en-GB X-Mailman-Approved-At: Tue, 30 Apr 2019 10:18:36 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: owner-freebsd-current@freebsd.org Sender: owner-freebsd-current@freebsd.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1556619726; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id: list-unsubscribe:list-subscribe:list-post; bh=3uPMqOLVHtYQgxrxynDls3RQ8GV3oY9kZDZ+kHumHuA=; b=g4gbI4tF+/541QaCNzdPW/8LRGuA+PoEp0ah9c1fu35Ve9czp4mOhHlEdhWziFbcMi5n0Y 2POyiFKnnVlYSaLCRrvMcnn+SLNvODlCpyPvkav6Vq5POiLnblaI3wAT64DTU/LiW26Flz l2Iu8DRmtrPLOGO0FyeQdMECGN76V1a1UcMHQLJSlru+mIOvlXUdVLlgSYm4LO2UJvTGng iGNkO677hHmmSOxXf/bu5D+27ip3n9SVpHPNX7f+5JkeuAhqtDbtffeQ+2VfJz6tQJDjZW LyZtGyR/z46AfT9BoE/5p9fMaYRSCvIDhN69D9k7VcKbhJKNx06xGuEchqQaLw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1556619726; a=rsa-sha256; cv=none; b=Kg+ptVck8q6po4TCQr2gAt0NuNEYt+ot69Q2MxR86D99LgRUTnCTl3uyWZh0uqJ9cyh5Ih Dc19+UW5/dU33txeVvzIiOVj/cLNPnmqsXYFNFcTjyOK+oOmQmeXwv2Cm0AuvIVHu9X0KN fTHL9JvNmOejAGy+/0oHiQU3P9bjiCRDFcAtYU/vgYK7/7jUzoNrv8xOY0DV/ZaQsQQOnz rKtNPBjF/1GKrvyH0wEVlqNIPgSFHM6ZCLcWJKhb7aTUwQoNAe9EASpFp56MecR9XLxIsN 6xSQdaSEIW+i2ABKbMaPr7lNhZHbzqvr9G5tX2C+9sPI2Z/0L2mtNSZdhkdHsQ== ARC-Authentication-Results: i=1; mx1.freebsd.org X-Rspamd-Queue-Id: 18D2687BEF X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=freebsd.org header.s=dkim header.b=cOCLsWyB; spf=pass (mx1.freebsd.org: domain of root@sfc.wide.ad.jp designates 203.178.142.133 as permitted sender) smtp.mailfrom=root@sfc.wide.ad.jp X-Spamd-Result: default: False [-3.71 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.sfc.wide.ad.jp]; HAS_XAW(0.00)[]; DKIM_TRACE(0.00)[freebsd.org:+]; MX_GOOD(-0.01)[cached: ironport1.otemachi.wide.ad.jp]; NEURAL_HAM_SHORT(-0.92)[-0.923,0]; MAILLIST(-0.20)[mailman]; FORGED_SENDER(0.00)[theraven@FreeBSD.org,root@sfc.wide.ad.jp]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; DATE_IN_PAST(1.00)[]; ASN(0.00)[asn:2500, ipnet:203.178.128.0/17, country:JP]; FROM_NEQ_ENVFROM(0.00)[theraven@FreeBSD.org,root@sfc.wide.ad.jp]; ARC_ALLOW(-1.00)[i=1]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[freebsd.org:s=dkim]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[FreeBSD.org]; HAS_LIST_UNSUB(-0.01)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; IP_SCORE(-1.56)[ip: (-5.91), ipnet: 203.178.128.0/17(-1.19), asn: 2500(-0.65), country: JP(-0.06)]; RCVD_COUNT_SEVEN(0.00)[9]; FORGED_SENDER_MAILLIST(0.00)[]; SUSPICIOUS_RECIPS(1.50)[] X-Mailman-Approved-At: Thu, 02 May 2019 11:21:24 +0000 X-BeenThere: freebsd-stable@freebsd.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 10:47:54 -0000 On 29/04/2019 21:12, Joe Maloney wrote: > With CFT version you chose to build, and package individual components such as sendmail with a port option. That does entirely solve the problem of being able to reinstall sendmail after the fact without a rebuild of the userland (base) port but perhaps base flavors could solve that problem assuming flavors could extend beyond python. This sounds very much like local optimisation. It's now easy to create a custom base image. Great. But how do I express dependencies in ports on a specific base configuration? This is easy if I depend on a specific base package, but how does this work in your model? For example, if I have a package that depends on a library that is an optional part of the base system, how do I express that pkg needs to either refuse to install it, or install a userland pkg that includes that library in place of my existing version as part of the install process? More importantly for the container use case, if I want to take a completely empty jail and do pkg ins nginx (for example), what does the maintainer of the nginx port need to do to express the minimum set of the base system that needs to be installed to allow nginx to work? One of the goals for the pkg base concept was to allow this kind of use case, easily creating a minimal environment required to run a single service. With a monolithic base package set, you're going to need some mechanism other than packages to express the specific base subset package that you need and I think that you need to justify why this mechanism is better than using small individual packages. David _______________________________________________ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu May 2 11:09:10 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E43351591609; Thu, 2 May 2019 11:09:09 +0000 (UTC) (envelope-from v.velox@vvelox.net) Received: from vulpes.vvelox.net (vulpes.vvelox.net [96.95.67.25]) by mx1.freebsd.org (Postfix) with ESMTP id 0AF2D893B7; Thu, 2 May 2019 11:09:08 +0000 (UTC) (envelope-from v.velox@vvelox.net) Received: from vvelox.net (localhost [127.0.0.1]) (Authenticated sender: kitsune) by vulpes.vvelox.net (Postfix) with ESMTPA id 0D4B023355C1; Thu, 2 May 2019 06:09:00 -0500 (CDT) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Thu, 02 May 2019 06:09:00 -0500 From: "Zane C. B-H." To: Miroslav Lachman <000.fbsd@quip.cz> Cc: David Chisnall , FreeBSD Stable , freebsd-ports@freebsd.org, =?UTF-8?Q?Gor?= =?UTF-8?Q?an_Meki=C4=87?= , freebsd-hackers@freebsd.org, FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, owner-freebsd-pkgbase@freebsd.org Subject: Re: CFT: FreeBSD Package Base In-Reply-To: <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz> References: <201904291931.x3TJV73d079802@slippy.cwsent.com> <6EB65F49-47CD-48B2-8C58-BF387D8C7A58@ixsystems.com> <292eadc6-3662-ec43-1175-53fc252487bd@quip.cz> Message-ID: <8d664f5eb336c955475c97332bcd2b2b@vvelox.net> X-Sender: v.velox@vvelox.net User-Agent: Roundcube Webmail/1.3.3 X-Rspamd-Queue-Id: 0AF2D893B7 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of v.velox@vvelox.net designates 96.95.67.25 as permitted sender) smtp.mailfrom=v.velox@vvelox.net X-Spamd-Result: default: False [-1.52 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:96.95.67.25/32]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[vvelox.net]; NEURAL_SPAM_SHORT(0.14)[0.139,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mail.vvelox.net]; RCPT_COUNT_SEVEN(0.00)[10]; IP_SCORE(-0.95)[ip: (-0.90), ipnet: 96.64.0.0/11(-3.92), asn: 7922(0.14), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7922, ipnet:96.64.0.0/11, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUSPICIOUS_RECIPS(1.50)[]; RCVD_COUNT_TWO(0.00)[2] X-Mailman-Approved-At: Thu, 02 May 2019 11:46:24 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 11:09:10 -0000 On 2019-04-30 17:03, Miroslav Lachman wrote: > David Chisnall wrote on 2019/04/30 10:22: >> On 29/04/2019 21:12, Joe Maloney wrote: >>> With CFT version you chose to build, and package individual >>> components such as sendmail with a port option.  That does entirely >>> solve the problem of being able to reinstall sendmail after the fact >>> without a rebuild of the userland (base) port but perhaps base >>> flavors could solve that problem assuming flavors could extend beyond >>> python. >> >> This sounds very much like local optimisation. It's now easy to create >> a custom base image.  Great.  But how do I express dependencies in >> ports on a specific base configuration? This is easy if I depend on a >> specific base package, but how does this work in your model?  For >> example, if I have a package that depends on a library that is an >> optional part of the base system, how do I express that pkg needs to >> either refuse to install it, or install a userland pkg that includes >> that library in place of my existing version as part of the install >> process? >> >> More importantly for the container use case, if I want to take a >> completely empty jail and do pkg ins nginx (for example), what does >> the maintainer of the nginx port need to do to express the minimum set >> of the base system that needs to be installed to allow nginx to work? >> >> One of the goals for the pkg base concept was to allow this kind of >> use case, easily creating a minimal environment required to run a >> single service. With a monolithic base package set, you're going to >> need some mechanism other than packages to express the specific base >> subset package that you need and I think that you need to justify why >> this mechanism is better than using small individual packages. > > Will it not be maintainer's nightmare to take care of all the > dependencies on the base packages for each port we have in the ports > tree? Speaking as a ports maintainer, it will be very annoying. Splitting it into a handful of large ass packages, same as you are presented with during install, would be best. From owner-freebsd-stable@freebsd.org Thu May 2 15:36:13 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4B481598B29 for ; Thu, 2 May 2019 15:36:12 +0000 (UTC) (envelope-from njm@njm.me.uk) Received: from smtp002.apm-internet.net (smtp002.apm-internet.net [85.119.248.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 21D809387B for ; Thu, 2 May 2019 15:36:10 +0000 (UTC) (envelope-from njm@njm.me.uk) Received: (qmail 34266 invoked from network); 2 May 2019 15:29:27 -0000 X-APM-Authkey: 18389/1(18389/1) 2 Received: from unknown (HELO meld.njm.me.uk) (90.195.126.124) by smtp002.apm-internet.net with SMTP; 2 May 2019 15:29:27 -0000 Received: from triton.njm.me.uk (triton.njm.me.uk [192.168.144.133]) by meld.njm.me.uk (8.15.2/8.15.2) with ESMTP id x42FTPKh099652; Thu, 2 May 2019 16:29:26 +0100 (BST) (envelope-from njm@njm.me.uk) Received: from localhost (localhost [127.0.0.1]) by triton.njm.me.uk (8.15.2/8.15.2) with ESMTP id x42FTPZA060530; Thu, 2 May 2019 16:29:25 +0100 (BST) (envelope-from njm@njm.me.uk) Date: Thu, 02 May 2019 16:29:25 +0100 From: "N.J. Mann" To: Michelle Sullivan cc: freebsd-stable Subject: Re: ZFS... Message-ID: <22E6AED197D46F831645E296@triton.njm.me.uk> In-Reply-To: <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> X-Mailer: Mulberry/4.0.8 (Linux/x86) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Rspamd-Queue-Id: 21D809387B X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [1.42 / 15.00]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.28)[0.284,0]; NEURAL_HAM_LONG(-0.55)[-0.554,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[njm.me.uk]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.47)[0.471,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[b.mx.fluent.ltd.uk,b.mx.fluent.ltd.uk,b.mx.fluent.ltd.uk,b.mx.fluent.ltd.uk,b.mx.fluent.ltd.uk]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; R_SPF_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[221.248.119.85.list.dnswl.org : 127.0.3.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:35259, ipnet:85.119.248.0/21, country:GB]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-0.02)[country: GB(-0.09)]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 15:36:13 -0000 Hi, On Thursday, May 02, 2019 09:27:36 +1000 Michelle Sullivan wrote: >> On 02 May 2019, at 02:16, Chris wrote: >>=20 >> Your story is so unusual I am wondering if its not fiction, I mean all >> sorts of power cuts where it just so happens the UPS fails every time, >=20 > The only =E2=80=9Cfiction=E2=80=9D is the date.. was the 10th not the 19th March... https://www.southcoastregister.com.au/story/5945663/homes-left-without-power-after-electrical-pole-destroyed-in-sanctuary-point-accident/ >=20 > UPSes do glitch out sometimes, but rarely.. they do have problems when 11kv comes down a 240v line though... I am sorry to hear about your loss of data, but where does the 11kV come from? I can understand 415V, i.e. two phases in contact, but the type of overhead lines in the pictures you reference are three phase each typically 240V to neutral and 415V between two phases. Best wishes, Nick. --=20 From owner-freebsd-stable@freebsd.org Thu May 2 16:24:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E768159A2A8 for ; Thu, 2 May 2019 16:24:16 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id CC7DC9599A for ; Thu, 2 May 2019 16:24:14 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-transfer-encoding: 8BIT Content-type: text/plain; charset=UTF-8; format=flowed Received: from isux.com (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQV00JFFY7LXW00@hades.sorbs.net> for freebsd-stable@freebsd.org; Thu, 02 May 2019 09:38:11 -0700 (PDT) Subject: Re: ZFS... To: "N.J. Mann" Cc: freebsd-stable References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> <22E6AED197D46F831645E296@triton.njm.me.uk> From: Michelle Sullivan Message-id: <51f1813c-5666-33a4-2155-59ba706a1948@sorbs.net> Date: Fri, 03 May 2019 02:24:10 +1000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 In-reply-to: <22E6AED197D46F831645E296@triton.njm.me.uk> X-Rspamd-Queue-Id: CC7DC9599A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.63 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.979,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.48)[ip: (-1.23), ipnet: 72.12.192.0/19(-0.62), asn: 11114(-0.49), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.91)[-0.910,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; CTE_CASE(0.50)[]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 16:24:16 -0000 N.J. Mann wrote: > Hi, > > > On Thursday, May 02, 2019 09:27:36 +1000 Michelle Sullivan wrote: >>> On 02 May 2019, at 02:16, Chris wrote: >>> >>> Your story is so unusual I am wondering if its not fiction, I mean all >>> sorts of power cuts where it just so happens the UPS fails every time, >> The only “fiction” is the date.. was the 10th not the 19th March... https://www.southcoastregister.com.au/story/5945663/homes-left-without-power-after-electrical-pole-destroyed-in-sanctuary-point-accident/ >> >> UPSes do glitch out sometimes, but rarely.. they do have problems when 11kv comes down a 240v line though... > I am sorry to hear about your loss of data, but where does the 11kV come from? > I can understand 415V, i.e. two phases in contact, but the type of overhead > lines in the pictures you reference are three phase each typically 240V to > neutral and 415V between two phases. > Bottom lines on the power pole are normal 240/415 .. top lines are the 11KV distribution network. > Best wishes, > Nick. -- Michelle Sullivan http://www.mhix.org/ From owner-freebsd-stable@freebsd.org Thu May 2 17:00:10 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66F79159B157 for ; Thu, 2 May 2019 17:00:10 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 1D65C97073 for ; Thu, 2 May 2019 17:00:08 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQV00JK1ZVFXW00@hades.sorbs.net> for freebsd-stable@freebsd.org; Thu, 02 May 2019 10:14:06 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <51f1813c-5666-33a4-2155-59ba706a1948@sorbs.net> Date: Fri, 03 May 2019 03:00:05 +1000 Cc: freebsd-stable Content-transfer-encoding: quoted-printable Message-id: <0D0C3E6C-3ED3-4629-BD26-B4D23ABC3800@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> <22E6AED197D46F831645E296@triton.njm.me.uk> <51f1813c-5666-33a4-2155-59ba706a1948@sorbs.net> To: "N.J. Mann" X-Rspamd-Queue-Id: 1D65C97073 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.82 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.975,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.47)[ip: (-1.20), ipnet: 72.12.192.0/19(-0.61), asn: 11114(-0.47), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.62)[-0.618,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 17:00:10 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 03 May 2019, at 02:24, Michelle Sullivan wrote: >=20 > N.J. Mann wrote: >> Hi, >>=20 >>=20 >> On Thursday, May 02, 2019 09:27:36 +1000 Michelle Sullivan wrote: >>>> On 02 May 2019, at 02:16, Chris wrote: >>>>=20 >>>> Your story is so unusual I am wondering if its not fiction, I mean all >>>> sorts of power cuts where it just so happens the UPS fails every time, >>> The only =E2=80=9Cfiction=E2=80=9D is the date.. was the 10th not the 19= th March... https://www.southcoastregister.com.au/story/5945663/homes-left-w= ithout-power-after-electrical-pole-destroyed-in-sanctuary-point-accident/ >>>=20 >>> UPSes do glitch out sometimes, but rarely.. they do have problems when 1= 1kv comes down a 240v line though... >> I am sorry to hear about your loss of data, but where does the 11kV come f= rom? >> I can understand 415V, i.e. two phases in contact, but the type of overhe= ad >> lines in the pictures you reference are three phase each typically 240V t= o >> neutral and 415V between two phases. >>=20 > Bottom lines on the power pole are normal 240/415 .. top lines are the 11K= V distribution network. >=20 Oh and just so you know, it=E2=80=99s sorta impossible to get 415 down a 24= 0v connection, for that to happen you=E2=80=99d need to disconnect any retur= n (neutral) and then connect it to another phase... and as most connections= are TNE that means dropping the cables on the incoming connection and havin= g a really dodgy connection. =20 It is very unusual to get 11kv down a distribution phase and pretty much the= only time I have ever seen it is when a cable is cut (a 11kv line) and it f= alls onto a supply cable... lightning strikes are the most common case. The= second most common (and they are really not) is someone taking out a pole. The system is designed to drop power out instantly when it happens which is w= hy it is rare to do damage but we (sparkles) all know that it=E2=80=99s mill= iseconds not =E2=80=9Cinstantly=E2=80=9D. Unfortunate part of this is if th= ey hit the power pole next up the road.. there was no 11kv distribution cabl= es.. as you can=E2=80=99t see in the photos the pole on the other side of t= he road had a transformer (not the same one that blew the night before...) It was a classic example of Murphy=E2=80=99s law, (if something can go wrong= to make @ bad situation worse, it will).. if it wasn=E2=80=99t in the midd= le of a resilver it would not have had this issue. If a transformer hadn=E2= =80=99t blown on the network 6 hours before I wouldn=E2=80=99t have even bee= n connected to that substation. If the transformer that blew hadn=E2=80=99t= ZFS would have just don=E2=80=99t the rollback I did the night before... i= f I hadn=E2=80=99t gone to the server room and checked everything and restar= ted the resilver it wouldn=E2=80=99t have been doing anything to the drives.= If the UPSes hadn=E2=80=99t failed on the night it would have probably bee= n waiting for me to rollback the 5 seconds when the pole was taken out...etc= ...etc... So many... =E2=80=9Cif this didn=E2=80=99t happens it would have been ok...=E2= =80=9D >=20 >> Best wishes, >> Nick. >=20 >=20 > --=20 > Michelle Sullivan > http://www.mhix.org/ >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu May 2 17:18:21 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 35EFC159BBAC for ; Thu, 2 May 2019 17:18:21 +0000 (UTC) (envelope-from njm@njm.me.uk) Received: from smtp002.apm-internet.net (smtp002.apm-internet.net [85.119.248.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DFBAF97E4A for ; Thu, 2 May 2019 17:18:19 +0000 (UTC) (envelope-from njm@njm.me.uk) Received: (qmail 32397 invoked from network); 2 May 2019 17:18:17 -0000 X-APM-Authkey: 18389/1(18389/1) 5 Received: from unknown (HELO meld.njm.me.uk) (90.195.126.124) by smtp002.apm-internet.net with SMTP; 2 May 2019 17:18:17 -0000 Received: from triton.njm.me.uk (triton.njm.me.uk [192.168.144.133]) by meld.njm.me.uk (8.15.2/8.15.2) with ESMTP id x42HIGFe000933; Thu, 2 May 2019 18:18:17 +0100 (BST) (envelope-from njm@njm.me.uk) Received: from localhost (localhost [127.0.0.1]) by triton.njm.me.uk (8.15.2/8.15.2) with ESMTP id x42HIGiO088535; Thu, 2 May 2019 18:18:16 +0100 (BST) (envelope-from njm@njm.me.uk) Date: Thu, 02 May 2019 18:18:16 +0100 From: "N.J. Mann" To: Michelle Sullivan cc: freebsd-stable Subject: Re: ZFS... Message-ID: <5D9EAA497C84C50D0DAA5491@triton.njm.me.uk> In-Reply-To: <0D0C3E6C-3ED3-4629-BD26-B4D23ABC3800@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> <22E6AED197D46F831645E296@triton.njm.me.uk> <51f1813c-5666-33a4-2155-59ba706a1948@sorbs.net> <0D0C3E6C-3ED3-4629-BD26-B4D23ABC3800@sorbs.net> X-Mailer: Mulberry/4.0.8 (Linux/x86) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Rspamd-Queue-Id: DFBAF97E4A X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [2.85 / 15.00]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.83)[0.835,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[njm.me.uk]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.60)[0.600,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: b.mx.fluent.ltd.uk]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.20)[0.196,0]; SUBJ_ALL_CAPS(0.45)[6]; R_SPF_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[221.248.119.85.list.dnswl.org : 127.0.3.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:35259, ipnet:85.119.248.0/21, country:GB]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-0.02)[country: GB(-0.09)]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 17:18:21 -0000 Hi, On Friday, May 03, 2019 03:00:05 +1000 Michelle Sullivan wrote: >>> I am sorry to hear about your loss of data, but where does the 11kV come from? >>> I can understand 415V, i.e. two phases in contact, but the type of overhead >>> lines in the pictures you reference are three phase each typically 240V to >>> neutral and 415V between two phases. >>>=20 >> Bottom lines on the power pole are normal 240/415 .. top lines are the 11KV distribution network. >=20 > Oh and just so you know, it=E2=80=99s sorta impossible to get 415 down a 240v connection No it is not. As I said, if two phases come into contact you can have 415v between live and neutral. Best wishes, Nick. --=20 From owner-freebsd-stable@freebsd.org Thu May 2 17:44:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0646159CA00 for ; Thu, 2 May 2019 17:44:40 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 5E2DA6A7C5 for ; Thu, 2 May 2019 17:44:39 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQW00JO31XLXW00@hades.sorbs.net> for freebsd-stable@freebsd.org; Thu, 02 May 2019 10:58:36 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <5D9EAA497C84C50D0DAA5491@triton.njm.me.uk> Date: Fri, 03 May 2019 03:44:35 +1000 Cc: freebsd-stable Content-transfer-encoding: quoted-printable Message-id: <914C23D6-4C44-4640-9215-3EA3E779BC47@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> <22E6AED197D46F831645E296@triton.njm.me.uk> <51f1813c-5666-33a4-2155-59ba706a1948@sorbs.net> <0D0C3E6C-3ED3-4629-BD26-B4D23ABC3800@sorbs.net> <5D9EAA497C84C50D0DAA5491@triton.njm.me.uk> To: "N.J. Mann" X-Rspamd-Queue-Id: 5E2DA6A7C5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.97 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.978,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.46)[ip: (-1.16), ipnet: 72.12.192.0/19(-0.59), asn: 11114(-0.46), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.77)[-0.772,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 17:44:40 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 03 May 2019, at 03:18, N.J. Mann wrote: >=20 > Hi, >=20 >=20 > On Friday, May 03, 2019 03:00:05 +1000 Michelle Sullivan wrote: >>>> I am sorry to hear about your loss of data, but where does the 11kV com= e from? >>>> I can understand 415V, i.e. two phases in contact, but the type of over= head >>>> lines in the pictures you reference are three phase each typically 240V= to >>>> neutral and 415V between two phases. >>>>=20 >>> Bottom lines on the power pole are normal 240/415 .. top lines are the 1= 1KV distribution network. >>=20 >> Oh and just so you know, it=E2=80=99s sorta impossible to get 415 down a= 240v connection >=20 > No it is not. As I said, if two phases come into contact you can have 415= v between > live and neutral. >=20 >=20 You=E2=80=99re not an electrician then.. the connection point on my house h= as the earth connected to the return on the pole and that also connected to t= he ground stake (using 16mm copper). You=E2=80=99d have to cut that link be= fore dropping a phase on the return to get 415 past the distribution board..= . sorta impossible... cut the ground link first then it=E2=80=99s possible..= . but as every connection has the same, that=E2=80=99s a lot of ground links= to cut to make it happen... unless you drop the return on both sizes of you= r pole and your ground stake and then drop a phase on that floating terminal= ...=20 > Best wishes, > Nick. > --=20 >=20 >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu May 2 18:04:19 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A36F159D17F for ; Thu, 2 May 2019 18:04:19 +0000 (UTC) (envelope-from njm@njm.me.uk) Received: from smtp002.apm-internet.net (smtp002.apm-internet.net [85.119.248.221]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 33E6E6B79B for ; Thu, 2 May 2019 18:04:18 +0000 (UTC) (envelope-from njm@njm.me.uk) Received: (qmail 66196 invoked from network); 2 May 2019 18:04:16 -0000 X-APM-Authkey: 18389/1(18389/1) 7 Received: from unknown (HELO meld.njm.me.uk) (90.195.126.124) by smtp002.apm-internet.net with SMTP; 2 May 2019 18:04:16 -0000 Received: from triton.njm.me.uk (triton.njm.me.uk [192.168.144.133]) by meld.njm.me.uk (8.15.2/8.15.2) with ESMTP id x42I4EFg001450; Thu, 2 May 2019 19:04:14 +0100 (BST) (envelope-from njm@njm.me.uk) Received: from localhost (localhost [127.0.0.1]) by triton.njm.me.uk (8.15.2/8.15.2) with ESMTP id x42I4E66000458; Thu, 2 May 2019 19:04:14 +0100 (BST) (envelope-from njm@njm.me.uk) Date: Thu, 02 May 2019 19:04:14 +0100 From: "N.J. Mann" To: Michelle Sullivan cc: freebsd-stable Subject: Re: ZFS... Message-ID: In-Reply-To: <914C23D6-4C44-4640-9215-3EA3E779BC47@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> <22E6AED197D46F831645E296@triton.njm.me.uk> <51f1813c-5666-33a4-2155-59ba706a1948@sorbs.net> <0D0C3E6C-3ED3-4629-BD26-B4D23ABC3800@sorbs.net> <5D9EAA497C84C50D0DAA5491@triton.njm.me.uk> <914C23D6-4C44-4640-9215-3EA3E779BC47@sorbs.net> X-Mailer: Mulberry/4.0.8 (Linux/x86) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Rspamd-Queue-Id: 33E6E6B79B X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [3.00 / 15.00]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.75)[0.753,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[njm.me.uk]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.82)[0.818,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: b.mx.fluent.ltd.uk]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.21)[0.210,0]; SUBJ_ALL_CAPS(0.45)[6]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:35259, ipnet:85.119.248.0/21, country:GB]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-0.02)[country: GB(-0.09)]; RCVD_IN_DNSWL_LOW(-0.10)[221.248.119.85.list.dnswl.org : 127.0.3.1] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 18:04:19 -0000 Hi, I will ignore the insult and just say again "come into contact". Yes, I do know what I am talking about and have even seen it happen. Best wishes, Nick. -- From owner-freebsd-stable@freebsd.org Thu May 2 18:21:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B949B159D9A5 for ; Thu, 2 May 2019 18:21:28 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id C686E6C446 for ; Thu, 2 May 2019 18:21:27 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQW00JT43MXXW00@hades.sorbs.net> for freebsd-stable@freebsd.org; Thu, 02 May 2019 11:35:24 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Fri, 03 May 2019 04:21:23 +1000 Cc: freebsd-stable Content-transfer-encoding: quoted-printable Message-id: <179C011C-FBE6-4CAE-BB9E-463908EC4783@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <6D7D690B-31DA-4A86-BB34-64A977B91D4F@sorbs.net> <22E6AED197D46F831645E296@triton.njm.me.uk> <51f1813c-5666-33a4-2155-59ba706a1948@sorbs.net> <0D0C3E6C-3ED3-4629-BD26-B4D23ABC3800@sorbs.net> <5D9EAA497C84C50D0DAA5491@triton.njm.me.uk> <914C23D6-4C44-4640-9215-3EA3E779BC47@sorbs.net> To: "N.J. Mann" X-Rspamd-Queue-Id: C686E6C446 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-2.86 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.985,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.44)[ip: (-1.13), ipnet: 72.12.192.0/19(-0.58), asn: 11114(-0.45), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.67)[-0.668,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 18:21:28 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 03 May 2019, at 04:04, N.J. Mann wrote: >=20 > Hi, >=20 >=20 > I will ignore the insult and just say again "come into contact". Yes, > I do know what I am talking about and have even seen it happen. >=20 It wasn=E2=80=99t an insult (certainly not intended).. and I did say, =E2=80= =9Csorta impossible=E2=80=9D and there are a whole series of events that wou= ld have to be a precursor for it to happen.. all of which I=E2=80=99d consi= der highly unlikely (less than 0.0001% chance) where they happened and were n= ot corrected before an event happened... even if you were to actually drop t= he phase on a floating terminal and take out the probability of that happeni= ng... Ie you=E2=80=99d have to cut/drop three safety connections, have them g= o unnoticed and then drop cables in two directions on a pole before you drop= you phase on the connection...=20 I should say.. bit of a reach... but then having a transformer blow up, and s= omeone take out a power pole and have 11kv line drop on a 240v phase within 6= hours whilst resilvering a drive would be a reach as well... so point conce= ded, it could happen. >=20 > Best wishes, > Nick. > --=20 >=20 From owner-freebsd-stable@freebsd.org Thu May 2 19:49:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E773159FBE6 for ; Thu, 2 May 2019 19:49:54 +0000 (UTC) (envelope-from lucy.wilson@infodigimedia.com) Received: from n1nlsmtp03.shr.prod.ams1.secureserver.net (n1nlsmtp03.shr.prod.ams1.secureserver.net [188.121.43.193]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay-hosting.secureserver.net", Issuer "Starfield Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 720FF6F8AF for ; Thu, 2 May 2019 19:49:43 +0000 (UTC) (envelope-from lucy.wilson@infodigimedia.com) Received: from n3plcpnl0152.prod.ams3.secureserver.net ([160.153.155.4]) by : HOSTING RELAY : with ESMTP id MHh9htXHPRzlGMHh9hegjy; Thu, 02 May 2019 12:48:39 -0700 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infodigimedia.com; s=default; h=Content-Type:MIME-Version:Message-ID:Date: Subject:To:From:Sender:Reply-To:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=QiqZSdbfrIr32npur7OfkEY9Gvi1bK44ybCevDOOI7k=; b=RjAlkJkGVAvUMiAI2m1Psa+ZO2 PIJNvX4XVoRr56JCsGWVu2aCbaO4m7gqvK93FVJMHzfbeW9nIhMjnCpNW3CZt95hiVa0HFl3CJj01 36bYl9FnSmXZXNtWluDpNAbzkc1rPKufAvSPALZKPJvt5xoP1U5SygWm+Imsh8KQJLYPwpIngBwop QA2OGQDHM0SVGN2hDQzmUcaEWUAk0g2r9rxo4GisYF99ky2Zpaw2Zqe08kNYBd7YyikwbASeQtgBg L3TQlB2rQlt6f9jESRXtB5S9AzzJQoCacQGGbzBl6HxTU8jJk8FtYFZkMDGT0A7BTvpZWoL38W0Jh rB8al0iA==; Received: from [122.166.166.3] (port=49638 helo=CyberdomPC) by n3plcpnl0152.prod.ams3.secureserver.net with esmtpsa (TLSv1:ECDHE-RSA-AES256-SHA:256) (Exim 4.91) (envelope-from ) id 1hMH9w-004RQz-7k for freebsd-stable@freebsd.org; Thu, 02 May 2019 12:14:20 -0700 From: "Lucy Wilson" To: Subject: F5 Networks Users | Please Advise Date: Thu, 2 May 2019 15:13:59 -0400 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdUBGwVCXir47y/wQAqlkafLo3YvWg== Content-Language: en-us X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - n3plcpnl0152.prod.ams3.secureserver.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - infodigimedia.com X-Get-Message-Sender-Via: n3plcpnl0152.prod.ams3.secureserver.net: authenticated_id: lucy.wilson@infodigimedia.com X-Authenticated-Sender: n3plcpnl0152.prod.ams3.secureserver.net: lucy.wilson@infodigimedia.com X-Source: X-Source-Args: X-Source-Dir: X-CMAE-Envelope: MS4wfBJBYA5G2Q62fEypDgxXFuYSOzeQDIp/0r7whhD/xiKjvZ8vcMqGxvLEr00dJ9C794/cFz2NPsCVkbp2SI40lEB7hcnUtpLFEf1RSvrEfFivNueeKORF fvnmL5tsS9uhBY3psCyaC4zruiijAePslHd98JhSQqf+1VLUUd+qjQIrvo57lj9T93UJFTjJM2BQWqKg1GLrMSyIKKj2uHfEE92G9Qj4QGuHgRzIkIrI2FoT X-Rspamd-Queue-Id: 720FF6F8AF X-Spamd-Bar: ++++++ Authentication-Results: mx1.freebsd.org; dkim=none (invalid DKIM record) header.d=infodigimedia.com header.s=default header.b=RjAlkJkG; spf=pass (mx1.freebsd.org: domain of lucy.wilson@infodigimedia.com designates 188.121.43.193 as permitted sender) smtp.mailfrom=lucy.wilson@infodigimedia.com X-Spamd-Result: default: False [6.76 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MX_INVALID(0.50)[greylisted]; R_SPF_ALLOW(0.00)[+ip4:188.121.43.0/24]; HAS_X_SOURCE(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[infodigimedia.com:~]; HAS_X_ANTIABUSE(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[3.166.166.122.zen.spamhaus.org : 127.0.0.10]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:26496, ipnet:188.121.40.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[]; HAS_X_AS(0.00)[lucy.wilson@infodigimedia.com]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_XBL(3.00)[3.166.166.122.zen.spamhaus.org : 127.0.0.4]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.91)[0.908,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[infodigimedia.com]; NEURAL_SPAM_MEDIUM(0.99)[0.994,0]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; IP_SCORE(0.36)[ip: (1.15), ipnet: 188.121.40.0/22(0.47), asn: 26496(0.22), country: US(-0.06)]; NEURAL_SPAM_LONG(1.00)[1.000,0]; RCVD_IN_DNSWL_NONE(0.00)[193.43.121.188.list.dnswl.org : 127.0.5.0]; R_DKIM_PERMFAIL(0.00)[infodigimedia.com:s=default]; HAS_X_GMSV(0.00)[lucy.wilson@infodigimedia.com]; RCVD_TLS_ALL(0.00)[] X-Spam: Yes Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 19:49:54 -0000 Hello, I just went through your LinkedIn page and found out your company and I was curious to know if you are interested in acquiring F5 Networks Users Contac List with email, phone and complete details? Titles like - IT Decision Makers, CEO, CFO, IT Engineers and many more. Information fields: Names, Title, Email, Phone, Company Name, Company URL, Company physical address, SIC Code, Industry and Specialty (Revenue and Employee). Let me know your thoughts on this, so that I can get back with more information on your interest. Best Regards, Lucy Wilson Demand Generation From owner-freebsd-stable@freebsd.org Thu May 2 19:59:40 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 884EB15A0006 for ; Thu, 2 May 2019 19:59:40 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from slim.berklix.org (slim.berklix.org [94.185.90.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "slim.berklix.org", Issuer "slim.berklix.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0203B6FFCD for ; Thu, 2 May 2019 19:59:38 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from mart.js.berklix.net (p2E52CADA.dip0.t-ipconnect.de [46.82.202.218]) (authenticated bits=128) by slim.berklix.org (8.15.2/8.15.2) with ESMTPSA id x42IHrVb042035 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 2 May 2019 20:17:57 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by mart.js.berklix.net (8.14.3/8.14.3) with ESMTP id x42IHreW046747; Thu, 2 May 2019 20:17:53 +0200 (CEST) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.7/8.14.7) with ESMTP id x42IHG7U052982; Thu, 2 May 2019 20:17:34 +0200 (CEST) (envelope-from jhs@berklix.com) Message-Id: <201905021817.x42IHG7U052982@fire.js.berklix.net> To: "N.J. Mann" cc: Michelle Sullivan , freebsd-stable Subject: Re: ZFS... From: "Julian H. Stacey" Organization: http://berklix.com/jhs http://stolenvotes.uk User-agent: EXMH on FreeBSD http://berklix.com/free/ X-From: http://www.berklix.org/~jhs/ In-reply-to: Your message "Thu, 02 May 2019 18:18:16 +0100." <5D9EAA497C84C50D0DAA5491@triton.njm.me.uk> Date: Thu, 02 May 2019 20:17:16 +0200 X-Rspamd-Queue-Id: 0203B6FFCD X-Spamd-Bar: +++++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [7.35 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: slim.berklix.com]; SUBJ_ALL_CAPS(0.45)[6]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:33824, ipnet:94.185.88.0/22, country:DE]; RCVD_TLS_LAST(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[218.202.82.46.zen.spamhaus.org : 127.0.0.10]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.70)[0.696,0]; R_BAD_CTE_7BIT(3.50)[7bit]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[berklix.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.97)[0.966,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.93)[0.933,0]; RCVD_IN_DNSWL_NONE(0.00)[68.90.185.94.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; GREYLIST(0.00)[pass,body]; IP_SCORE(-0.09)[asn: 33824(-0.43), country: DE(-0.01)] X-Spam: Yes X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 19:59:40 -0000 Hi, Reference: > From: "N.J. Mann" > Date: Thu, 02 May 2019 18:18:16 +0100 "N.J. Mann" wrote: > Hi, > > > On Friday, May 03, 2019 03:00:05 +1000 Michelle Sullivan wrote: > >>> I am sorry to hear about your loss of data, but where does the 11kV come from? > >>> I can understand 415V, i.e. two phases in contact, but the type of overhead > >>> lines in the pictures you reference are three phase each typically 240V to > >>> neutral and 415V between two phases. > >>> > >> Bottom lines on the power pole are normal 240/415 .. top lines are the 11KV distribution network. > > > > Oh and just so you know, it’s sorta impossible to get 415 down a 240v connection > > No it is not. As I said, if two phases come into contact you can have 415v between > live and neutral. With 3 phase: - 415 between each of the 3 live phases - 240 between neutral & the 3 live phases (To visualise: draw a circle of radius 12 cm, mark off dots on circle at 120 degrees. measure between dots: 20.75cm, Neutral is compass hole in paper ) PS UK/Britain was nominal 240V, but EU inc UK is now nominal 230V +10%/-6% https://en.wikipedia.org/wiki/Mains_electricity#Standardisation Cheers, Julian -- Julian Stacey, Consultant Systems Engineer, BSD Linux Unix, Munich Aachen Kent http://stolenvotes.uk Brexit ref. stole votes from 700,000 Brits in EU. Lies bought; Campaigns fined; 1.9 M young had no vote, 1.3 M old leavers dead. From owner-freebsd-stable@freebsd.org Thu May 2 20:16:29 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB74C15A054F for ; Thu, 2 May 2019 20:16:28 +0000 (UTC) (envelope-from matpockuh@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4C68670900 for ; Thu, 2 May 2019 20:16:28 +0000 (UTC) (envelope-from matpockuh@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 0928815A054E; Thu, 2 May 2019 20:16:28 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7A5615A054D for ; Thu, 2 May 2019 20:16:27 +0000 (UTC) (envelope-from matpockuh@gmail.com) Received: from mail-ot1-x332.google.com (mail-ot1-x332.google.com [IPv6:2607:f8b0:4864:20::332]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C3AF0708FF for ; Thu, 2 May 2019 20:16:26 +0000 (UTC) (envelope-from matpockuh@gmail.com) Received: by mail-ot1-x332.google.com with SMTP id o39so3311269ota.6 for ; Thu, 02 May 2019 13:16:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=mDXMRu2Lbtj+JPx7xSho71Ak55vV7lJsFJWZwYjNGWM=; b=pjdaicCbXyE3tcPWYxmmjs1vvgzjXWpWcQKNgBDHv5T5YTYGp6ZSZY2ASOcsTrAdLu aW0MI/hpS9SbDoFAuWEW3wofkrNRAl/QsB5J3wUHss/Zr/JGU0l2qAQc+J9WWMp3A2lZ gKMNfTFlvNnLUiXLYA6dIXE2g7I9GWJUx8+pj7EC/sHbTLiPEY9F2qcSzIMuB/MNSmS5 nycznSO5zs0rdwrdCQPRvCk+UaVmhZqrFEz4L3h6MOSoSB9PhAWJuNQs38fX05UgjLxi 4zy2XmC/OIOf5tVTrIdmj/mt/m2hYZAUX8Pj0vP5k8UOKRDcgYCMETP+wTyD9aipQX27 splw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=mDXMRu2Lbtj+JPx7xSho71Ak55vV7lJsFJWZwYjNGWM=; b=bN/xD+MTHbKDtwOZL1kPFvF/npiIQDLLxa9gXS/FRQYGQzQ2Va43sNktB8684zHwcH znvkUTN8c6h/VahEcCJ/gpY61dDKkUY3BIkvMSk9ggdD7eNwaSE4Y5gncoEjqIxTSFOE 7OWb6jk0N7rTFt+fZNA0+Si6ozS0FLW4OISEAE4zf3wAKLAzw/bzjbH/t0TvFul5zRWh N9hIB/tEPRM+5wlGsBFuvv1bxUu0vk4qAKMCVOHm9gjoP+OCSXFMtLNCgs3cwn6gr1Rl A4HIaDfLtVUCAI6xAj/ekal+fBtTqReoXuYkO0HW+0eEkzV3vHSQM2bveGn77w8FF0yk nfEA== X-Gm-Message-State: APjAAAWQWNZKQe5yNwlSvHkvXMQVK3kCrnxTtZqwmwQATZYbiT+rzsZI y155fXIaqFtyXqyHwGClTsHQhGudfKSWzRQxzV5Pp8i2 X-Google-Smtp-Source: APXvYqzmNRjUDOPwIf8JS9ocQq+dwm60VZY72Y8coKKPpKq4mXFGtCOgnRkGXGPwlVo4yxW/8xdCpLe/dS4cce4SIy8= X-Received: by 2002:a05:6830:1251:: with SMTP id s17mr199709otp.186.1556828185401; Thu, 02 May 2019 13:16:25 -0700 (PDT) MIME-Version: 1.0 From: KOT MATPOCKuH Date: Thu, 2 May 2019 23:16:38 +0300 Message-ID: Subject: route based ipsec To: stable@freebsd.org X-Rspamd-Queue-Id: C3AF0708FF X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=pjdaicCb; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of matpockuh@gmail.com designates 2607:f8b0:4864:20::332 as permitted sender) smtp.mailfrom=matpockuh@gmail.com X-Spamd-Result: default: False [-5.69 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.69)[-0.691,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.99)[ip: (-9.46), ipnet: 2607:f8b0::/32(-3.20), asn: 15169(-2.25), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[2.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; HTTP_TO_IP(1.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 20:16:29 -0000 Hello! I'm trying to make a full mesh vpn using route based ipsec between four hosts under FreeBSD 12. I'm used racoon from security/ipsec-tools (as it recommended in https://www.freebsd.org/doc/handbook/ipsec.html) Result looks work, but I got some problems: 0.The ipsec-tools port currently does not have a maintainer (C) portmaster ... Does this solution really supported? Or I should switch to use another IKE daemon? 1. racoon was 3 times crashed with core dump (2 times on one host, 1 times on another host): (gdb) bt #0 0x000000000024417f in isakmp_info_recv () #1 0x00000000002345f4 in isakmp_main () #2 0x00000000002307d0 in isakmp_handler () #3 0x000000000022f10d in session () #4 0x000000000022e62a in main () 2. racoon generated 2 SA for each traffic direction (from hostA to hostB). IMHO one SA for one each traffic direction should be enough. 3. ping and TCP taffic works over ipsec tunnels, but, for example, bird can't establish OSPF neighborhood over some (!) ipsec tunnels. I'm tried to watch traffic on ipsec tunnels and got some strange behavior. For example, ping hostA from hostD: > ping -c 2 192.168.31.9 PING 192.168.31.9 (192.168.31.9): 56 data bytes 64 bytes from 192.168.31.9: icmp_seq=0 ttl=64 time=1.334 ms 64 bytes from 192.168.31.9: icmp_seq=1 ttl=64 time=1.280 ms tcpdump on this hostD: # tcpdump -pni ipsec2 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ipsec2, link-type NULL (BSD loopback), capture size 262144 bytes 23:08:53.362318 IP 192.168.31.10 > 192.168.31.9: ICMP echo request, id 29396, seq 0, length 64 23:08:53.363604 IP 192.168.31.9 > 192.168.31.10: ICMP echo reply, id 29396, seq 0, length 64 23:08:54.384518 IP 192.168.31.10 > 192.168.31.9: ICMP echo request, id 29396, seq 1, length 64 23:08:54.385731 IP 192.168.31.9 > 192.168.31.10: ICMP echo reply, id 29396, seq On second side: # tcpdump -pni ipsec2 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ipsec2, link-type NULL (BSD loopback), capture size 262144 bytes 23:08:53.362196 IP 192.168.31.9 > 192.168.31.10: ICMP echo reply, id 29396, seq 0, length 64 23:08:54.384441 IP 192.168.31.9 > 192.168.31.10: ICMP echo reply, id 29396, seq 1, length 64 I think it's may be result of two SA's for each direction, and some traffic can be passed to kernel using second SA, but can't be associated with proper ipsecX interface. What You can recommend to solve this problems? PS. Not using IPSec on FreeBSD i as known, but wrong answer :) -- MATPOCKuH From owner-freebsd-stable@freebsd.org Thu May 2 21:17:49 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B67AA15A1894 for ; Thu, 2 May 2019 21:17:49 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 84888724D8 for ; Thu, 2 May 2019 21:17:38 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from [194.32.164.30] ([194.32.164.30]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id x42LHUKE066216; Thu, 2 May 2019 22:17:30 +0100 (BST) (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: ZFS... From: Bob Bishop In-Reply-To: <201905021817.x42IHG7U052982@fire.js.berklix.net> Date: Thu, 2 May 2019 22:17:30 +0100 Cc: "N.J. Mann" , freebsd-stable , Michelle Sullivan Content-Transfer-Encoding: quoted-printable Message-Id: References: <201905021817.x42IHG7U052982@fire.js.berklix.net> To: "Julian H. Stacey" X-Mailer: Apple Mail (2.3273) X-Rspamd-Queue-Id: 84888724D8 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of rb@gid.co.uk designates 194.32.164.250 as permitted sender) smtp.mailfrom=rb@gid.co.uk X-Spamd-Result: default: False [-2.84 / 15.00]; MX_INVALID(0.50)[greylisted]; R_SPF_ALLOW(-0.20)[+mx]; MV_CASE(0.50)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.92)[-0.917,0]; SUBJ_ALL_CAPS(0.45)[6]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42831, ipnet:194.32.164.0/24, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.988,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[gid.co.uk]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[250.164.32.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-1.09)[ip: (-3.09), ipnet: 194.32.164.0/24(-1.55), asn: 42831(-0.72), country: GB(-0.09)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 21:17:49 -0000 Hi, > On 2 May 2019, at 19:17, Julian H. Stacey wrote: >=20 > Hi, Reference: >> From: "N.J. Mann" >> Date: Thu, 02 May 2019 18:18:16 +0100 >=20 > "N.J. Mann" wrote: >> Hi, >>=20 >>=20 >> On Friday, May 03, 2019 03:00:05 +1000 Michelle Sullivan = wrote: >>>>> I am sorry to hear about your loss of data, but where does the = 11kV come from? >>>>> I can understand 415V, i.e. two phases in contact, but the type of = overhead >>>>> lines in the pictures you reference are three phase each typically = 240V to >>>>> neutral and 415V between two phases. >>>>>=20 >>>> Bottom lines on the power pole are normal 240/415 .. top lines are = the 11KV distribution network. >>>=20 >>> Oh and just so you know, it=E2=80=99s sorta impossible to get 415 = down a 240v connection >>=20 >> No it is not. As I said, if two phases come into contact you can = have 415v between >> live and neutral. >=20 > With 3 phase: > - 415 between each of the 3 live phases > - 240 between neutral & the 3 live phases ... and that=E2=80=99s RMS, so 586 and 325 peak respectively. > (To visualise: draw a circle of radius 12 cm, mark off dots on circle = at > 120 degrees. measure between dots: 20.75cm, Neutral is compass hole in = paper ) >=20 > PS UK/Britain was nominal 240V, but EU inc UK is now nominal 230V = +10%/-6% > https://en.wikipedia.org/wiki/Mains_electricity#Standardisation >=20 > Cheers, > Julian > --=20 > Julian Stacey, Consultant Systems Engineer, BSD Linux Unix, Munich = Aachen Kent > http://stolenvotes.uk Brexit ref. stole votes from 700,000 Brits in = EU. > Lies bought; Campaigns fined; 1.9 M young had no vote, 1.3 M old = leavers dead. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" -- Bob Bishop rb@gid.co.uk From owner-freebsd-stable@freebsd.org Thu May 2 22:48:03 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C783A15A326D for ; Thu, 2 May 2019 22:48:02 +0000 (UTC) (envelope-from rblayzor.bulk@inoc.net) Received: from cp1.c0.inoc.net (cp1.c0.inoc.net [64.246.141.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2F09474D8C for ; Thu, 2 May 2019 22:47:51 +0000 (UTC) (envelope-from rblayzor.bulk@inoc.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=inoc.net; s=default; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:MIME-Version :Date:Message-ID:From:References:To:Subject:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=AZX/jlLOehuQyYGtSIFDCp7AP05EJEn8xfc6vV2WHvY=; b=FIa7cW4kMbmLal/YwNCGwHCT1d RoaTUjN8SMow1gqT9DZldZNrUTfw38BwUMKoRDoWZhF15DfA2bIyc4551aUr2NR0FSvfRlUWgg6Mf DuFO9JLdJTnsyV94ELQcZ0Gr3EgVeQZOSxTl/qssoQ2TlTnhwnEHYgUr9oUX3siFr6slOobUehicp 3TrxAMdvh8vEy63VornSLihEZULYBb58k+FNh97BqMswgr2j4hTxXh4tCeKVmIG0VQUe3aXJG+Lg3 xKTrf9j9CVE/WPdfXAOuxj/aKxrUOlNrlR+t+Odvx496iNp5GeWrLahTPbbinwXXxnuvXSRbX71vO OYiUJqWg==; Received: from pool-71-164-79-202.albyny.east.verizon.net ([71.164.79.202]:63896 helo=Void.local) by cp1.c0.inoc.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.91) (envelope-from ) id 1hMKUR-0008NW-Se for freebsd-stable@freebsd.org; Thu, 02 May 2019 22:47:44 +0000 Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> From: Robert Blayzor Message-ID: <82613c6a-d013-68c3-ed91-1bfb030868f5@inoc.net> Date: Thu, 2 May 2019 18:47:43 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - cp1.c0.inoc.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - inoc.net X-Get-Message-Sender-Via: cp1.c0.inoc.net: authenticated_id: rblayzor.bulk@inoc.net X-Authenticated-Sender: cp1.c0.inoc.net: rblayzor.bulk@inoc.net X-Source: X-Source-Args: X-Source-Dir: X-Rspamd-Queue-Id: 2F09474D8C X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=inoc.net header.s=default header.b=FIa7cW4k; dmarc=pass (policy=none) header.from=inoc.net; spf=pass (mx1.freebsd.org: domain of rblayzor.bulk@inoc.net designates 64.246.141.1 as permitted sender) smtp.mailfrom=rblayzor.bulk@inoc.net X-Spamd-Result: default: False [-6.74 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MX_INVALID(0.50)[greylisted]; R_SPF_ALLOW(-0.20)[+mx]; HAS_X_SOURCE(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[inoc.net:+]; DMARC_POLICY_ALLOW(-0.50)[inoc.net,none]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.99)[-0.986,0]; HAS_X_ANTIABUSE(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[202.79.164.71.zen.spamhaus.org : 127.0.0.10]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:22302, ipnet:64.246.140.0/23, country:US]; IP_SCORE(-3.70)[ip: (-9.80), ipnet: 64.246.140.0/23(-4.90), asn: 22302(-3.74), country: US(-0.06)]; MID_RHS_MATCH_FROM(0.00)[]; HAS_X_AS(0.00)[rblayzor.bulk@inoc.net]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[inoc.net:s=default]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_ONE(0.00)[1]; HAS_X_GMSV(0.00)[rblayzor.bulk@inoc.net]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 May 2019 22:48:03 -0000 On 4/29/19 8:41 PM, Michelle Sullivan wrote: >> 2) Backups are essential with any filesystem, not just ZFS. After >> all, no amount of RAID will protect you from an accidental "rm -rf /". That's what snapshots are for! ;-) -- inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP: https://inoc.net/~rblayzor/ From owner-freebsd-stable@freebsd.org Fri May 3 07:09:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD5A21586CD9; Fri, 3 May 2019 07:09:42 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8316D8AFDB; Fri, 3 May 2019 07:09:41 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.250] (unknown [192.148.167.11]) by proxypop01.sare.net (Postfix) with ESMTPA id 48F6C9DD34F; Fri, 3 May 2019 09:09:31 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Borja Marcos In-Reply-To: Date: Fri, 3 May 2019 09:09:29 +0200 Cc: Xin LI , owner-freebsd-stable@freebsd.org, Andrea Venturoli , freebsd-stable , rainer@ultra-secure.de Content-Transfer-Encoding: quoted-printable Message-Id: <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> To: Michelle Sullivan X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 8316D8AFDB X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.44 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.16.151.0/24]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_FIVE(0.00)[6]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.sarenet.es,smtp.sarenet.es,smtp.sarenet.es]; DMARC_POLICY_ALLOW(-0.50)[sarenet.es,reject]; RCVD_IN_DNSWL_NONE(0.00)[151.151.16.195.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-2.65)[ip: (-7.33), ipnet: 195.16.128.0/19(-3.42), asn: 3262(-2.54), country: ES(0.04)]; NEURAL_HAM_SHORT(-0.93)[-0.928,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 07:09:43 -0000 > On 1 May 2019, at 04:26, Michelle Sullivan wrote: >=20 > mfid8 ONLINE 0 0 0 Anyway I think this is a mistake (mfid). I know, HBA makers have been = insisting on having their firmware getting in the middle, which is a bad thing. The right way to use disks is to give ZFS access to the plain CAM = devices, not thorugh some so-called JBOD on a RAID controller which, at least for a long time, has been a *logical* = =E2=80=9CRAID0=E2=80=9D volume on a single disk. That additional layer = can=20 completely break the semantics of transaction writes and cache flushes.=20= With some older cards it can be tricky to achieve, from patching source = drivers to enabling a sysctl tunable or even flashing the card to turn it into a plain HBA with no RAID features (or = minimal ones). If your drives are not called /dev/daX or /dev/adaX you are likely to be = in trouble. Unless something has really changed recently you don=E2=80=99t want =E2=80=9Cmfid=E2=80=9D or =E2=80=9Cmfisyspd=E2=80=9D= . I have suffered hidden data corruption due to a faulty HBA and failures = of old disks, and in all cases ZFS has survived brilianty. And actually ZFS works on somewhat unreliable hardware. The problem is = not non-perfect hardware, but *evil* hardware with=20 firmware based on some assumptions that won=E2=80=99t work with ZFS.=20 But I agree, non-ECC memory can be a problem. In my case all of the = servers had ECC. Borja.= From owner-freebsd-stable@freebsd.org Fri May 3 09:55:10 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E77F158AD00 for ; Fri, 3 May 2019 09:55:10 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE00069ECB for ; Fri, 3 May 2019 09:55:08 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [82.47.240.30] (helo=foula.local) by constantine.ingresso.co.uk with esmtpsa (TLSv1.3:TLS_AES_128_GCM_SHA256:128) (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hMUuC-0002at-RN for freebsd-stable@freebsd.org; Fri, 03 May 2019 09:55:00 +0000 Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> From: Pete French Message-ID: Date: Fri, 3 May 2019 10:55:00 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:67.0) Gecko/20100101 Thunderbird/67.0 MIME-Version: 1.0 In-Reply-To: <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: CE00069ECB X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-2.61 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.93)[-0.934,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:31.24.6.74]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.26)[asn: 16082(-1.21), country: GB(-0.09)]; MX_GOOD(-0.01)[us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mimecast.com, us-smtp-inbound-1.mimecast.com, us-smtp-inbound-2.mi mecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com,us-smtp-inbound-1.mimecast.com,us-smtp-inbound-2.mimecast.com]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.06)[-0.061,0]; RECEIVED_SPAMHAUS_PBL(0.00)[30.240.47.82.zen.spamhaus.org : 127.0.0.11]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 09:55:10 -0000 On 03/05/2019 08:09, Borja Marcos via freebsd-stable wrote: > The right way to use disks is to give ZFS access to the plain CAM devices, not thorugh some so-called JBOD on a RAID > controller which, at least for a long time, has been a *logical* “RAID0” volume on a single disk. That additional layer can > completely break the semantics of transaction writes and cache flushes. > > With some older cards it can be tricky to achieve, from patching source drivers to enabling a sysctl tunable or even > flashing the card to turn it into a plain HBA with no RAID features (or minimal ones). Oddly enough I got bitten by something like this yesteray. I have a machine containing an HP P400 RAID controller, which is nice enough, but I run ZFS so I have made the drives all into RAID-0 as being as close as I can get to accessing the raw SAS drives. BSD seems them as da0, da1, da2, da3 - but the RAID controller oly presents one of them to the BIOS, so my booting has to be all from that drive. This has been da0 for as long as I can remember, but yesteday it decided to start using what BSD sees as da1. Of course this is very hard to recognise as da0 and da1 are pretty much mirrors of each other. Spent a long time trying to work out why the fixes I was applying to da0 were not being used at boot time. ( Having to use Windows XP to talk to the iLo due to browsers dropping support for the old ciphers doesnt help either, what do other people with oldish HP hardware do about this ? I know its off topic, but theres a lot of it out there... ) -pete. From owner-freebsd-stable@freebsd.org Fri May 3 10:09:14 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D214A158B0D2 for ; Fri, 3 May 2019 10:09:13 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 81E2F6A59B for ; Fri, 3 May 2019 10:09:12 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.250] (unknown [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPA id 686249DC8E0; Fri, 3 May 2019 12:09:01 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Borja Marcos In-Reply-To: Date: Fri, 3 May 2019 12:09:00 +0200 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> To: Pete French X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 81E2F6A59B X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.15 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.16.148.0/24]; MV_CASE(0.50)[]; MX_GOOD(-0.01)[cached: smtp.sarenet.es]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[sarenet.es,reject]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.90)[-0.901,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[151.148.16.195.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.39)[ip: (-6.15), ipnet: 195.16.128.0/19(-3.35), asn: 3262(-2.48), country: ES(0.04)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 10:09:14 -0000 > On 3 May 2019, at 11:55, Pete French = wrote: >=20 >=20 >=20 > On 03/05/2019 08:09, Borja Marcos via freebsd-stable wrote: >=20 >> The right way to use disks is to give ZFS access to the plain CAM = devices, not thorugh some so-called JBOD on a RAID >> controller which, at least for a long time, has been a *logical* = =E2=80=9CRAID0=E2=80=9D volume on a single disk. That additional layer = can >> completely break the semantics of transaction writes and cache = flushes. >> With some older cards it can be tricky to achieve, from patching = source drivers to enabling a sysctl tunable or even >> flashing the card to turn it into a plain HBA with no RAID features = (or minimal ones). >=20 > Oddly enough I got bitten by something like this yesteray. I have a = machine containing an HP P400 RAID controller, which is nice enough, but = I run ZFS so I have made the drives all into RAID-0 as being as close as = I can get to accessing the raw SAS drives. >=20 > BSD seems them as da0, da1, da2, da3 - but the RAID controller oly = presents one of them to the BIOS, so my booting has to be all from that = drive. This has been da0 for as long as I can remember, but yesteday it = decided to start using what BSD sees as da1. Of course this is very hard = to recognise as da0 and da1 are pretty much mirrors of each other. Spent = a long time trying to work out why the fixes I was applying to da0 were = not being used at boot time. Hmm What happens when you do a =E2=80=9Ccamcontrol devlist=E2=80=9D? Camcontrol tags da0 -v? How is the controller recognized by FreeBSD? For some of them it=E2=80=99s= possible to instruct the controller to present the physical devices to = CAM. Of course you need to be careful to avoid any logical volume configuration in that = case.=20 But I would only tinker with this at system installation time, making = such a change on a running system with valid data can be disastrous. For mfi recognized cards there is a tunable: = hw.mfi.allow_cam_disk_passthrough For aac cards it was a matter of commenting a couple of source code = lines in the driver (at your own risk of course). I=E2=80=99ve been = running a=20 server for years doing that. Borja. From owner-freebsd-stable@freebsd.org Fri May 3 10:25:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05317158B78C for ; Fri, 3 May 2019 10:25:24 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 2601C6ADE3 for ; Fri, 3 May 2019 10:25:18 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQX00GIFC96YO10@hades.sorbs.net> for freebsd-stable@freebsd.org; Fri, 03 May 2019 03:39:09 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: Date: Fri, 03 May 2019 20:25:08 +1000 Cc: Pete French , freebsd-stable@freebsd.org Content-transfer-encoding: quoted-printable Message-id: <62803130-9C40-4A98-B5A4-A2DFAC0FAD65@sorbs.net> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> To: Borja Marcos X-Rspamd-Queue-Id: 2601C6ADE3 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.02 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.979,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[battlestar.sorbs.net,anaconda.sorbs.net,ninja.sorbs.net,catapilla.sorbs.net,scorpion.sorbs.net,desperado.sorbs.net]; NEURAL_HAM_SHORT(-0.84)[-0.844,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.43)[ip: (-1.10), ipnet: 72.12.192.0/19(-0.56), asn: 11114(-0.44), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 10:25:24 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 03 May 2019, at 20:09, Borja Marcos via freebsd-stable wrote: >=20 >=20 >=20 >> On 3 May 2019, at 11:55, Pete French wrote: >>=20 >>=20 >>=20 >>> On 03/05/2019 08:09, Borja Marcos via freebsd-stable wrote: >>>=20 >>> The right way to use disks is to give ZFS access to the plain CAM device= s, not thorugh some so-called JBOD on a RAID >>> controller which, at least for a long time, has been a *logical* =E2=80=9C= RAID0=E2=80=9D volume on a single disk. That additional layer can >>> completely break the semantics of transaction writes and cache flushes. >>> With some older cards it can be tricky to achieve, from patching source d= rivers to enabling a sysctl tunable or even >>> flashing the card to turn it into a plain HBA with no RAID features (or m= inimal ones). >>=20 >> Oddly enough I got bitten by something like this yesteray. I have a machi= ne containing an HP P400 RAID controller, which is nice enough, but I run ZFS= so I have made the drives all into RAID-0 as being as close as I can get to= accessing the raw SAS drives. I got bitten by that on this hardware originally... switching to raid-0 and= separate drives then switching to write-through (not write back and definit= ely not write back with bad bbu) seemed to solve it. Michelle >>=20 >> BSD seems them as da0, da1, da2, da3 - but the RAID controller oly presen= ts one of them to the BIOS, so my booting has to be all from that drive. Thi= s has been da0 for as long as I can remember, but yesteday it decided to sta= rt using what BSD sees as da1. Of course this is very hard to recognise as d= a0 and da1 are pretty much mirrors of each other. Spent a long time trying t= o work out why the fixes I was applying to da0 were not being used at boot t= ime. >=20 > Hmm What happens when you do a =E2=80=9Ccamcontrol devlist=E2=80=9D? >=20 > Camcontrol tags da0 -v? >=20 > How is the controller recognized by FreeBSD? For some of them it=E2=80=99s= possible to instruct the controller to present the physical devices to CAM.= Of course > you need to be careful to avoid any logical volume configuration in that c= ase.=20 >=20 > But I would only tinker with this at system installation time, making such= a change on a running system with valid data can be disastrous. >=20 > For mfi recognized cards there is a tunable: hw.mfi.allow_cam_disk_passthr= ough >=20 > For aac cards it was a matter of commenting a couple of source code lines i= n the driver (at your own risk of course). I=E2=80=99ve been running a=20 > server for years doing that. >=20 >=20 >=20 >=20 >=20 >=20 > Borja. >=20 >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Fri May 3 11:51:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B51E6158D590 for ; Fri, 3 May 2019 11:51:43 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A9656DAB6 for ; Fri, 3 May 2019 11:51:42 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [82.47.240.30] (helo=foula.local) by constantine.ingresso.co.uk with esmtpsa (TLSv1.3:TLS_AES_128_GCM_SHA256:128) (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hMWj6-0004X5-68 for freebsd-stable@freebsd.org; Fri, 03 May 2019 11:51:40 +0000 Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> From: Pete French Message-ID: <37e71963-31d3-5d9d-02b8-88f50195afc4@ingresso.co.uk> Date: Fri, 3 May 2019 12:51:40 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:67.0) Gecko/20100101 Thunderbird/67.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 7A9656DAB6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-2.55 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.95)[-0.953,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:31.24.6.74]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.30)[asn: 16082(-1.40), country: GB(-0.09)]; NEURAL_SPAM_SHORT(0.06)[0.056,0]; MX_GOOD(-0.01)[cached: us-smtp-inbound-1.mimecast.com]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; SUBJ_ALL_CAPS(0.45)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[30.240.47.82.zen.spamhaus.org : 127.0.0.11]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 11:51:43 -0000 > Hmm What happens when you do a “camcontrol devlist”? root@toybox:/usr/local/etc # camcontrol devlist at scbus0 target 0 lun 0 (pass0,da0) at scbus0 target 1 lun 0 (pass1,da1) at scbus0 target 2 lun 0 (pass2,da2) at scbus0 target 3 lun 0 (pass3,da3) at scbus2 target 0 lun 0 (pass4,cd0) > Camcontrol tags da0 -v? root@toybox:/usr/local/etc # camcontrol tags da0 -v (pass0:ciss0:0:0:0): dev_openings 255 (pass0:ciss0:0:0:0): dev_active 0 (pass0:ciss0:0:0:0): allocated 0 (pass0:ciss0:0:0:0): queued 0 (pass0:ciss0:0:0:0): held 0 (pass0:ciss0:0:0:0): mintags 2 (pass0:ciss0:0:0:0): maxtags 255 > How is the controller recognized by FreeBSD? For some of them it’s possible to instruct the controller to present the physical devices to CAM. Of course > you need to be careful to avoid any logical volume configuration in that case. I have been using these a long time, back to the old ones which were not ciss and ditn sit under CAM. Drives which arent configured dont show up at all, so I think the above is the best I can do. > But I would only tinker with this at system installation time, making such a change on a running system with valid data can be disastrous. Heh, yes ;-) Did you learn that the hard way ? I did! -pete. From owner-freebsd-stable@freebsd.org Fri May 3 12:11:50 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9CEFA158E8F5 for ; Fri, 3 May 2019 12:11:50 +0000 (UTC) (envelope-from SRS0=orHC=TD=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1B4116E915 for ; Fri, 3 May 2019 12:11:50 +0000 (UTC) (envelope-from SRS0=orHC=TD=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 5A53628416; Fri, 3 May 2019 14:11:39 +0200 (CEST) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 6601828411; Fri, 3 May 2019 14:11:38 +0200 (CEST) Subject: Re: ZFS... To: Pete French , freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <8a53df38-a094-b14f-9b7d-8def8ce42491@quip.cz> Date: Fri, 3 May 2019 14:11:39 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 1B4116E915 X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [5.64 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.99)[0.992,0]; IP_SCORE(1.01)[ip: (0.72), ipnet: 94.124.104.0/21(0.36), asn: 42000(3.87), country: CZ(0.08)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(1.00)[1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: elsa.codelab.cz]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_SPAM_LONG(1.00)[1.000,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=orHC=TD=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=orHC=TD=quip.cz=000.fbsd@elsa.codelab.cz]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 12:11:50 -0000 Pete French wrote on 2019/05/03 11:55: > Oddly enough I got bitten by something like this yesteray. I have a > machine containing an HP P400 RAID controller, which is nice enough, but > I run ZFS so I have made the drives all into RAID-0 as being as close as > I can get to accessing the raw SAS drives. > > BSD seems them as da0, da1, da2, da3 - but the RAID controller oly > presents one of them to the BIOS, so my booting has to be all from that > drive. This has been da0 for as long as I can remember, but yesteday it > decided to start using what BSD sees as da1. Of course this is very hard > to recognise as da0 and da1 are pretty much mirrors of each other. Spent > a long time trying to work out why the fixes I was applying to da0 were > not being used at boot time. I had this problem in the past too. I am not sure if it was on Dell or HP machine - controller presents first disk only in the boot time so I created small (10 - 15GB partition) on each disk and use them all in 4 way mirror. Cannot say if it was gmirror with UFS or ZFS mirroring. The rest of the each disk was used for ZFS RAIDZ. > ( Having to use Windows XP to talk to the iLo due to browsers dropping > support for the old ciphers doesnt help either, what do other people > with oldish HP hardware do about this ? I know its off topic, but theres > a lot of it out there... ) I have WinXP guest in VirtualBox exactly for this situation. With old browser, with old Java, with Flash - believe it or not, Cisco UCS C200 has remote management created in Flash! Miroslav Lachman From owner-freebsd-stable@freebsd.org Fri May 3 12:28:44 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B6BD158EFD2 for ; Fri, 3 May 2019 12:28:44 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5F4C86F1BC for ; Fri, 3 May 2019 12:28:43 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [2001:470:6cc4:1:225:ff:fe46:71cf] (helo=foula.local) by constantine.ingresso.co.uk with esmtpsa (TLSv1.3:TLS_AES_128_GCM_SHA256:128) (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hMXIv-00058A-PG for freebsd-stable@freebsd.org; Fri, 03 May 2019 12:28:41 +0000 Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> <8a53df38-a094-b14f-9b7d-8def8ce42491@quip.cz> From: Pete French Message-ID: <30d1cf2a-b80c-822b-11f9-2139532f7858@ingresso.co.uk> Date: Fri, 3 May 2019 13:28:42 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:67.0) Gecko/20100101 Thunderbird/67.0 MIME-Version: 1.0 In-Reply-To: <8a53df38-a094-b14f-9b7d-8def8ce42491@quip.cz> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 5F4C86F1BC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-2.66 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.962,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:31.24.6.74]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.33)[asn: 16082(-1.57), country: GB(-0.09)]; MX_GOOD(-0.01)[cached: us-smtp-inbound-1.mimecast.com]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.01)[-0.011,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 12:28:44 -0000 On 03/05/2019 13:11, Miroslav Lachman wrote: > I had this problem in the past too. I am not sure if it was on Dell or > HP machine - controller presents first disk only in the boot time so I > created small (10 - 15GB partition) on each disk and use them all in 4 > way mirror. Cannot say if it was gmirror with UFS or ZFS mirroring. The > rest of the each disk was used for ZFS RAIDZ. Snap :-) Thats exactly what I have done - but the bits you cant mirror are the GPT partititons for bootcode. I was fiddling with hose of da0 without realising it was now using da1 to boot. If it ever chooses da2 or da3 then I will need to mirror it there too, so I have it partitioned like that, but am currently using those as swap as it shows no signs of wanting to boot from them for now. > I have WinXP guest in VirtualBox exactly for this situation. With old > browser, with old Java, with Flash - believe it or not, Cisco UCS C200 > has remote management created in Flash! Heh, yes, I can believe it. I too have XP in VirtualBox. Maybe thats just the best solution to this after all. cheers, -pete. From owner-freebsd-stable@freebsd.org Fri May 3 13:08:00 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C27DA1590121 for ; Fri, 3 May 2019 13:07:59 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id 1CC8B70C35 for ; Fri, 3 May 2019 13:07:57 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQX00G6ZJSEYO20@hades.sorbs.net> for freebsd-stable@freebsd.org; Fri, 03 May 2019 06:21:53 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <20190503125118.GA11226@neutralgood.org> Date: Fri, 03 May 2019 23:07:51 +1000 Cc: freebsd-stable@freebsd.org, Borja Marcos , Pete French Content-transfer-encoding: quoted-printable Message-id: <2A7B5457-371A-4014-8C1E-972BA2FD10DF@sorbs.net> References: <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> <62803130-9C40-4A98-B5A4-A2DFAC0FAD65@sorbs.net> <20190503125118.GA11226@neutralgood.org> To: "Kevin P. Neal" X-Rspamd-Queue-Id: 1CC8B70C35 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.07 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.977,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; NEURAL_HAM_SHORT(-0.91)[-0.908,0]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.42)[ip: (-1.07), ipnet: 72.12.192.0/19(-0.55), asn: 11114(-0.43), country: US(-0.06)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 13:08:00 -0000 Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 03 May 2019, at 22:51, Kevin P. Neal wrote: >=20 > On Fri, May 03, 2019 at 08:25:08PM +1000, Michelle Sullivan wrote: >>> On 03 May 2019, at 20:09, Borja Marcos via freebsd-stable wrote: >>>=20 >>>=20 >>>=20 >>>> On 3 May 2019, at 11:55, Pete French wrote:= >>>>=20 >>>>=20 >>>>=20 >>>>> On 03/05/2019 08:09, Borja Marcos via freebsd-stable wrote: >>>>>=20 >>>>> The right way to use disks is to give ZFS access to the plain CAM devi= ces, not thorugh some so-called JBOD on a RAID >>>>> controller which, at least for a long time, has been a *logical* =E2=80= =9CRAID0=E2=80=9D volume on a single disk. That additional layer can >>>>> completely break the semantics of transaction writes and cache flushes= . >>>>> With some older cards it can be tricky to achieve, from patching sourc= e drivers to enabling a sysctl tunable or even >>>>> flashing the card to turn it into a plain HBA with no RAID features (o= r minimal ones). >>>>=20 >>>> Oddly enough I got bitten by something like this yesteray. I have a mac= hine containing an HP P400 RAID controller, which is nice enough, but I run Z= FS so I have made the drives all into RAID-0 as being as close as I can get t= o accessing the raw SAS drives. >>=20 >> I got bitten by that on this hardware originally... switching to raid-0 a= nd separate drives then switching to write-through (not write back and defin= itely not write back with bad bbu) seemed to solve it. >=20 > I have an old Dell R610 with a PERC 6/i and Megaraid SAS driver Ver 4.23. > When I use mfiutil to set caching to write through it still goes through > the cache. Which means that if a drive fails and the machine reboots the > firmware stops the boot because it has data in the cache that it wants > to store on the failed drive. So a normal failure of a drive in a three > way ZFS mirror that shouldn't cause a loss of service actually does. >=20 I don=E2=80=99t think it will do that in write through.. it will everytime i= n write back. > Thumbs down to RAID cards. > --=20 > Kevin P. Neal http://www.pobox.com/~kpn/ >=20 > "Good grief, I've just noticed I've typed in a rant. Sorry chaps!" > Keir Finlow Bates, circa 1998 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Fri May 3 13:30:13 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A237159072D for ; Fri, 3 May 2019 13:30:13 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0CE50715CE for ; Fri, 3 May 2019 13:30:11 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [82.47.240.30] (helo=foula.local) by constantine.ingresso.co.uk with esmtpsa (TLSv1.3:TLS_AES_128_GCM_SHA256:128) (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hMYGQ-00067h-1o for freebsd-stable@freebsd.org; Fri, 03 May 2019 13:30:10 +0000 Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> <62803130-9C40-4A98-B5A4-A2DFAC0FAD65@sorbs.net> <20190503125118.GA11226@neutralgood.org> <2A7B5457-371A-4014-8C1E-972BA2FD10DF@sorbs.net> From: Pete French Message-ID: <7b9ce013-e50c-7cfc-f5c1-c829855f8ee2@ingresso.co.uk> Date: Fri, 3 May 2019 14:30:09 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:67.0) Gecko/20100101 Thunderbird/67.0 MIME-Version: 1.0 In-Reply-To: <2A7B5457-371A-4014-8C1E-972BA2FD10DF@sorbs.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 0CE50715CE X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-3.18 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.97)[-0.970,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:31.24.6.74]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.36)[asn: 16082(-1.72), country: GB(-0.09)]; MX_GOOD(-0.01)[cached: us-smtp-inbound-1.mimecast.com]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.49)[-0.486,0]; RECEIVED_SPAMHAUS_PBL(0.00)[30.240.47.82.zen.spamhaus.org : 127.0.0.11]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 13:30:13 -0000 On 03/05/2019 14:07, Michelle Sullivan wrote: > I don’t think it will do that in write through.. it will everytime in write back. Yes, it really shouldnt do that. My server is so old that the battery on the RAID has failed, which definitely makes it go into write though mode. However not an ideal solution ;-) From owner-freebsd-stable@freebsd.org Fri May 3 20:57:15 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64497159A72F for ; Fri, 3 May 2019 20:57:15 +0000 (UTC) (envelope-from mike@sentex.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id ACDA688031 for ; Fri, 3 May 2019 20:57:14 +0000 (UTC) (envelope-from mike@sentex.net) Received: by mailman.ysv.freebsd.org (Postfix) id 6D1F8159A72E; Fri, 3 May 2019 20:57:14 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4AADB159A72D for ; Fri, 3 May 2019 20:57:14 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene.sentex.ca (unknown [IPv6:2607:f3e0:0:3::18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9F1BF8802D for ; Fri, 3 May 2019 20:57:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.26] ([192.168.43.26]) by pyroxene.sentex.ca (8.15.2/8.15.2) with ESMTP id x43Kv0aB032999; Fri, 3 May 2019 16:57:00 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: route based ipsec To: KOT MATPOCKuH , stable@freebsd.org References: From: Mike Tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= mQENBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAG0HG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT6JATgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnm5AQ0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAYkBHwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: Date: Fri, 3 May 2019 16:56:59 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 9F1BF8802D X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::18 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-1.49 / 15.00]; ARC_NA(0.00)[]; RDNS_NONE(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; NEURAL_HAM_LONG(-1.00)[-0.998,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[sentex.net]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.sentex.ca]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.96)[-0.961,0]; IP_SCORE(-1.73)[ipnet: 2607:f3e0::/32(-4.95), asn: 11647(-3.60), country: CA(-0.09)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 20:57:15 -0000 On 5/2/2019 4:16 PM, KOT MATPOCKuH wrote: > 0.The ipsec-tools port currently does not have a maintainer (C) portmaster > ... Does this solution really supported? Or I should switch to use another > IKE daemon? Take a look at StrongSwan in the ports for your IKE daemon and google around for config examples / discussions. The bad news-- The ipsec docs really need updating. The good news, StrongSwan and IPSEC in RELENG_11 and 12 are really great and well maintained.  Documentation is sadly not in one place.     ---Mike ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada From owner-freebsd-stable@freebsd.org Sat May 4 00:05:07 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA4B6159DE40 for ; Sat, 4 May 2019 00:05:07 +0000 (UTC) (envelope-from michelle@sorbs.net) Received: from hades.sorbs.net (hades.sorbs.net [72.12.213.40]) by mx1.freebsd.org (Postfix) with ESMTP id B4B948E0DD for ; Sat, 4 May 2019 00:05:06 +0000 (UTC) (envelope-from michelle@sorbs.net) MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Received: from [10.10.0.230] (gate.mhix.org [203.206.128.220]) by hades.sorbs.net (Oracle Communications Messaging Server 7.0.5.29.0 64bit (built Jul 9 2013)) with ESMTPSA id <0PQY00GSGE7QYO20@hades.sorbs.net> for freebsd-stable@freebsd.org; Fri, 03 May 2019 17:19:04 -0700 (PDT) Subject: Re: ZFS... From: Michelle Sullivan X-Mailer: iPad Mail (16A404) In-reply-to: <7b9ce013-e50c-7cfc-f5c1-c829855f8ee2@ingresso.co.uk> Date: Sat, 04 May 2019 10:05:02 +1000 Cc: freebsd-stable@freebsd.org Content-transfer-encoding: quoted-printable Message-id: <0D6CF718-2D40-4457-ADAB-CC17B52124AA@sorbs.net> References: <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> <62803130-9C40-4A98-B5A4-A2DFAC0FAD65@sorbs.net> <20190503125118.GA11226@neutralgood.org> <2A7B5457-371A-4014-8C1E-972BA2FD10DF@sorbs.net> <7b9ce013-e50c-7cfc-f5c1-c829855f8ee2@ingresso.co.uk> To: Pete French X-Rspamd-Queue-Id: B4B948E0DD X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of michelle@sorbs.net designates 72.12.213.40 as permitted sender) smtp.mailfrom=michelle@sorbs.net X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.963,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:hades.sorbs.net]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sorbs.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: battlestar.sorbs.net]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.213.12.72.list.dnswl.org : 127.0.10.0]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.41)[ip: (-1.05), ipnet: 72.12.192.0/19(-0.54), asn: 11114(-0.42), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.87)[-0.866,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11114, ipnet:72.12.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 00:05:08 -0000 New batteries are only $19 on eBay for most battery types... Michelle Sullivan http://www.mhix.org/ Sent from my iPad > On 03 May 2019, at 23:30, Pete French wrote: >=20 >=20 >=20 >> On 03/05/2019 14:07, Michelle Sullivan wrote: >> I don=E2=80=99t think it will do that in write through.. it will everytim= e in write back. >=20 > Yes, it really shouldnt do that. >=20 > My server is so old that the battery on the RAID has failed, which > definitely makes it go into write though mode. However not an ideal soluti= on ;-) > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sat May 4 09:51:35 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D39C61588892 for ; Sat, 4 May 2019 09:51:35 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (constantine.ingresso.co.uk [31.24.6.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B53F28226A for ; Sat, 4 May 2019 09:51:34 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from [82.47.240.30] (helo=foula.local) by constantine.ingresso.co.uk with esmtpsa (TLSv1.3:TLS_AES_128_GCM_SHA256:128) (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hMrKO-000PiT-AP for freebsd-stable@freebsd.org; Sat, 04 May 2019 09:51:32 +0000 Subject: Re: ZFS... To: freebsd-stable@freebsd.org References: <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> <62803130-9C40-4A98-B5A4-A2DFAC0FAD65@sorbs.net> <20190503125118.GA11226@neutralgood.org> <2A7B5457-371A-4014-8C1E-972BA2FD10DF@sorbs.net> <7b9ce013-e50c-7cfc-f5c1-c829855f8ee2@ingresso.co.uk> <0D6CF718-2D40-4457-ADAB-CC17B52124AA@sorbs.net> From: Pete French Message-ID: Date: Sat, 4 May 2019 10:51:34 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:67.0) Gecko/20100101 Thunderbird/67.0 MIME-Version: 1.0 In-Reply-To: <0D6CF718-2D40-4457-ADAB-CC17B52124AA@sorbs.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: B53F28226A X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=ingresso.co.uk; spf=pass (mx1.freebsd.org: domain of petefrench@ingresso.co.uk designates 31.24.6.74 as permitted sender) smtp.mailfrom=petefrench@ingresso.co.uk X-Spamd-Result: default: False [-2.44 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.96)[-0.957,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:31.24.6.74]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.39)[asn: 16082(-1.86), country: GB(-0.09)]; NEURAL_SPAM_SHORT(0.27)[0.267,0]; MX_GOOD(-0.01)[cached: us-smtp-inbound-1.mimecast.com]; DMARC_POLICY_ALLOW(-0.50)[ingresso.co.uk,none]; SUBJ_ALL_CAPS(0.45)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[30.240.47.82.zen.spamhaus.org : 127.0.0.11]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16082, ipnet:31.24.0.0/21, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 09:51:36 -0000 On 04/05/2019 01:05, Michelle Sullivan wrote: > New batteries are only $19 on eBay for most battery types... Indeed, my problem is actual physical access to the machine, which I havent seen in ten years :-) I even have a relacement server sitting behind my desk which we never quite got around to installing. I think the next move it makes will be to the cloud though, so am not too worried. From owner-freebsd-stable@freebsd.org Sat May 4 17:58:42 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B78361594760 for ; Sat, 4 May 2019 17:58:42 +0000 (UTC) (envelope-from freebsd-lists-5@thismonkey.com) Received: from mail-01.thismonkey.com (mail-01.thismonkey.com [220.244.217.216]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thismonkey.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 915296B2FE for ; Sat, 4 May 2019 17:58:35 +0000 (UTC) (envelope-from freebsd-lists-5@thismonkey.com) X-TM-Via-MX: mail-01.thismonkey.com Received: from utility-01.thismonkey.com ([IPv6:2600:70ff:c033:1:250:56ff:fe8a:4043]) by mail-01.thismonkey.com (8.15.2/8.15.2) with ESMTPS id x44HIRLI079843 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Sun, 5 May 2019 03:18:29 +1000 (EST) (envelope-from freebsd-lists-5@thismonkey.com) Received: from utility-01.thismonkey.com (localhost [127.0.0.1]) by utility-01.thismonkey.com (8.15.2/8.15.2) with ESMTP id x44HIQCn053279 for ; Sun, 5 May 2019 03:18:27 +1000 (EST) (envelope-from freebsd-lists-5@thismonkey.com) Received: (from root@localhost) by utility-01.thismonkey.com (8.15.2/8.15.2/Submit) id x44HINGV053125 for freebsd-stable@freebsd.org; Sun, 5 May 2019 03:18:23 +1000 (EST) (envelope-from freebsd-lists-5@thismonkey.com) Date: Sun, 5 May 2019 03:18:22 +1000 From: Scott Aitken To: freebsd-stable@freebsd.org Subject: Re: route based ipsec Message-ID: <20190504171822.GA27671@thismonkey.com> Mail-Followup-To: freebsd-stable@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail-01.thismonkey.com X-Virus-Scanned: clamav-milter 0.101.1 at mail-01.thismonkey.com X-Virus-Status: Clean X-Greylist: inspected by milter-greylist-4.6.2 (mail-01.thismonkey.com [IPv6:2600:70ff:c033:2:250:56ff:fe8a:2eed]); Sun, 05 May 2019 03:18:43 +1000 (EST) for IP:'2600:70ff:c033:1:250:56ff:fe8a:4043' DOMAIN:'[IPv6:2600:70ff:c033:1:250:56ff:fe8a:4043]' HELO:'utility-01.thismonkey.com' FROM:'freebsd-lists-5@thismonkey.com' RCPT:'' SPF:' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mail-01.thismonkey.com [IPv6:2600:70ff:c033:2:250:56ff:fe8a:2eed]); Sun, 05 May 2019 03:18:43 +1000 (EST) X-Rspamd-Queue-Id: 915296B2FE X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dmarc=pass (policy=none) header.from=thismonkey.com; spf=pass (mx1.freebsd.org: domain of freebsd-lists-5@thismonkey.com designates 220.244.217.216 as permitted sender) smtp.mailfrom=freebsd-lists-5@thismonkey.com X-Spamd-Result: default: False [-0.31 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_MEDIUM(0.29)[0.293,0]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.30)[-0.300,0]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; MX_GOOD(-0.01)[mail-01.thismonkey.com,server-03.thismonkey.com]; DMARC_POLICY_ALLOW(-0.50)[thismonkey.com,none]; NEURAL_HAM_SHORT(-0.20)[-0.202,0]; IP_SCORE(0.71)[asn: 7545(3.57), country: AU(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7545, ipnet:220.244.216.0/22, country:AU]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 17:58:42 -0000 > On 5/2/2019 4:16 PM, KOT MATPOCKuH wrote: > > 0.The ipsec-tools port currently does not have a maintainer (C) portmaster > > ... Does this solution really supported? Or I should switch to use > > another IKE daemon? I've just started using IPSEC between a 12.0-RELEASE box, a 11.2-RELEASE-p9 box and a Cisco IOS router. I haven't seen any core dumps or crashes. I run routing between these devices (using RIPv2 rather than OSPF) - in order to do this you need to create tunnels between the devices because encrypting routing protocols and things that use multicast is tricky. I felt that that the handbook example was lacking - it should have been encrypting the tunnel endpoints and NOT the LAN traffic on either side of the tunnel. Anyway I built IPENCAP (aka IPinIP) tunnels using gif interfaces and configured racoon/ipsec-tools to build the SA/SADs using the tunnel endpoints and IP protocol 4 (IPENCAP). Step 1 was to confirm I could PING over the gif tunnel without crytpo. Then I fired up racoon (setkey to create the SA and racoon for IPSEC). If you want the configs let me know. Scott From owner-freebsd-stable@freebsd.org Sat May 4 19:36:51 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4CC5C1596E1F for ; Sat, 4 May 2019 19:36:51 +0000 (UTC) (envelope-from chrcoluk@gmail.com) Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FF2C6E9F9 for ; Sat, 4 May 2019 19:36:49 +0000 (UTC) (envelope-from chrcoluk@gmail.com) Received: by mail-wr1-x42a.google.com with SMTP id h15so6320792wrb.2 for ; Sat, 04 May 2019 12:36:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=ejNX2+nt+JcASIHvUh+DLhsrfOE9ncoTv6FhHApVeKQ=; b=JnddHg6xZyDNkADw3nHMygkqTb8ATb3d+Ly+VpZatu9LM088vLvO6rXFrLkQNat+9S J1ky1IGbDFmXHPtzMQNE9CezH50yJb5qwuCXKIvLwu0SIpmsrZdWyVFmDwbz9bp8bhpT /AZp8QtTJbJXkPbVnWwupPPVZG27bmoigHk5q6bAPcniqnlUwguFOAv/8L2YxMLwHO7f 2G4nwhIJOyo3OOWSfpgucO3zq+GiuHHUikVJ3dJ/g42UalGrQlWdliwS2QcpRoXwXUej V22X2jC23IYrzRRDpiYBv5Tmreqt/l+ujQHUEVXgNBto2AwYgacVIkbgWRyz0eUJouc6 QI6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ejNX2+nt+JcASIHvUh+DLhsrfOE9ncoTv6FhHApVeKQ=; b=gPrrtIQlgQFXRp0Sn7hkdBXddzVXnEeb6bt5hzIbhgAblfef52+lHbSoDht16/jRom L4yMBAZdwjzdDYOUY1Jl19p6rCLg1RHFC5Fm05pHTC8TJB5eMNY2sEOYQbWDSAFFuaTK pDUDUh2y5i+4qmcumVECA+Jz6AjxvxGF8nnSBM0WDgnMjqKXW1medF/KnBToguDqG9nM uD4VkEWELDBUWLZQ3SPrRElwHkl36Ason0soaoPZ9TEDCfSzVXiB1IsexyHuatrNMlcb YqHv/yzwpdat5TUTCk54X/G0Fi2S5cwBGz3/h8BM0g+8hUlliEsGXLfjzlBkoFLjxOAB LwGw== X-Gm-Message-State: APjAAAUMxkzlatapmEYa8vLrP691NItM3ptj8W0LUYdzIQOxiXESnQkV ctajX3iAmqC6uxg4j4qxyLjfuvkk/ptFfzPAZ9Q= X-Google-Smtp-Source: APXvYqzmlqXBgppvnFONLqrEia2wUygr4zXW+nr0PcYRjFTu1K1zYC0GAdLOl9/1m9Iygox5SIvQlYHMN7g5uPZfqcs= X-Received: by 2002:a5d:6b48:: with SMTP id x8mr11276437wrw.266.1556998607207; Sat, 04 May 2019 12:36:47 -0700 (PDT) MIME-Version: 1.0 References: <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> <62803130-9C40-4A98-B5A4-A2DFAC0FAD65@sorbs.net> <20190503125118.GA11226@neutralgood.org> <2A7B5457-371A-4014-8C1E-972BA2FD10DF@sorbs.net> <7b9ce013-e50c-7cfc-f5c1-c829855f8ee2@ingresso.co.uk> <0D6CF718-2D40-4457-ADAB-CC17B52124AA@sorbs.net> In-Reply-To: From: Chris Date: Sat, 4 May 2019 19:36:36 +0000 Message-ID: Subject: Re: ZFS... To: Pete French Cc: FreeBSD Stable Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 8FF2C6E9F9 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=JnddHg6x; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of chrcoluk@gmail.com designates 2a00:1450:4864:20::42a as permitted sender) smtp.mailfrom=chrcoluk@gmail.com X-Spamd-Result: default: False [-6.23 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.84)[-0.840,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[a.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.83)[ip: (-9.59), ipnet: 2a00:1450::/32(-2.23), asn: 15169(-2.25), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 19:36:51 -0000 Sorry t clarify, Michelle I do believe your tail of events, just I meant that it reads like a tale as its so unusual. I also agree that there probably at this point of time should be more zfs tools written for the few situations that do happen when things get broken. Although I still standby my opinion I consider ZFS a huge amount more robust than UFS, UFS always felt like I only had to sneeze the wrong way and I would get issues. There was even one occasion simply installing the OS on its defaults, gave me corrupted data on UFS (9.0 release had nasty UFS journalling bug which corrupted data without any power cuts etc.). In future I suggest you use mirror if the data matters. I know it costs more in capacity for redundancy but in todays era of large drives its the only real sensible option. On the drive failures you have clearly been quite unlucky, and the other stuff is unusual. Best of luck On Sat, 4 May 2019 at 09:54, Pete French wrote: > > > > On 04/05/2019 01:05, Michelle Sullivan wrote: > > New batteries are only $19 on eBay for most battery types... > > Indeed, my problem is actual physical access to the machine, which I > havent seen in ten years :-) I even have a relacement server sitting > behind my desk which we never quite got around to installing. I think > the next move it makes will be to the cloud though, so am not too worried. > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sat May 4 19:51:35 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A91441597339 for ; Sat, 4 May 2019 19:51:35 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (ns1.bitblocks.com [173.228.5.8]) by mx1.freebsd.org (Postfix) with ESMTP id 736A16F320 for ; Sat, 4 May 2019 19:51:34 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from [192.168.125.6] (ipad.bitblocks.com [192.168.125.6]) by mail.bitblocks.com (Postfix) with ESMTP id A4740156E40C; Sat, 4 May 2019 12:51:12 -0700 (PDT) Mime-Version: 1.0 (1.0) Subject: Re: ZFS... From: Bakul Shah X-Mailer: iPad Mail (16E227) In-Reply-To: <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> Date: Sat, 4 May 2019 12:51:12 -0700 Cc: freebsd-stable Message-Id: <36C9B52C-D832-42B3-B4A8-566B89784982@bitblocks.com> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <5ED8BADE-7B2C-4B73-93BC-70739911C5E3@sorbs.net> To: Michelle Sullivan X-Rspamd-Queue-Id: 736A16F320 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of bakul@bitblocks.com designates 173.228.5.8 as permitted sender) smtp.mailfrom=bakul@bitblocks.com X-Spamd-Result: default: False [-1.86 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.89)[-0.890,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:173.228.5.8/29]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[bitblocks.com]; NEURAL_HAM_LONG(-0.98)[-0.977,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[mail.bitblocks.com]; RCPT_COUNT_TWO(0.00)[2]; SUBJ_ALL_CAPS(0.45)[6]; IP_SCORE(-0.15)[asn: 46375(-0.67), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.59)[-0.586,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:46375, ipnet:173.228.0.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 19:51:35 -0000 See https://github.com/alcarithemad/zfsp =E2=80=94 zfs in python. May be use this as a *starting point* to build an interactive zpool/zfs expl= ore tool? On Apr 30, 2019, at 3:14 AM, Michelle Sullivan wrote: >> Speaking for a tool like 'fsck': I think I'm mostly convinced that it's n= ot necessary, because at the point ZFS says the metadata is corrupted, it me= ans that these metadata was really corrupted beyond repair (all replicas wer= e corrupted; otherwise it would recover by finding out the right block and r= ewrite the bad ones). >=20 > I see this message all the time and mostly agree.. actually I do agree wit= h possibly a minor exception, but so minor it=E2=80=99s probably not worth i= t. However as I suggested in my original post.. the pool says the files are= there, a tool that would send them (aka zfs send) but ignoring errors to sp= acemaps etc would be real useful (to me.) >=20 >>=20 >> An interactive tool may be useful (e.g. "I saw data structure version 1, 2= , 3 available, and all with bad checksum, choose which one you would want to= try"), but I think they wouldn't be very practical for use with large data p= ools -- unlike traditional filesystems, ZFS uses copy-on-write and heavily d= epends on the metadata to find where the data is, and a regular "scan" is no= t really useful. >=20 > Zdb -AAA showed (shows) 36m files.. which suggests the data is intact, bu= t it aborts the mount with I/o error because it says metadata has three erro= rs.. 2 =E2=80=98metadata=E2=80=99 and one =E2=80=9C=E2=80=9D (s= torage being the pool name).. it does import, and it attempts to resilver bu= t reports the resilver finishes at some 780M (ish).. export import and it do= es it all again... zdb without -AAA aborts loading metaslab 122. From owner-freebsd-stable@freebsd.org Sat May 4 21:33:32 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FA59159985C for ; Sat, 4 May 2019 21:33:32 +0000 (UTC) (envelope-from matpockuh@gmail.com) Received: from mail-ot1-x333.google.com (mail-ot1-x333.google.com [IPv6:2607:f8b0:4864:20::333]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 44A3D7248C for ; Sat, 4 May 2019 21:33:31 +0000 (UTC) (envelope-from matpockuh@gmail.com) Received: by mail-ot1-x333.google.com with SMTP id a10so8320731otl.12 for ; Sat, 04 May 2019 14:33:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=G1kJXNjR2p3LFFIMRbelGF6F8zyvf6Ohj8iHedVAmbk=; b=eWPOHZ01Y0I7HJareI2ck4XsntFBCJNGU+XIITOcDoSm2sFoXFMC/vwI+8BjgcHtJd KY3oWhp8eQ9WW5WFExEOoqJvJDtArVN0xKpznXXbBKXOv53mYBOB2OQIzn1pTHjThGBX J7RSAw8hUs/0oArTD2mVZ3xiN5c193ZYRMGhvYw43J/8oNBEuQOievKN5fKOFne43TGU ClCxH/7eJrN821ykmapTKBbvVUwLmzOauKZJ/+kG+Zj6Q3p7idbEH0+1el2EgV45jms7 23xVcF7JgCyQBae3QKn16ZrIVi+cVtF+EfRCAR0lBUMCSHR7ZqJ4QZrdq0TD+2hHaiIh tw8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=G1kJXNjR2p3LFFIMRbelGF6F8zyvf6Ohj8iHedVAmbk=; b=CnS+c8LWrKASy6/fUTNbm+1vZQUdrF/ucu+4MIxmore+jQnSgP2SsLylkSn62MRQ/h pMkHY4GMggwYmnMTgHVudmCUtfhqmNfFXnepCbds0t3ThSdLahyfp8GZakc+D2z/PTwg jOE+hU21Dv/S/3B6Kh+y9/vM8OJaPgKJ+4SAXFyyYnB0BU4Cl4Y4BOENq13neqxLpGxj D+I7uzB+ADRvZpSolmFCeSWCORElTA6lzPiTObZ9P/3F5h06af2x08+JDswWVYeuQc5N Yt5K3xxPBr1poA29IdidsarTwB6jb6l5NTPvV1c6EHssrsQSKvWFhKOO4A4HGBqCFVJJ tVgQ== X-Gm-Message-State: APjAAAUp97JlCCc9CEWJMh2XJv1ZTJhukJ+SXOINZRh6hHjXtVxQ8APw 9vaxwT+tFqekps0cQSCVK+p/NaTG3UnJmeObSX3ZKDjD X-Google-Smtp-Source: APXvYqyhxMO1F5Cx/y5MS9tCY+kZjzH2oFoF2WvEYy4fxsFMzp8EwBvWZPGbiGB2vOvARKZZVfH4ewixOM7fEjxZCo4= X-Received: by 2002:a9d:73d9:: with SMTP id m25mr4624884otk.356.1557005610172; Sat, 04 May 2019 14:33:30 -0700 (PDT) MIME-Version: 1.0 References: <20190504171822.GA27671@thismonkey.com> In-Reply-To: <20190504171822.GA27671@thismonkey.com> From: KOT MATPOCKuH Date: Sun, 5 May 2019 00:33:42 +0300 Message-ID: Subject: Re: route based ipsec To: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 44A3D7248C X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=eWPOHZ01; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of matpockuh@gmail.com designates 2607:f8b0:4864:20::333 as permitted sender) smtp.mailfrom=matpockuh@gmail.com X-Spamd-Result: default: False [-6.87 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.3.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; IP_SCORE(-2.88)[ip: (-8.86), ipnet: 2607:f8b0::/32(-3.21), asn: 15169(-2.25), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 21:33:32 -0000 Hello! =D1=81=D0=B1, 4 =D0=BC=D0=B0=D1=8F 2019 =D0=B3. =D0=B2 21:01, Scott Aitken = : > > On 5/2/2019 4:16 PM, KOT MATPOCKuH wrote: > > > 0.The ipsec-tools port currently does not have a maintainer (C) > portmaster > > > ... Does this solution really supported? Or I should switch to use > > > another IKE daemon? > > I've just started using IPSEC between a 12.0-RELEASE box, a 11.2-RELEASE-= p9 > box and a Cisco IOS router. > What type of peers_identifier are You using? I'm using asn1dn... And today I got a coredump on 3rd host in: #0 0x000000000024717f in privsep_init () I haven't seen any core dumps or crashes. I run routing between these > devices (using RIPv2 rather than OSPF) - in order to do this you need to > create tunnels between the devices because encrypting routing protocols a= nd > things that use multicast is tricky. I felt that that the handbook examp= le > was lacking - it should have been encrypting the tunnel endpoints and NOT > the > LAN traffic on either side of the tunnel. > I used pointtomultipoint topology and hardcoded peer's IP addresses for OSPF. No multicast =3D> no problems :) > Anyway I built IPENCAP (aka IPinIP) tunnels using gif interfaces and > configured racoon/ipsec-tools to build the SA/SADs using the tunnel > endpoints > and IP protocol 4 (IPENCAP). > I think my next step will be try to use gre tunnels over ipsec with psk authentication. If you want the configs let me know. > No, thanks You! :) --=20 MATPOCKuH From owner-freebsd-stable@freebsd.org Sat May 4 22:14:43 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB5DB159A650 for ; Sat, 4 May 2019 22:14:43 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com [IPv6:2607:f8b0:4864:20::235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C1C967355D for ; Sat, 4 May 2019 22:14:42 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-oi1-x235.google.com with SMTP id y64so7026429oia.7 for ; Sat, 04 May 2019 15:14:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=adTMeQ1a7Wa6WaHMQ+iNAcdmjz1qcSKzKvBgc3FBA+E=; b=WX0D2Q/TP5BkNCuhoIxUBLW7S9WFYsxRpXfBx/FIdJlVpOTRFjUpzK4md08JrhGvtI ePD1hQdYgVu2neFdk6WLB1K7eLV7np6MmoOUvHtXdKE0TPtzppP+s8LQlUOofXoY+xhz s2zYjcArGHGuxxKsqzMuj7nvTUHVRqZuXQhgwLLC1ZyIxPCfdcnR8I3kULmGakSFW3Ko EXk/XmojosxSHE1Xu3fYdmVO3UIOfxksps5aPE9qQfMPDEsxIKqAz2LnagmcaBno00vW CD6k9frrighvZsSPcBuBNhVGrME7+2vRcvpjrOwQmV04FHkiWaSLPsADMt9d0TgSpyXl +aAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=adTMeQ1a7Wa6WaHMQ+iNAcdmjz1qcSKzKvBgc3FBA+E=; b=luJBgUtP72XvWArzHRIUKSt6Sx4bANoQeWMRuuT1p9i3IFmWYMWQO+JWjoFYAkBDiy 70rhYThMUHJdr8KIWCOmlwt2yzuVcC0BNq+fucjWshjzO0BCzCFxVK2nflrlEfly1U9C dptvO7DFn0VQIJ5isS2rmUAnO+4DhR31jwRlscgbIjy4DU9MT6YdF+FaD89T19HEblU+ ke9/ySS52xe+AZBxCIVkbPVrZrHCfhVuL2tP3ngT84iqQi3ylqxMxpR51pVc7waSvN+Q kXWNTVRa08Q9q123VREuyDkBFZ491wXTAGPCia6dd/15aE9ON0VfXVNTG2HmbNgQagG4 nKTw== X-Gm-Message-State: APjAAAW6knpzvC4oWjpbkUzi1iT39Q3RN5zHTZ/2RHe77X1+FBA0xBJX Yare5gJOBlVwTiPeZschSW2BUZ3RV5zz3ZUFaxo= X-Google-Smtp-Source: APXvYqxGhMcZFBc/j6AX27o3RFrvQVBvXMtttuCLFVyQc8ea+PlAyqt87p2GXSUUDLd6I6RZ7GjT94nk8Yj4QLTYQLc= X-Received: by 2002:aca:6c47:: with SMTP id h68mr3448318oic.57.1557008081605; Sat, 04 May 2019 15:14:41 -0700 (PDT) MIME-Version: 1.0 References: <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> <62803130-9C40-4A98-B5A4-A2DFAC0FAD65@sorbs.net> <20190503125118.GA11226@neutralgood.org> <2A7B5457-371A-4014-8C1E-972BA2FD10DF@sorbs.net> <7b9ce013-e50c-7cfc-f5c1-c829855f8ee2@ingresso.co.uk> <0D6CF718-2D40-4457-ADAB-CC17B52124AA@sorbs.net> In-Reply-To: From: Kevin Oberman Date: Sat, 4 May 2019 15:14:24 -0700 Message-ID: Subject: Re: ZFS... To: Chris Cc: Pete French , FreeBSD Stable X-Rspamd-Queue-Id: C1C967355D X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=WX0D2Q/T; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of kob6558@gmail.com designates 2607:f8b0:4864:20::235 as permitted sender) smtp.mailfrom=kob6558@gmail.com X-Spamd-Result: default: False [-6.19 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; SUBJ_ALL_CAPS(0.45)[6]; FORGED_SENDER(0.30)[rkoberman@gmail.com,kob6558@gmail.com]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[rkoberman@gmail.com,kob6558@gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.93)[-0.933,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[5.3.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-3.00)[ip: (-9.49), ipnet: 2607:f8b0::/32(-3.21), asn: 15169(-2.25), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 May 2019 22:14:43 -0000 On Sat, May 4, 2019 at 12:38 PM Chris wrote: > Sorry t clarify, Michelle I do believe your tail of events, just I > meant that it reads like a tale as its so unusual. > As the late Sir Terry Pratchett wrote, "Scientists have calculated that the chances of something so patently absurd actually existing are millions to one. But magicians have calculated that million-to-one chances crop up nine times out of ten.=E2=80=9D -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683