From owner-freebsd-hardware@freebsd.org Tue Aug 8 02:49:31 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06522DB4D99 for ; Tue, 8 Aug 2017 02:49:31 +0000 (UTC) (envelope-from noloader@gmail.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CB9586A989 for ; Tue, 8 Aug 2017 02:49:30 +0000 (UTC) (envelope-from noloader@gmail.com) Received: by mail-oi0-x22a.google.com with SMTP id g131so20898827oic.3 for ; Mon, 07 Aug 2017 19:49:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:from:date:message-id:subject:to; bh=BK3H3dsqTpBvSl10ljL/9cax+4T7fiQD2uwkmA0cHEI=; b=c7NnVh15qp6H5LgAlhrnM+djdVF+G0IljX/z1M+g8DB/Z0Kyv3kIBuMC9i9L1+GHVd tFfokeArY5ZwgZr/eMAviEXDLk6K03IQr6rR40J6vQY8toK3nH7sTz6D+ulR/40cqt/W hE8oAX7C2geF6jnW81BkCs8MVVPbhCYXgYQVUgfAgAhfXGxfv7JrGvP+A2+4WJwsSdJF luqH+HFhrpahfuYII1eru7PE2W/6keSakfs8yctzjMAXBxwsG2+ikf9IwYGzGB3xBMoI ZTi8Raz4hv28qdMFmOrh6c7sPfyTCeXStyJSqH0tAWeJVhMS6GrViPPkexLm3tGNa+Tv 175Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:from:date:message-id :subject:to; bh=BK3H3dsqTpBvSl10ljL/9cax+4T7fiQD2uwkmA0cHEI=; b=brjRF5naXvcx15dwRwG8w2oI7f3DMSfCTqahgBYEXoNSOnl4u8BbffM7KGe5zd/rrU GCADgIBW/PXTEPc2ERP0uRkqqoYJfaKPhF76160QtDxoZQluMirYYYad4ak/9mRZUUOZ Wg49NnaxElnWMYfzG/TUfeB8YdnGZvs3J5VtPXexrxy+flzTU+txJQoYCLKSVkleVDzJ bSlYm2Feq3u21qvxB9x7hzQAqv4liF9Yj/qDVEgslaPesD0dhHcF7WNHYxqvxHHxvTi1 7wvNHaqVigpOZwZPv9ux12ecE78n6aSW/0jij68IFrJlzINiXchtbHkr0oRsYZrJqox8 hgCg== X-Gm-Message-State: AHYfb5hNSnPn3oR88OaC2YQr70b/P5L7UyIbtnGrMKON47xjVAnPgbsm 0F0tuuriIsxbXIQgjT3w+gDAEa3EeA9yMcM= X-Received: by 10.202.182.10 with SMTP id g10mr2569913oif.173.1502160569692; Mon, 07 Aug 2017 19:49:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.74.98.28 with HTTP; Mon, 7 Aug 2017 19:49:29 -0700 (PDT) Reply-To: noloader@gmail.com From: Jeffrey Walton Date: Mon, 7 Aug 2017 22:49:29 -0400 Message-ID: Subject: Determine availability of AES, PMULL, SHA1 and SHA2 on 64-bit ARM? To: freebsd-hardware@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2017 02:49:31 -0000 Hi Everyone, On Linux I can check for the presence of optional cpu features AES, PMULL, SHA1 and SHA2 using getauxval(3). It works for both Aarch32 and Aarch64. Also see "When to use AT_HWCAP or AT_HWCAP2", https://sourceware.org/ml/libc-help/2017-08/msg00012.html. I'm not having any luck determining the sysctl's to use for the same under the BSDs. How do I determine availability of AES, PMULL, SHA1 and SHA2 on FreeBSD? If possible, I'd like to know how to look this stuff up on FreeBSD since I am getting poor search results. Thanks in advance. Jeff From owner-freebsd-hardware@freebsd.org Wed Aug 9 14:27:40 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 26C16DCE64E for ; Wed, 9 Aug 2017 14:27:40 +0000 (UTC) (envelope-from mi+m@aldan.algebra.com) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E7DF37454E for ; Wed, 9 Aug 2017 14:27:39 +0000 (UTC) (envelope-from mi+m@aldan.algebra.com) X_CMAE_Category: , , X-CNFS-Analysis: v=2.2 cv=Aa3zJDfG c=1 sm=1 tr=0 a=2ZyxD0Un1vyHw3hS6RhAvw==:117 a=2ZyxD0Un1vyHw3hS6RhAvw==:17 a=kj9zAlcOel0A:10 a=4mwfk3PxCzGL85_U2lIA:9 a=CjuIK1q_8ugA:10 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: YW5hdEByY24uY29t Authentication-Results: smtp02.rcn.cmh.synacor.com header.from=mi+m@aldan.algebra.com; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.mail=mi+m@aldan.algebra.com; spf=neutral; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.user=anat; auth=pass (PLAIN) Received-SPF: neutral (smtp02.rcn.cmh.synacor.com: 38.125.162.34 is neither permitted nor denied by domain of aldan.algebra.com) Received: from [38.125.162.34] ([38.125.162.34:37989] helo=[192.168.25.92]) by smtp.rcn.com (envelope-from ) (ecelerity 3.6.23.54417 r(Core:3.6.23.0)) with ESMTPSA (cipher=DHE-RSA-AES256-GCM-SHA384) id 03/59-53214-ADB1B895; Wed, 09 Aug 2017 10:27:38 -0400 From: "Mikhail T." Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Date: Wed, 9 Aug 2017 10:27:37 -0400 Subject: Do I need SAS drives?.. Message-Id: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> To: freebsd-hardware@freebsd.org, freebsd-scsi@freebsd.org X-Mailer: iPhone Mail (14G60) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 14:27:40 -0000 My server has 8 "hot-plug" slots, that can accept both SATA and SAS drives. S= ATA ones tend to be cheaper for the same features (like cache-sizes), what a= m I getting for the extra money spent on SAS? Asking specifically about the protocol differences... It would seem, for exa= mple, SATA can not be as easily hot-plugged, but with camcontrol(8) that sho= uld not be a problem, right? What else? Thank you! --=20 Sent from mobile device, please, pardon shorthand. --=20 Sent from mobile device, please, pardon shorthand.= From owner-freebsd-hardware@freebsd.org Wed Aug 9 15:12:34 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50139DCF41A for ; Wed, 9 Aug 2017 15:12:34 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from bs1.fjl.org.uk (bs1.fjl.org.uk [84.45.41.196]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "bs1.fjl.org.uk", Issuer "bs1.fjl.org.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D7D167590E for ; Wed, 9 Aug 2017 15:12:33 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from [10.153.42.245] ([185.69.144.102]) (authenticated bits=0) by bs1.fjl.org.uk (8.14.4/8.14.4) with ESMTP id v79Etdwh030377 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Wed, 9 Aug 2017 15:55:40 +0100 (BST) (envelope-from frank2@fjl.co.uk) User-Agent: K-9 Mail for Android In-Reply-To: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> MIME-Version: 1.0 Subject: Re: Do I need SAS drives?.. From: "Frank Leonhardt (m)" Date: Wed, 09 Aug 2017 15:55:27 +0100 To: "Mikhail T." , "freebsd-hardware@freebsd.org." Message-ID: <362B0950-A244-4C65-89C7-898EFC6A4A1F@fjl.co.uk> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 15:12:34 -0000 Simple answer is to use either. You're running FreeBSD with ZFS, right? BSD will hot plug anything. I suspect 'hot plug' relates to Microsoft workaround hardware RAID. Hot plug enclosures will also let the host know a drive has been pulled. Otherwise ZFS won't know whether it was pulled or is unresponsive due to it being on fire or something. With 8 drives in your array you can probably figure this out yourself. SAS drives use SCSI commands, which are supposedly better than SATA commands. Electrically they are the same. SAS drives are more expensive and tend to be higher spec mechanically, but not always so. Incidentally, nearline SAS is a cheaper SATA drive that understands SAS protocol and has dual ports. Marketing. Basically, if you really want speed at all costs go for SAS. If you want best capacity for your money, go SATA. If in doubt, go for SATA. If you don't know you need SAS for some reason, you probably don't. Regards, Frank. On 9 August 2017 15:27:37 BST, "Mikhail T." wrote: >My server has 8 "hot-plug" slots, that can accept both SATA and SAS >drives. SATA ones tend to be cheaper for the same features (like >cache-sizes), what am I getting for the extra money spent on SAS? > >Asking specifically about the protocol differences... It would seem, >for example, SATA can not be as easily hot-plugged, but with >camcontrol(8) that should not be a problem, right? What else? Thank >you! >-- >Sent from mobile device, please, pardon shorthand. > > >-- >Sent from mobile device, please, pardon shorthand. >_______________________________________________ >freebsd-hardware@freebsd.org mailing list >https://lists.freebsd.org/mailman/listinfo/freebsd-hardware >To unsubscribe, send any mail to >"freebsd-hardware-unsubscribe@freebsd.org" -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From owner-freebsd-hardware@freebsd.org Wed Aug 9 15:29:54 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B66ADCF7D8 for ; Wed, 9 Aug 2017 15:29:54 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 72DA076047 for ; Wed, 9 Aug 2017 15:29:53 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id D9BE821E1E for ; Wed, 9 Aug 2017 11:29:52 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute1.internal (MEProxy); Wed, 09 Aug 2017 11:29:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=XjIQZC a/Fx92kXw3EGonRahA1Gq54/d3LwvcauKBww0=; b=f68nxfIspqbkXdzoMeAgyt uaBXQqHRUcL2ndafiNpoZyBD/EORljHyQtMWC/Eu6RcSdh7eeYDvIcRBiVPVE2bR n3DNdf12T5G6fSquzakmZUKPzvcZAavRsxqgbAF6QsDjEy8h6rH2AE0hDqdgdYjq YW0FM0R9ufGanvlD5qwGJ24zkqBRV8GMnVDfFd+LUzF5TBhR35qMZVKKGGcZWLkY VKsy+iN948liRXOcp5uO6VWM+V1jbnbfEuNeyrUD0uxrENCD4W4giQlEMiddNKPA NGhmhfJYoBfl+mjLZXCTz6gB73Qi5TzJ4SGNCwxW6rZzTxJCz42EAGMeVBWXpAIw == X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id BA6D29E772; Wed, 9 Aug 2017 11:29:52 -0400 (EDT) Message-Id: <1502292592.2001426.1068163912.1191A246@webmail.messagingengine.com> From: Josh Paetzel To: freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-b56c1ff4 In-Reply-To: <362B0950-A244-4C65-89C7-898EFC6A4A1F@fjl.co.uk> Subject: Re: Do I need SAS drives?.. References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> <362B0950-A244-4C65-89C7-898EFC6A4A1F@fjl.co.uk> Date: Wed, 09 Aug 2017 10:29:52 -0500 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 15:29:54 -0000 On Wed, Aug 9, 2017, at 09:55 AM, Frank Leonhardt (m) wrote: > Simple answer is to use either. You're running FreeBSD with ZFS, right? > BSD will hot plug anything. I suspect 'hot plug' relates to Microsoft > workaround hardware RAID. > > Hot plug enclosures will also let the host know a drive has been pulled. > Otherwise ZFS won't know whether it was pulled or is unresponsive due to > it being on fire or something. With 8 drives in your array you can > probably figure this out yourself. > > SAS drives use SCSI commands, which are supposedly better than SATA > commands. Electrically they are the same. SAS drives are more expensive > and tend to be higher spec mechanically, but not always so. Incidentally, > nearline SAS is a cheaper SATA drive that understands SAS protocol and > has dual ports. Marketing. > > Basically, if you really want speed at all costs go for SAS. If you want > best capacity for your money, go SATA. If in doubt, go for SATA. If you > don't know you need SAS for some reason, you probably don't. > > Regards, Frank. > > > On 9 August 2017 15:27:37 BST, "Mikhail T." > wrote: > >My server has 8 "hot-plug" slots, that can accept both SATA and SAS > >drives. SATA ones tend to be cheaper for the same features (like > >cache-sizes), what am I getting for the extra money spent on SAS? > > > >Asking specifically about the protocol differences... It would seem, > >for example, SATA can not be as easily hot-plugged, but with > >camcontrol(8) that should not be a problem, right? What else? Thank > >you! > >-- I have a different take on this. For starters SAS and SATA aren't electrically compatible. There's a reason SAS drives are keyed so you can't plug them in to a SATA controller. It keeps the magic smoke inside the drive. SAS controllers can tunnel SATA (They confusingly call this STP (Not Spanning Tree Protocol, but SATA Tunneling Protocol) It's imperfect but good enough for 8 drives. You really do not want to put 60 SATA drives in a SAS JBOD) SAS can be a shared fabric, which means a group of drives are like a room full of people having a conversation. If someone starts screaming and spurting blood it can disrupt the conversations of everyone in the room. Modern RAID controllers are pretty good at disconnecting drives that are not working properly but not completely dead. Modern HBAs not so much. If your controller is an HBA trying to keep a SAS fabric stable with SATA drives can be more problematic than if you use SAS drives...and as Frank pointed out nearline SAS drives are essentially SATA drives with a SAS interface (and are typically under a $20 premium) If performance was an issue we'd be talking about SSDs. While SAS drives do have a performance advantage over SATA in multiuser/multiapplication environments (they have a superior queuing implementation) it's not worth considering when the real solution is SSDs. My recommendation is if you have SAS expanders and an HBA use SAS drives. If you have direct wired SAS or a RAID controller you can use either SAS or SATA. If your application demands performance or concurrency get a couple SSDs. They'll smoke anything any spinning drive can do. -- Thanks, Josh Paetzel From owner-freebsd-hardware@freebsd.org Wed Aug 9 15:53:58 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1E26DD00C8 for ; Wed, 9 Aug 2017 15:53:58 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from bs1.fjl.org.uk (bs1.fjl.org.uk [84.45.41.196]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "bs1.fjl.org.uk", Issuer "bs1.fjl.org.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 70E3776F10 for ; Wed, 9 Aug 2017 15:53:57 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from [10.153.42.245] ([185.69.144.102]) (authenticated bits=0) by bs1.fjl.org.uk (8.14.4/8.14.4) with ESMTP id v79FrtMc043338 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO); Wed, 9 Aug 2017 16:53:55 +0100 (BST) (envelope-from frank2@fjl.co.uk) User-Agent: K-9 Mail for Android In-Reply-To: <1502292592.2001426.1068163912.1191A246@webmail.messagingengine.com> References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> <362B0950-A244-4C65-89C7-898EFC6A4A1F@fjl.co.uk> <1502292592.2001426.1068163912.1191A246@webmail.messagingengine.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Subject: Re: Do I need SAS drives?.. From: "Frank Leonhardt (m)" Date: Wed, 09 Aug 2017 16:53:38 +0100 To: Josh Paetzel , freebsd-hardware@freebsd.org Message-ID: <30F28D62-B262-47C6-929D-58902CBF37A6@fjl.co.uk> X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 15:53:58 -0000 On 9 August 2017 16:29:52 BST, Josh Paetzel wrote: > > >On Wed, Aug 9, 2017, at 09:55 AM, Frank Leonhardt (m) wrote: >> Simple answer is to use either. You're running FreeBSD with ZFS, >right? >> BSD will hot plug anything. I suspect 'hot plug' relates to Microsoft >> workaround hardware RAID. >> >> Hot plug enclosures will also let the host know a drive has been >pulled. >> Otherwise ZFS won't know whether it was pulled or is unresponsive due >to >> it being on fire or something. With 8 drives in your array you can >> probably figure this out yourself. >> >> SAS drives use SCSI commands, which are supposedly better than SATA >> commands. Electrically they are the same. SAS drives are more >expensive >> and tend to be higher spec mechanically, but not always so. >Incidentally, >> nearline SAS is a cheaper SATA drive that understands SAS protocol >and >> has dual ports. Marketing. >> >> Basically, if you really want speed at all costs go for SAS. If you >want >> best capacity for your money, go SATA. If in doubt, go for SATA. If >you >> don't know you need SAS for some reason, you probably don't. >> >> Regards, Frank. >> >> >> On 9 August 2017 15:27:37 BST, "Mikhail T." >> wrote: >> >My server has 8 "hot-plug" slots, that can accept both SATA and SAS >> >drives. SATA ones tend to be cheaper for the same features (like >> >cache-sizes), what am I getting for the extra money spent on SAS? >> > >> >Asking specifically about the protocol differences... It would seem, >> >for example, SATA can not be as easily hot-plugged, but with >> >camcontrol(8) that should not be a problem, right? What else? Thank >> >you! >> >-- > >I have a different take on this. For starters SAS and SATA aren't >electrically compatible. There's a reason SAS drives are keyed so you >can't plug them in to a SATA controller. It keeps the magic smoke >inside the drive. SAS controllers can tunnel SATA (They confusingly >call this STP (Not Spanning Tree Protocol, but SATA Tunneling Protocol) > >It's imperfect but good enough for 8 drives. You really do not want to >put 60 SATA drives in a SAS JBOD) > >SAS can be a shared fabric, which means a group of drives are like a >room full of people having a conversation. If someone starts screaming >and spurting blood it can disrupt the conversations of everyone in the >room. Modern RAID controllers are pretty good at disconnecting drives >that are not working properly but not completely dead. Modern HBAs not >so much. If your controller is an HBA trying to keep a SAS fabric >stable with SATA drives can be more problematic than if you use SAS >drives...and as Frank pointed out nearline SAS drives are essentially >SATA drives with a SAS interface (and are typically under a $20 >premium) > >If performance was an issue we'd be talking about SSDs. While SAS >drives do have a performance advantage over SATA in >multiuser/multiapplication environments (they have a superior queuing >implementation) it's not worth considering when the real solution is >SSDs. > >My recommendation is if you have SAS expanders and an HBA use SAS >drives. If you have direct wired SAS or a RAID controller you can use >either SAS or SATA. If your application demands performance or >concurrency get a couple SSDs. They'll smoke anything any spinning >drive can do. There are differences, but not relevant to an 8 drive system IME. Electrically SAS works at a higher voltage on the differential pair, which means the cables can be a lot longer. Most (all?) SAS Expanders can handle STP so talk to SATA drives, but in an eight-way config I doubt a SAS expander comes in to it - they're not cheap! Incidentally, SATA allows for expanders now. Okay, SAS has tagged command queueing, but SATA has native command queuing. Incidentally, the slightly different notched drive connector is simply to stop you plugging a SAS drive on to a SATA HBA, because it wouldn't know how to talk to it. It won't go bang if you do it by mistake. OTOH a SAS HBA can talk both, so has a notch to match the raised bit on SATA. Could go on about drives for ever (and have done in the past) but this is just an array of eight drives. Have you thought about Fibre channel :-) Regards, Frank. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From owner-freebsd-hardware@freebsd.org Wed Aug 9 15:59:16 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38B80DD017E; Wed, 9 Aug 2017 15:59:16 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-wr0-x22e.google.com (mail-wr0-x22e.google.com [IPv6:2a00:1450:400c:c0c::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8A9476FBE; Wed, 9 Aug 2017 15:59:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-wr0-x22e.google.com with SMTP id y43so25542206wrd.3; Wed, 09 Aug 2017 08:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=dz8yUgnrMCg4tFPB9xlOlypi9GdLXMQSmmy65spPk5U=; b=PpDWgjR1C/YJsOX0HYB+/2kxBc6SwWrb5vODdZlr1BvRf0MfJZnKtPDEILe0gzRobd 1XQ1UAOKS00jST8RuIo5gfRYVD0hHuRcbrPrCqc3xRYQXQY1MY4eT37WmaeybjvYKhlJ A+uuaRtVFVlL5EhhDRPNG0XRYq499Uc7s7itVuMtKOVOvVm+wiPzipSzBvA7NfXgTUpZ QpLp51VVz5V9zCxbdmoNdUCuQ/4Yp5IN6iwmEL0Knw2huZ36xkCdGsLfb8Jsg8fnDE8N ku6sqe3E29tnoXT22EOkQ7y5V+3jgYtmu/7u52Hc5lfQ3MHM+0/TR7qgqokaCpMSQ8Ak uMJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=dz8yUgnrMCg4tFPB9xlOlypi9GdLXMQSmmy65spPk5U=; b=QcKy/mWo1dLMEaVTUKgFwKaaXfVQApu483BDV/CwUlUglVQTO0jYV4Ds3l2mTg7pkK L+BB3eiBsuC8k+vr+Y9uvDtkP6bYztRAybHvM5CeMrzGd1uPF3mkfNEipKobfskxvaSq vYD4xXbHJvYDfyOwRve+GmYgefowWV7zrI2Dwgu/TSj28PgSERH+SE/rbgJssnfdwTQl RZnpArHDD6pUEV8o8mv/zXUpgrMSfPLFML8qsxXxYQ2tBCvMBOFzsoVy+mQecFcI1tCW hpkFQr7o6tRRIifZ0HX7vx2nXM8GL47nrAhlag8jmein/smu3bQICNeKhdmw/6B/cuFK gAMQ== X-Gm-Message-State: AHYfb5joMo6aYf3exLx0RGu+7U71EJKzYnOGTFr2mGz+S/PRdf5HpT59 EnTdTFiPbdMub/WJLaZrmyzp/JvDlw== X-Received: by 10.223.174.209 with SMTP id y75mr5668371wrc.19.1502294354228; Wed, 09 Aug 2017 08:59:14 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.28.208.3 with HTTP; Wed, 9 Aug 2017 08:59:13 -0700 (PDT) In-Reply-To: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> From: Alan Somers Date: Wed, 9 Aug 2017 09:59:13 -0600 X-Google-Sender-Auth: NuITEyLAB4lYAZ4VdGC7OeHpXPM Message-ID: Subject: Re: Do I need SAS drives?.. To: "Mikhail T." Cc: freebsd-hardware@freebsd.org, FreeBSD-scsi Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 15:59:16 -0000 On Wed, Aug 9, 2017 at 8:27 AM, Mikhail T. wrote: > My server has 8 "hot-plug" slots, that can accept both SATA and SAS drives. SATA ones tend to be cheaper for the same features (like cache-sizes), what am I getting for the extra money spent on SAS? > > Asking specifically about the protocol differences... It would seem, for example, SATA can not be as easily hot-plugged, but with camcontrol(8) that should not be a problem, right? What else? Thank you! > -- > Sent from mobile device, please, pardon shorthand. Good question. First of all, hot-plugability has more to do with the controller than the protocol. Since you have a SAS controller, you should have no problem hot plugging SATA drives. But SAS drives still have a few advantages: 1) When a SATA drive goes into error recovery, it can lock up the bus indefinitely. This won't matter if your drives are directly connected to a SAS HBA. But if you have an expander with say, 4 SAS lanes going to the HBA, then a flaky SATA drive can reduce the bandwidth available to the good drives. 2) Even with NCQ, the SATA protocol is limited to queueing one or more write commands OR one or more read commands. You can't queue a mixture of reads and writes at the same time. SAS does not have that limitation. In this sense, SAS is theoretically more performant. However, I've never heard of anybody observing a performance problem that can be definitely blamed on this effect. 3) SAS drives have a lot of fancy features that you may not need or care about. For example, they often have features that are useful in multipath setups (dual ports, persistent reservations), their error reporting capabilities are more sophisticated than SMART, their self encrypting command set is more sophisticated, etc etc. 4) The SAS activity LED is the opposite of SATA's. With SATA, the LED is off for an idle drive or blinking for a busy drive. With SAS, it's on for an idle drive or blinking for a busy drive. This makes it easier to see at a glance how many SAS drives you have installed. I think some SATA drives have a way to change the LEDs behavior, though. 5) Desktop class SATA drives can spend an indefinite amount of time in error recovery mode. If your RAID stack doesn't timeout a command, that can cause your array to hang. But SAS drives and RAID class SATA drives will fail any command than spends too much time in error recovery mode. 6) But the most important difference isn't something you'll find on any datasheet or protocol manual. SAS drives are built to a higher standard of quality than SATA drives, and have accordingly lower failure rates. I'm guessing that you don't have an expander (since you only have 8 slots), so item 1 doesn't matter to you. I'll guess that item 3 doesn't matter either, or you wouldn't have asked this question. Item 5 can be dealt with simply by buying the higher end SATA drives. So item 6 is really the most important. If this system needs to have very high uptime and consistent bandwidth, or if it will be difficult to access for maintenance, then you probably want to use SAS drives. If not, then you can save some money by using SATA. Hope that helps. -Alan From owner-freebsd-hardware@freebsd.org Wed Aug 9 21:23:09 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7C93DD6440 for ; Wed, 9 Aug 2017 21:23:09 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from bs1.fjl.org.uk (bs1.fjl.org.uk [84.45.41.196]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "bs1.fjl.org.uk", Issuer "bs1.fjl.org.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 69E1B2131 for ; Wed, 9 Aug 2017 21:23:09 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from [192.168.1.35] (host86-191-18-76.range86-191.btcentralplus.com [86.191.18.76]) (authenticated bits=0) by bs1.fjl.org.uk (8.14.4/8.14.4) with ESMTP id v79LN0Hs006897 for ; Wed, 9 Aug 2017 22:23:01 +0100 (BST) (envelope-from frank2@fjl.co.uk) Subject: Re: Do I need SAS drives?.. To: freebsd-hardware@freebsd.org References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> From: Frank Leonhardt Message-ID: Date: Wed, 9 Aug 2017 22:23:00 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 21:23:10 -0000 On 09/08/2017 16:59, Alan Somers wrote: > On Wed, Aug 9, 2017 at 8:27 AM, Mikhail T. wrote: >> My server has 8 "hot-plug" slots, that can accept both SATA and SAS drives. SATA ones tend to be cheaper for the same features (like cache-sizes), what am I getting for the extra money spent on SAS? >> >> Asking specifically about the protocol differences... It would seem, for example, SATA can not be as easily hot-plugged, but with camcontrol(8) that should not be a problem, right? What else? Thank you! >> -- >> Sent from mobile device, please, pardon shorthand. > Good question. First of all, hot-plugability has more to do with the > controller than the protocol. Since you have a SAS controller, you > should have no problem hot plugging SATA drives. But SAS drives still > have a few advantages: > > 1) When a SATA drive goes into error recovery, it can lock up the bus > indefinitely. This won't matter if your drives are directly connected > to a SAS HBA. But if you have an expander with say, 4 SAS lanes going > to the HBA, then a flaky SATA drive can reduce the bandwidth available > to the good drives. > > 2) Even with NCQ, the SATA protocol is limited to queueing one or more > write commands OR one or more read commands. You can't queue a > mixture of reads and writes at the same time. SAS does not have that > limitation. In this sense, SAS is theoretically more performant. > However, I've never heard of anybody observing a performance problem > that can be definitely blamed on this effect. > > 3) SAS drives have a lot of fancy features that you may not need or > care about. For example, they often have features that are useful in > multipath setups (dual ports, persistent reservations), their error > reporting capabilities are more sophisticated than SMART, their self > encrypting command set is more sophisticated, etc etc. > > 4) The SAS activity LED is the opposite of SATA's. With SATA, the LED > is off for an idle drive or blinking for a busy drive. With SAS, it's > on for an idle drive or blinking for a busy drive. This makes it > easier to see at a glance how many SAS drives you have installed. I > think some SATA drives have a way to change the LEDs behavior, though. > > 5) Desktop class SATA drives can spend an indefinite amount of time in > error recovery mode. If your RAID stack doesn't timeout a command, > that can cause your array to hang. But SAS drives and RAID class > SATA drives will fail any command than spends too much time in error > recovery mode. > > 6) But the most important difference isn't something you'll find on > any datasheet or protocol manual. SAS drives are built to a higher > standard of quality than SATA drives, and have accordingly lower > failure rates. > > I'm guessing that you don't have an expander (since you only have 8 > slots), so item 1 doesn't matter to you. I'll guess that item 3 > doesn't matter either, or you wouldn't have asked this question. Item > 5 can be dealt with simply by buying the higher end SATA drives. So > item 6 is really the most important. If this system needs to have > very high uptime and consistent bandwidth, or if it will be difficult > to access for maintenance, then you probably want to use SAS drives. > If not, then you can save some money by using SATA. Hope that helps. > > -Alan Alan makes a good point about SAS expanders and their tendency to stick when some SATA drives go off on a trip. I'm also assuming Mikhail(?)'s setup doesn't use one. On BSD with ZFS, a SATA drive chucking a shoe doesn't make any difference if they're directly connected to the HBA (same applies to GEOM RAID/MIRROR). "Dive silent?", "Detach it". I'm not at all convinced that SAS is any more reliable than SATA per se. This is based on 30+ years experience with Winchesters starting with ST506. In the UK I used to write most of the storage articles for a couple of major tech publishers, and I spent a lot of time talking to and visiting the manufacturers and looking around the factories. Some of this may now be out-of-date (Conner went bust for a start). The thing is that if you opened a XXX brand SCSI disk and the IDE version, guess what? They were the same inside. I spoke to the makers, and apparently the electronics on the SCSI version is a lot more expensive. Why? Well we don't sell as many, er, um. Okay, they don't make cheap and nasty SCSI (or SAS) drives, but they do make low-end IDE/SATA. They also make some very nice drives that are only available as SAS. An equivalent quality SAS/SATA drive will be just as reliable - there's no mechanical reason for them not to be. They come off the same line. Then there's the MTBF and the unrecoverable error rates. On high-end drives the latter is normally claimed to be 10x better than the cheap ones. Pretty much always, and exactly 10x. This is utter bilge. What they're saying is that the unrecoverable error rate is this figure or better, and any study in to this has shown that it's usually a lot better than both figures. So both figures are technically correct; it just makes the SATA drive look worse. If anyone has any actual evidence of equivalent SAS and SATA drives having a different error rate, please get in touch. MTBF? Okay, SATA drives do fail more quickly. Run a drive 24/7 for a couple of years in an array and it only spins up once and runs at a constant speed; doesn't get knocked and has properly organised air conditioning (no thermal shocks). The SATA drive in a desktop, on the other hand, gets turned on and off and generally abused. It may be running for less actual time but the odds are stacked against it. How many light bulbs fail while they're running vs. how many fail when you turn them on? Finally, there's been my experience running a load of drives in data centres for many years. In some servers there are SAS drives. In others there are SATA server drives (supplied by Dell at 4x the cost of cheap ones). And in others there are cheapo drives than were around and whacked in when half a mirror failed. You know what's coming, don't you? So I won't say it. Regards, Frank. From owner-freebsd-hardware@freebsd.org Wed Aug 9 21:37:15 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D96AEDD6778 for ; Wed, 9 Aug 2017 21:37:15 +0000 (UTC) (envelope-from lnb@freebsdsystems.com) Received: from smtpd.barontel.net (smtpd.barontel.net [104.193.49.23]) by mx1.freebsd.org (Postfix) with SMTP id A1E5C264C for ; Wed, 9 Aug 2017 21:37:15 +0000 (UTC) (envelope-from lnb@freebsdsystems.com) Received: (qmail 96204 invoked by uid 89); 9 Aug 2017 21:30:33 -0000 Received: from unknown (HELO smtpd.barontel.net) (172.16.15.9) by vpopmail with SMTP; 9 Aug 2017 21:30:33 -0000 Authentication-Results: smtpd.barontel.net; iprev=pass; auth=pass (plain); spf=pass smtp.mailfrom=freebsdsystems.com Received-SPF: Pass (smtpd.barontel.net: domain of freebsdsystems.com designates 99.237.72.210 as permitted sender) receiver=smtpd.barontel.net; identity=mailfrom; client-ip=99.237.72.210; helo=[192.168.25.24]; envelope-from= Received: from [192.168.25.24] (CPE4c5e0c417c51-CMbc4dfbbdc320.cpe.net.cable.rogers.com [99.237.72.210]) by smtpd.barontel.net (Haraka/2.8.13) with ESMTPSA id 6FE6F683-43FE-4B66-A8A0-75A73CC244CA.1 envelope-from (authenticated bits=0) (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES128-GCM-SHA256 verify=FAIL); Wed, 09 Aug 2017 17:30:32 -0400 Subject: Re: Do I need SAS drives?.. To: freebsd-hardware@freebsd.org References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> <362B0950-A244-4C65-89C7-898EFC6A4A1F@fjl.co.uk> <1502292592.2001426.1068163912.1191A246@webmail.messagingengine.com> From: Lanny Baron Organization: Freedom Technologies Corp. FreeBSD Systems Message-ID: <4cd42cce-8e6c-b421-9bf2-08a72945779d@freebsdsystems.com> Date: Wed, 9 Aug 2017 17:30:31 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <1502292592.2001426.1068163912.1191A246@webmail.messagingengine.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Haraka-FCrDNS: cpe4c5e0c417c51-cmbc4dfbbdc320.cpe.net.cable.rogers.com X-Haraka-Karma: score: 14, good: 327, bad: 7, connections: 336, history: 320, awards: 130, 133, 162, 182, pass:relaying, fail:rcpt_to X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 21:37:15 -0000 Not sure what kind of server you are referring to but our servers can take SAS and SATA at the same time. We build plenty of servers running FreeBSD which in some cases have SATA SSD for boot drives (in a RAID-1) and then X amount of either SATA or SAS or both in a different RAID configuration all connected to the same high quality RAID Controller. I have yet to see any complaint with the configurations we've done for our clients. SAS drives can be much faster. 15K RPM vs. SATA 7.2K. Your choices would depend on how busy the server is. Regards, Lanny On 8/9/2017 11:29 AM, Josh Paetzel wrote: > > > On Wed, Aug 9, 2017, at 09:55 AM, Frank Leonhardt (m) wrote: >> Simple answer is to use either. You're running FreeBSD with ZFS, right? >> BSD will hot plug anything. I suspect 'hot plug' relates to Microsoft >> workaround hardware RAID. >> >> Hot plug enclosures will also let the host know a drive has been pulled. >> Otherwise ZFS won't know whether it was pulled or is unresponsive due to >> it being on fire or something. With 8 drives in your array you can >> probably figure this out yourself. >> >> SAS drives use SCSI commands, which are supposedly better than SATA >> commands. Electrically they are the same. SAS drives are more expensive >> and tend to be higher spec mechanically, but not always so. Incidentally, >> nearline SAS is a cheaper SATA drive that understands SAS protocol and >> has dual ports. Marketing. >> >> Basically, if you really want speed at all costs go for SAS. If you want >> best capacity for your money, go SATA. If in doubt, go for SATA. If you >> don't know you need SAS for some reason, you probably don't. >> >> Regards, Frank. >> >> >> On 9 August 2017 15:27:37 BST, "Mikhail T." >> wrote: >>> My server has 8 "hot-plug" slots, that can accept both SATA and SAS >>> drives. SATA ones tend to be cheaper for the same features (like >>> cache-sizes), what am I getting for the extra money spent on SAS? >>> >>> Asking specifically about the protocol differences... It would seem, >>> for example, SATA can not be as easily hot-plugged, but with >>> camcontrol(8) that should not be a problem, right? What else? Thank >>> you! >>> -- > > I have a different take on this. For starters SAS and SATA aren't > electrically compatible. There's a reason SAS drives are keyed so you > can't plug them in to a SATA controller. It keeps the magic smoke > inside the drive. SAS controllers can tunnel SATA (They confusingly > call this STP (Not Spanning Tree Protocol, but SATA Tunneling Protocol) > It's imperfect but good enough for 8 drives. You really do not want to > put 60 SATA drives in a SAS JBOD) > > SAS can be a shared fabric, which means a group of drives are like a > room full of people having a conversation. If someone starts screaming > and spurting blood it can disrupt the conversations of everyone in the > room. Modern RAID controllers are pretty good at disconnecting drives > that are not working properly but not completely dead. Modern HBAs not > so much. If your controller is an HBA trying to keep a SAS fabric > stable with SATA drives can be more problematic than if you use SAS > drives...and as Frank pointed out nearline SAS drives are essentially > SATA drives with a SAS interface (and are typically under a $20 premium) > > If performance was an issue we'd be talking about SSDs. While SAS > drives do have a performance advantage over SATA in > multiuser/multiapplication environments (they have a superior queuing > implementation) it's not worth considering when the real solution is > SSDs. > > My recommendation is if you have SAS expanders and an HBA use SAS > drives. If you have direct wired SAS or a RAID controller you can use > either SAS or SATA. If your application demands performance or > concurrency get a couple SSDs. They'll smoke anything any spinning > drive can do. > From owner-freebsd-hardware@freebsd.org Thu Aug 10 13:44:54 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36C33DD33FE; Thu, 10 Aug 2017 13:44:54 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE26A1687; Thu, 10 Aug 2017 13:44:53 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x231.google.com with SMTP id f15so22964428wmg.1; Thu, 10 Aug 2017 06:44:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=D+xEiEBYveIASKYA1Wjt8vh9UegHK2Qh2mqs7H5oWyQ=; b=aH8vBoKkRzSWzQJtkmYOLVy2qdqdRiu22dYB4Nwxh2hy7ta0HGOfrIjIVGRJnZzN2B w0C/aL4O/WcvjVrpCiXuj/lG7So6CmIphzP7ZwiVyIOk/iscW5MN444pb9AVIo7hbSGK Rvw7kmJwX3oZLebjIwR+pAtKJnBJnITxhoOMc2UiZSNB2TkkfdkODgp9bbx/JEqkQqGJ Q/ALFxnbv5xgbIu6VuuBy8dTg3JB/6GqzaNuLgd9wFjCXDJ+IvyzyxkSP8CO5VVKVa0D 9nP0MIJTMQES++0J79GHbOFKsk2HGo83q0FBi8h1n0lOnRM7QICjnI7xiKNOvsiJPaK2 od7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=D+xEiEBYveIASKYA1Wjt8vh9UegHK2Qh2mqs7H5oWyQ=; b=lGizENCQBgU0LsS0tf9W4neoOTz5RneyZXARRqphtuiopHFbHdE5We8/MTlutDO45u A0TSnTV9wFNfiioE6ihX9mFz7J+0B54X8DZYMajrSKXt+4wXDutQ5jEj5nOFBzJSPUK/ f54lUA2U3gXl+/yfPvnS3OaHe4PfTk5H4Ny9kRlZo5b+gySSxT29Rvz12X+EX2ak3gy8 ki8IH9MWkbvsinW0YB2ejVT7Dbe2WlywD89DPZiaG0ZCOfzuQhgnU+bCHz4KyFRjJtuq PAkHHvLV1AqneVDlyJmIsLFprWDapNId5p7v+OjFAMhTs7lrM051G4udRZ1I1tzDH8Wx Mrgg== X-Gm-Message-State: AHYfb5jislAGrR76HNYaCLsGhFN3iiGnDHPt0yDwf8qKR6pELgJThAts krYw67haJyyVxjGmkfE= X-Received: by 10.28.9.9 with SMTP id 9mr7457520wmj.93.1502372692020; Thu, 10 Aug 2017 06:44:52 -0700 (PDT) Received: from ben.home (LFbn-1-6951-179.w90-116.abo.wanadoo.fr. [90.116.132.179]) by smtp.gmail.com with ESMTPSA id 60sm4388472wrd.20.2017.08.10.06.44.51 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 10 Aug 2017 06:44:51 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Do I need SAS drives?.. From: Ben RUBSON In-Reply-To: Date: Thu, 10 Aug 2017 15:44:50 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> To: FreeBSD-scsi , freebsd-hardware@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2017 13:44:54 -0000 > On 09 Aug 2017, at 17:59, Alan Somers wrote: >=20 > 3) SAS drives have a lot of fancy features that you may not need or > care about. For example, (...) their error > reporting capabilities are more sophisticated than SMART Really interesting answer Alan, thank you very much ! Slightly off-topic but I take this opportunity, how do you check SAS drives health ? I personally cron a background long test every 2 weeks (using = smartmontools). I did not experience SAS drive error yet, so not sure how this behaves. Does the drive reports to FreeBSD when its read or write error rate = cross a threshold (so that we can replace it before it fails) ? Or perhaps smartd will do ? As an example below a SAS error counter log returned by smartctl : Errors Corrected by Total Correction Gigabytes = Total ECC rereads/ errors algorithm processed = uncorrected fast | delayed rewrites corrected invocations [10^9 bytes] = errors read: 0 49 0 49 233662 73743.588 = 0 write: 0 3 0 3 83996 9118.895 = 0 verify: 0 0 0 0 28712 0.000 = 0 Thank you ! Ben From owner-freebsd-hardware@freebsd.org Thu Aug 10 14:01:49 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03535DD3AC9; Thu, 10 Aug 2017 14:01:49 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D4B6215A; Thu, 10 Aug 2017 14:01:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-wm0-x236.google.com with SMTP id f15so23400468wmg.1; Thu, 10 Aug 2017 07:01:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Z7bKJx5yIn2Q8yVVDmcOhwU4mPEDxWqePFVlFI0CfL0=; b=fpv7yE6o23ctmSUYEsDpDcY3BcbpIQWgXQfyU20v4EuKRxgb9PbWkC7tKDVi/aghdj m3X8hUbIXqg5yOPZdaIL9S6ERpZRjc6iG8L9WLynZI9soqYrPnRubGjQg+IqbFWSb4jC RQJQtDe1/SL72IPErfeLcX7sUYFx2QeiZaUwtsIct1NNH+I0OhoOAR6vfiBjQsLwQn3Z Xa2+zDxY1DbkFMuLCYTyelm0VE1/atPJkM6AYwcP6Px8yyTvDFoQFOjtRKH9AoLMXp9o AYcSBI9uWS+sy371tZ7SKrfNSrXLWupbIvh10T9CGT7qocQ+GDlPUf9lWkPOGJcXGyHN 3HjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Z7bKJx5yIn2Q8yVVDmcOhwU4mPEDxWqePFVlFI0CfL0=; b=gubTab91Sa7Pzv7gFWqeCouHvXc9A4HXUitTaf4tBfGehm3CC+TCndBB2P6LIoF3jZ UuaFVHA476Nv855kpuJjL7NdqGqnaMkuAqTKruAyXz+wYArtafF4vn50mEoOEs1o4Vr7 kc4MaM62MhjnEEbJN1IQFVEf3SlTHJTv7WW1UL2wEwhbyRKKx8ilftHJf6bgWhlxOdK6 5do8vfqYL6BvHWfAEOUNgAAfzvhIU7F9lDZ1vKMwhn87SZ8siZf0xub97bfsfaaYeL6t ZCaV0EE9qHMvOr3O243cDQzU4L5JNuzl38tExhQ6U43juM+pEwhSKOWkgqcASU3pfnHG 6n+A== X-Gm-Message-State: AHYfb5jDb5XFCT9C6tTQrDQKxbplH0vjxOGdn33sOwgzWlGyc9jlznHt TZXQKqF0rVwlwa+VS2iua1CtN33YxQ== X-Received: by 10.28.170.18 with SMTP id t18mr7603839wme.6.1502373706889; Thu, 10 Aug 2017 07:01:46 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.28.208.3 with HTTP; Thu, 10 Aug 2017 07:01:46 -0700 (PDT) In-Reply-To: References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> From: Alan Somers Date: Thu, 10 Aug 2017 08:01:46 -0600 X-Google-Sender-Auth: V3Ta4p3yRSRdEMKZCbwrv6N80-I Message-ID: Subject: Re: Do I need SAS drives?.. To: Ben RUBSON Cc: FreeBSD-scsi , freebsd-hardware@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2017 14:01:49 -0000 On Thu, Aug 10, 2017 at 7:44 AM, Ben RUBSON wrote: >> On 09 Aug 2017, at 17:59, Alan Somers wrote: >> >> 3) SAS drives have a lot of fancy features that you may not need or >> care about. For example, (...) their error >> reporting capabilities are more sophisticated than SMART > > Really interesting answer Alan, thank you very much ! > Slightly off-topic but I take this opportunity, > how do you check SAS drives health ? > I personally cron a background long test every 2 weeks (using smartmontools). > I did not experience SAS drive error yet, so not sure how this behaves. > Does the drive reports to FreeBSD when its read or write error rate cross > a threshold (so that we can replace it before it fails) ? > Or perhaps smartd will do ? > > As an example below a SAS error counter log returned by smartctl : > Errors Corrected by Total Correction Gigabytes Total > ECC rereads/ errors algorithm processed uncorrected > fast | delayed rewrites corrected invocations [10^9 bytes] errors > read: 0 49 0 49 233662 73743.588 0 > write: 0 3 0 3 83996 9118.895 0 > verify: 0 0 0 0 28712 0.000 0 > > Thank you ! > > Ben smartmontools is probably the best way to read SAS error logs. Interpreting them can be hard, though. The Backblaze blog is probably the best place to get current advice. But the easiest thing to do is certainly to wait until something fails hard. With ZFS, you can have up to 3 drives' worth of redundancy, and hotspares too. -Alan From owner-freebsd-hardware@freebsd.org Thu Aug 10 15:23:15 2017 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07A0DDD5839 for ; Thu, 10 Aug 2017 15:23:15 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from bs1.fjl.org.uk (bs1.fjl.org.uk [84.45.41.196]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "bs1.fjl.org.uk", Issuer "bs1.fjl.org.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A1C3864972 for ; Thu, 10 Aug 2017 15:23:13 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from [192.168.1.35] (host86-191-18-76.range86-191.btcentralplus.com [86.191.18.76]) (authenticated bits=0) by bs1.fjl.org.uk (8.14.4/8.14.4) with ESMTP id v7AFMwNO020008 for ; Thu, 10 Aug 2017 16:23:00 +0100 (BST) (envelope-from frank2@fjl.co.uk) Subject: Re: Do I need SAS drives?.. To: freebsd-hardware@freebsd.org References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> From: Frank Leonhardt Message-ID: <25450400-4ba2-76d4-605c-fce37c1c905b@fjl.co.uk> Date: Thu, 10 Aug 2017 16:22:59 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2017 15:23:15 -0000 On 10/08/2017 15:01, Alan Somers wrote: > Really interesting answer Alan, thank you very much ! >> Slightly off-topic but I take this opportunity, >> how do you check SAS drives health ? >> I personally cron a background long test every 2 weeks (using smartmontools). >> I did not experience SAS drive error yet, so not sure how this behaves. >> Does the drive reports to FreeBSD when its read or write error rate cross >> a threshold (so that we can replace it before it fails) ? >> Or perhaps smartd will do ? >> >> As an example below a SAS error counter log returned by smartctl : >> Errors Corrected by Total Correction Gigabytes Total >> ECC rereads/ errors algorithm processed uncorrected >> fast | delayed rewrites corrected invocations [10^9 bytes] errors >> read: 0 49 0 49 233662 73743.588 0 >> write: 0 3 0 3 83996 9118.895 0 >> verify: 0 0 0 0 28712 0.000 0 >> >> Thank you ! >> >> Ben > smartmontools is probably the best way to read SAS error logs. > Interpreting them can be hard, though. The Backblaze blog is probably > the best place to get current advice. But the easiest thing to do is > certainly to wait until something fails hard. With ZFS, you can have > up to 3 drives' worth of redundancy, and hotspares too. I concur with Alan. Trying to predict drive failure is a mug's game. Very through research (e.g. Google, 2007) has shown it's a waste of time trying. With ZFS (or geom mirror) a drive will be "failed" as soon as there's a problem and you can get notification using a cron job that sends an email if the output of zpool status (or gmirror status ) contains "DEGRADED". That said, I've found it useful to use smartctl to pick up when a drive is overheating, usually due to fan failure. You might also find the new (11.0+?) sesutil handy to monitor components on a SAS expander IF YOU HAVE ONE. Things like fans and temperature sensors are readable this way. Regards, Frank.