From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 12 00:59:58 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 957C5E49 for ; Sun, 12 Apr 2015 00:59:58 +0000 (UTC) Received: from udns.ultimatedns.net (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6C6A9EDA for ; Sun, 12 Apr 2015 00:59:58 +0000 (UTC) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id t3C10fG3038794; Sat, 11 Apr 2015 18:00:47 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) To: freebsd-hackers@freebsd.org, Wojciech Puchar In-Reply-To: References: From: "Chris H" Subject: Re: Help with 8TB Seagate Archive Date: Sat, 11 Apr 2015 18:00:47 -0700 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2015 00:59:58 -0000 On Sun, 12 Apr 2015 00:05:20 +0200 (CEST) Wojciech Puchar wrote > got it and connected > > in logs i see > > (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 > 00 40 00 00 00 00 46 00 > (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error > (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) > (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 > (aprobe0:ahcich0:0:0:0): Retrying command > (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 > 00 40 00 00 00 00 46 00 > (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error > (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) > (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted Not much to go on. But maybe drive contention? In other words; is this the only drive on the wire? If need be, is it shunted correctly? Just some thoughts. --Chris > > > > > > and it doesn't connect. > > any ideas? > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 12 07:25:32 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5E2D9916 for ; Sun, 12 Apr 2015 07:25:32 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D7B96911 for ; Sun, 12 Apr 2015 07:25:31 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t3C7PGrK072568 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 12 Apr 2015 09:25:16 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t3C7PHiu000732; Sun, 12 Apr 2015 09:25:17 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t3C7PBc9000729; Sun, 12 Apr 2015 09:25:12 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Sun, 12 Apr 2015 09:25:11 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Chris H Subject: Re: Help with 8TB Seagate Archive In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Sun, 12 Apr 2015 09:25:16 +0200 (CEST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2015 07:25:32 -0000 >> (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 >> (aprobe0:ahcich0:0:0:0): Retrying command >> (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 >> 00 40 00 00 00 00 46 00 >> (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error >> (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) >> (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > > Not much to go on. But maybe drive contention? > In other words; is this the only drive on the wire? If need be, > is it shunted correctly? yes. i took out older drive and put new in place. this is intel atom board with 2 SATA used as backup server. with 2 disks, one is new seagate archive. can this be cable problem? From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 12 19:10:15 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 203E52E0 for ; Sun, 12 Apr 2015 19:10:15 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 98906A08 for ; Sun, 12 Apr 2015 19:10:14 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t3CJ9sIw093322 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 12 Apr 2015 21:09:54 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t3CJ9sKG018448; Sun, 12 Apr 2015 21:09:54 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t3CJ9mvJ018445; Sun, 12 Apr 2015 21:09:49 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Sun, 12 Apr 2015 21:09:48 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Chris H Subject: Re: Help with 8TB Seagate Archive In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Sun, 12 Apr 2015 21:09:54 +0200 (CEST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2015 19:10:15 -0000 thanks for help. Disk arrived failed - doesn't work on 2 different computers, in FreeBSD and windows 7. On Sat, 11 Apr 2015, Chris H wrote: > On Sun, 12 Apr 2015 00:05:20 +0200 (CEST) Wojciech Puchar > wrote > >> got it and connected >> >> in logs i see >> >> (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 >> 00 40 00 00 00 00 46 00 >> (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error >> (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) >> (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 >> (aprobe0:ahcich0:0:0:0): Retrying command >> (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 >> 00 40 00 00 00 00 46 00 >> (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error >> (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) >> (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > > Not much to go on. But maybe drive contention? > In other words; is this the only drive on the wire? If need be, > is it shunted correctly? > > Just some thoughts. > > --Chris > >> >> >> >> >> >> and it doesn't connect. >> >> any ideas? >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 12 19:19:23 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60E6A4F6 for ; Sun, 12 Apr 2015 19:19:23 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DAB6CB33 for ; Sun, 12 Apr 2015 19:19:22 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t3CJJKIM093515 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sun, 12 Apr 2015 21:19:20 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t3CJJKL6018557 for ; Sun, 12 Apr 2015 21:19:20 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t3CJJFw9018554 for ; Sun, 12 Apr 2015 21:19:15 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Sun, 12 Apr 2015 21:19:15 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: freebsd-hackers@freebsd.org Subject: stack pointer on 64-bit architecture Message-ID: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Sun, 12 Apr 2015 21:19:20 +0200 (CEST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2015 19:19:23 -0000 what decides where stack is located when process starts on 64-bit machine (x86-64)? it starts normally below 0x0000800000000000 can starting address be set to different (smaller) value? From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 12 21:06:01 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6051A7DB for ; Sun, 12 Apr 2015 21:06:01 +0000 (UTC) Received: from udns.ultimatedns.net (unknown [IPv6:2602:d1:b4d6:e600:4261:86ff:fef6:aa2a]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3491B87A for ; Sun, 12 Apr 2015 21:06:01 +0000 (UTC) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id t3CL6mfY079262; Sun, 12 Apr 2015 14:06:54 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) To: Wojciech Puchar In-Reply-To: References: , From: "Chris H" Subject: Re: Help with 8TB Seagate Archive Date: Sun, 12 Apr 2015 14:06:54 -0700 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: <26d432bfaa053da09d9fb56bf68254ad@ultimatedns.net> Content-Transfer-Encoding: 8bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Apr 2015 21:06:01 -0000 On Sun, 12 Apr 2015 21:09:48 +0200 (CEST) Wojciech Puchar wrote > thanks for help. Disk arrived failed - doesn't work on 2 different > computers, in FreeBSD and windows 7. Sorry to reply so late. Sorry to hear the bad news. At least you know the cause now. I would have suggested connecting the new drive to the port that your current working drive was plugged in to, and booting from the bootonly CD/DVD, and see if the drive came up normally (w/o error). If not, know the drive was bad. Anyway, seems you found out for yourself. All the best. --Chris > > On Sat, 11 Apr 2015, Chris H wrote: > > > On Sun, 12 Apr 2015 00:05:20 +0200 (CEST) Wojciech Puchar > > wrote > > > >> got it and connected > >> > >> in logs i see > >> > >> (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 > >> 00 40 00 00 00 00 46 00 > >> (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error > >> (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) > >> (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 > >> (aprobe0:ahcich0:0:0:0): Retrying command > >> (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 > >> 00 40 00 00 00 00 46 00 > >> (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error > >> (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) > >> (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > > > > Not much to go on. But maybe drive contention? > > In other words; is this the only drive on the wire? If need be, > > is it shunted correctly? > > > > Just some thoughts. > > > > --Chris > > > >> > >> > >> > >> > >> > >> and it doesn't connect. > >> > >> any ideas? > >> _______________________________________________ > >> freebsd-hackers@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > > > > From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 00:09:48 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 612828BB for ; Mon, 13 Apr 2015 00:09:48 +0000 (UTC) Received: from mail-qk0-f180.google.com (mail-qk0-f180.google.com [209.85.220.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21092BB4 for ; Mon, 13 Apr 2015 00:09:47 +0000 (UTC) Received: by qku63 with SMTP id 63so149611944qku.3 for ; Sun, 12 Apr 2015 17:09:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=L02SG44spPNvR8kjh5z4j7nKAfNW4wooWggm+++w6UA=; b=Qz7i44XOR8PCRle1UKCNqAno6GDoC6z4fgtP+F4/E2qrmvpw8KpnI9dORIuIjPB9+7 8s/cBIF4EPavqUVAaka2oU/5O3nNE3YYDrYMtrBV0NunIXzKrQYguj3sXxudFaHuWkoZ h3BZ5Dj3fP+a/AxD75bCTP/xnYY0r5+JIwl6HktIu2+LA9DITHsgnmfckE3FP/TA9mWn KaFtWa+SBq6B6NoyMMWgovAM4O+jgTSRgJYapLDWCXejYKiSUa9nbeoEmWjjbcaY72A1 sSzb5hvR4EoDXVzPyxPtIPVzw2DtFnSJWcJILC3b7QBh/UnIOcyQYKnFtvYbo8LB9tJX HizQ== X-Gm-Message-State: ALoCoQk/AxsNGFAKJW3VWix6KQN4JubqOzVwDaN4MMCsNwRLH3WBbFQM9V/JaxdDN89W5BxlLc8R MIME-Version: 1.0 X-Received: by 10.182.105.66 with SMTP id gk2mr10202321obb.76.1428883371465; Sun, 12 Apr 2015 17:02:51 -0700 (PDT) Received: by 10.202.80.6 with HTTP; Sun, 12 Apr 2015 17:02:51 -0700 (PDT) In-Reply-To: References: Date: Mon, 13 Apr 2015 02:02:51 +0200 Message-ID: Subject: Re: stack pointer on 64-bit architecture From: Oliver Pinter To: Wojciech Puchar Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 00:09:48 -0000 The stack address come from this code: https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_exec.c#L1065 where sv->sv_usrstack depends on architecture and on image activator. and this the other place, where the stack address is "hardcoded": https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_exec.c#L1240 Just grep against sv_psstrings and sv_usrstack to see what depend on current hardcoded stack address. ;) On Sun, Apr 12, 2015 at 9:19 PM, Wojciech Puchar wrote: > what decides where stack is located when process starts on 64-bit machine > (x86-64)? > > it starts normally below 0x0000800000000000 > > can starting address be set to different (smaller) value? > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 07:14:04 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A19C4C8 for ; Mon, 13 Apr 2015 07:14:04 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D297B80 for ; Mon, 13 Apr 2015 07:14:03 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t3D7DrrY008226 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 13 Apr 2015 09:13:53 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t3D7Ds6V000731; Mon, 13 Apr 2015 09:13:54 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t3D7Dlqr000728; Mon, 13 Apr 2015 09:13:48 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Mon, 13 Apr 2015 09:13:47 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Oliver Pinter Subject: Re: stack pointer on 64-bit architecture In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Mon, 13 Apr 2015 09:13:54 +0200 (CEST) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 07:14:04 -0000 thanks On Mon, 13 Apr 2015, Oliver Pinter wrote: > The stack address come from this code: > > https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_exec.c#L1065 > where sv->sv_usrstack depends on architecture and on image activator. > > and this the other place, where the stack address is "hardcoded": > https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_exec.c#L1240 > > > Just grep against sv_psstrings and sv_usrstack to see what depend on > current hardcoded stack address. ;) > > On Sun, Apr 12, 2015 at 9:19 PM, Wojciech Puchar wrote: >> what decides where stack is located when process starts on 64-bit machine >> (x86-64)? >> >> it starts normally below 0x0000800000000000 >> >> can starting address be set to different (smaller) value? >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 07:31:10 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 97327920 for ; Mon, 13 Apr 2015 07:31:10 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1EA3DD6E for ; Mon, 13 Apr 2015 07:31:09 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t3D7V7e1008718 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 13 Apr 2015 09:31:07 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t3D7V79N000939; Mon, 13 Apr 2015 09:31:08 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t3D7V24e000936; Mon, 13 Apr 2015 09:31:02 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Mon, 13 Apr 2015 09:31:02 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Oliver Pinter Subject: another question - VM mappings Message-ID: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Mon, 13 Apr 2015 09:31:07 +0200 (CEST) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 07:31:10 -0000 below is mapping for very simple process (no libc etc) [wojtek@laptop ~]$ procstat -v 917 PID START END PRT RES PRES REF SHD FL TP PATH 917 0x400000 0x401000 r-x 1 0 1 0 CN-- vn /home/wojtek/test/1 917 0x600000 0x601000 rw- 1 0 1 0 ---- df 917 0x7ffffffdf000 0x7ffffffff000 rw- 1 0 1 0 ---D df 917 0x7ffffffff000 0x800000000000 r-x 0 0 39 0 ---- ph what is "ph" mapping and why read&executable? From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 09:26:19 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB97B650 for ; Mon, 13 Apr 2015 09:26:19 +0000 (UTC) Received: from mail-ie0-x22f.google.com (mail-ie0-x22f.google.com [IPv6:2607:f8b0:4001:c03::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B78FB8D for ; Mon, 13 Apr 2015 09:26:19 +0000 (UTC) Received: by iejt8 with SMTP id t8so59705691iej.2 for ; Mon, 13 Apr 2015 02:26:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:subject:message-id:date:to:mime-version; bh=iTdJW8PdBBYmthAmd42fo5u75dRWFyeGcAZMmGoCcfY=; b=scybwaUs4RTJpUlaCqwWE/1GjNE8QLi9axGOwb/g4ceNxdtQ8oudf35LjrHnuJ3M78 qycPViUMQhMlc87B1SDEfOuyIZa7cxBnNigVAh5wjlaJNWhMkAXAseThJJ0MQ6pn3D/j ui2T91heVvp7rxs9SM8cdEiVYfR6tnHRvGPILNA9Zq4TzK94ilT5qgbTFBgVmcq3+wqJ HCbgpLmgsW5BBds84tkyuPRZChWkN1hjqO+TIHnCMy8uF89ZqTctFbE2vYvECLft73ss bDF7FcS/MwUcDMawaE3tzxhAAc7AKomJId6SynfpVxpbpRP2EaZFZPoqUDORLRhap5AB /dCQ== X-Received: by 10.68.68.134 with SMTP id w6mr24946565pbt.25.1428917178797; Mon, 13 Apr 2015 02:26:18 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:95d9:58db:99b9:a8df? ([2601:8:ab80:7d6:95d9:58db:99b9:a8df]) by mx.google.com with ESMTPSA id tg14sm6709802pac.15.2015.04.13.02.26.17 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 13 Apr 2015 02:26:18 -0700 (PDT) From: Garrett Cooper Content-Type: multipart/signed; boundary="Apple-Mail=_ED6AE77C-6742-4CB3-A01A-7A4D6C8A49B0"; protocol="application/pgp-signature"; micalg=pgp-sha512 Subject: Where is gtags? Message-Id: <361064DF-B04A-4F21-983F-08B905296F7C@gmail.com> Date: Mon, 13 Apr 2015 02:26:14 -0700 To: FreeBSD Mailing List Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) X-Mailer: Apple Mail (2.1878.6) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 09:26:19 -0000 --Apple-Mail=_ED6AE77C-6742-4CB3-A01A-7A4D6C8A49B0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 Hi, A number of spots in the base system reference gtags, but I = can=92t seem to find it in the base system or in ports. Was gtags = replaced by ctags, or is it in a more obscure location? Thanks! --Apple-Mail=_ED6AE77C-6742-4CB3-A01A-7A4D6C8A49B0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVK4u2AAoJEMZr5QU6S73eClAIAIY0URhMLqEVQb2K2IlIeUhp U+B9BMUhWTzUOnDEzCS6s+I+B16bPELaA+bVjxBNDmzpnZULWmy3UJDjvYwIEusd VBWfdzNcMAgsIf1qBSpbwa+27uUUhCsfxGBwtZOi8VQYqwC9L7cmCgq2UVsH49Px gLbd1/RDqCUOjP/XWpxe6PqXufp6xZZnYA9lFmeo0IQ9TWhQks19dETpbRjLEgin XaN+jKy8G8vb5ilbamwiNJ404UXV8xekDYKUhlHwXSnHpzFPFvlST+0AFfF0sLJ2 njuwa7UOBXdMfY5Ql9iVYEfko9mD3oenjaeXNa6Z3ZezUFh1pMKwZM5sa1kPI1Q= =zzZv -----END PGP SIGNATURE----- --Apple-Mail=_ED6AE77C-6742-4CB3-A01A-7A4D6C8A49B0-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 09:30:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7CEB5758 for ; Mon, 13 Apr 2015 09:30:14 +0000 (UTC) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F3F8CBA4 for ; Mon, 13 Apr 2015 09:30:13 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1Yhah1-000LTA-Jg; Mon, 13 Apr 2015 12:30:11 +0300 Date: Mon, 13 Apr 2015 12:30:11 +0300 From: Slawa Olhovchenkov To: Garrett Cooper Subject: Re: Where is gtags? Message-ID: <20150413093011.GD18547@zxy.spb.ru> References: <361064DF-B04A-4F21-983F-08B905296F7C@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <361064DF-B04A-4F21-983F-08B905296F7C@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false Cc: FreeBSD Mailing List X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 09:30:14 -0000 On Mon, Apr 13, 2015 at 02:26:14AM -0700, Garrett Cooper wrote: > Hi, > A number of spots in the base system reference gtags, but I > can't seem to find it in the base system or in ports. Was > gtags replaced by ctags, or is it in a more obscure location? devel/global From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 11:25:13 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6FF9D481 for ; Mon, 13 Apr 2015 11:25:13 +0000 (UTC) Received: from mail-vn0-f54.google.com (mail-vn0-f54.google.com [209.85.216.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2E51A9BA for ; Mon, 13 Apr 2015 11:25:12 +0000 (UTC) Received: by vnbf62 with SMTP id f62so18511310vnb.13 for ; Mon, 13 Apr 2015 04:25:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=EIlnAjpozSoh3ejbtFVsp9wUWLzB9nc2eeV+UFPmXY4=; b=YVtEW5MR70WmhNj73gGm5LpmVqyXxFBoQWw+gZUHnXyDZiyhNZmZH/uWSNgnrhFmEJ yM1n+R+z877UJYM4egr0OW2jvbjvrEhBwvlK9BjANryllYHB+MQY6CG2wUhoqsM4uLZ3 2/xR7FaGcYvdupoegaCAa2Wzn3nhhp75qgtUjHbTkhpuN1TIdIbsoDN2WunVIkZ66PaY 7vMbxCNbOFgzEFZaA7CdqGJWAVJ8BDFAU7a2FMffkyFMLKmZQPOxldyybDByxuY9yRjH uBgOPqx0PwmpcCLDmWmZ+62ZFd5SQeUEVX6fSkFy1on+kPEpzxur4UkDzxEDPVY00FuB AVYQ== X-Gm-Message-State: ALoCoQmrWYWaYxM2cFufSclGpD41z0XKhbrfIMkN8/ovGnEU56R2nefUjmE02tP3+RFcfRLZFED6 MIME-Version: 1.0 X-Received: by 10.182.204.6 with SMTP id ku6mr11830730obc.48.1428924305949; Mon, 13 Apr 2015 04:25:05 -0700 (PDT) Received: by 10.202.80.6 with HTTP; Mon, 13 Apr 2015 04:25:05 -0700 (PDT) In-Reply-To: References: Date: Mon, 13 Apr 2015 13:25:05 +0200 Message-ID: Subject: Re: another question - VM mappings From: Oliver Pinter To: Wojciech Puchar Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 11:25:13 -0000 Under amd64 exists an so called shared-page - similar to linux's vdso mechanism - https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_exec.c#L1045 . This page is double mapped with user-space and with kernel. From security reason it's only RO, otherwise, when mapped with RW or RWX, then you could write kernel memory from user-space. On Mon, Apr 13, 2015 at 9:31 AM, Wojciech Puchar wrote: > below is mapping for very simple process (no libc etc) > > [wojtek@laptop ~]$ procstat -v 917 > PID START END PRT RES PRES REF SHD FL TP > PATH > 917 0x400000 0x401000 r-x 1 0 1 0 CN-- vn > /home/wojtek/test/1 > 917 0x600000 0x601000 rw- 1 0 1 0 ---- df > 917 0x7ffffffdf000 0x7ffffffff000 rw- 1 0 1 0 ---D df > 917 0x7ffffffff000 0x800000000000 r-x 0 0 39 0 ---- ph > > > what is "ph" mapping and why read&executable? From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 11:34:04 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F8B66D6 for ; Mon, 13 Apr 2015 11:34:04 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9EDBEAB9 for ; Mon, 13 Apr 2015 11:34:03 +0000 (UTC) Received: by layy10 with SMTP id y10so54693029lay.0 for ; Mon, 13 Apr 2015 04:34:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=iQSyRmRJ72eS3XUY6rCpEK6FVLDpw2WjBbsXpUFw13E=; b=BUGaYkkVXA7PhuXoAmI2tAcBMjSihENwemPdn8TaaTWnkJEo8J9tTYCohm2yuJoAAz UlzyHJ10owoMjoKw4wPk3kiapJ6qdFblFf5c+Geb3oU6ZuQA6sz2kN/e5vjNJsr5Pzpn c5Zp5yj4adI2gVwlUGxIYDbAusCoFQOEIwt9JqLbQLNzCW9ANhJn/vKcIkwVbSR2cgy3 Nx9AEaq0pEuSXP4QgWki6zJG69z3wTKFQueVVzGNU0Mwue48n2sJYSl+c/E+uqvhamDu vURtMRyZdmrDO2eX1Dov0cE6ZOGilkdA30hjigPBwfe/xifd7wCWOMWH3CokPcHmocQN B2bQ== MIME-Version: 1.0 X-Received: by 10.112.98.201 with SMTP id ek9mr12900045lbb.68.1428924841734; Mon, 13 Apr 2015 04:34:01 -0700 (PDT) Received: by 10.152.18.226 with HTTP; Mon, 13 Apr 2015 04:34:01 -0700 (PDT) In-Reply-To: References: Date: Mon, 13 Apr 2015 14:34:01 +0300 Message-ID: Subject: Re: another question - VM mappings From: Kimmo Paasiala To: Oliver Pinter Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Wojciech Puchar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 11:34:04 -0000 On Mon, Apr 13, 2015 at 2:25 PM, Oliver Pinter wrote: > Under amd64 exists an so called shared-page - similar to linux's vdso > mechanism - https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_exec.c#L1045 > . > > This page is double mapped with user-space and with kernel. From > security reason it's only RO, otherwise, when mapped with RW or RWX, > then you could write kernel memory from user-space. > > On Mon, Apr 13, 2015 at 9:31 AM, Wojciech Puchar wrote: >> below is mapping for very simple process (no libc etc) >> >> [wojtek@laptop ~]$ procstat -v 917 >> PID START END PRT RES PRES REF SHD FL TP >> PATH >> 917 0x400000 0x401000 r-x 1 0 1 0 CN-- vn >> /home/wojtek/test/1 >> 917 0x600000 0x601000 rw- 1 0 1 0 ---- df >> 917 0x7ffffffdf000 0x7ffffffff000 rw- 1 0 1 0 ---D df >> 917 0x7ffffffff000 0x800000000000 r-x 0 0 39 0 ---- ph >> >> >> what is "ph" mapping and why read&executable? > _______________________________________________ What is the function is this page, why is it there for every process? -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 13:14:31 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 800E6D80 for ; Mon, 13 Apr 2015 13:14:31 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 13E1585C for ; Mon, 13 Apr 2015 13:14:31 +0000 (UTC) Received: by wgsk9 with SMTP id k9so80312588wgs.3 for ; Mon, 13 Apr 2015 06:14:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-transfer-encoding; bh=p09bFG1d8Q5llbVSZUkmVVa1Fdx8jrYjZG/yJc925QM=; b=0hl9r1OZBbCosy2Q2LD19uUBz0rfZfgADO39lWeLRErG00JFgpv2SgPYKgE7QOnBFQ wQQn/uDTilcJdCZRbFyQSTRilxsls5zUDxURfuLEs7LHdevLPsWk4YxGzg1AfArLiLK2 Ew7/5135HrxoaSSAWn3+lqH1etlR7tGoarAayiUmlkRzGRUEFpLbzLJEljdamrxXML8X m3cayLt1C+BJpSe42horIXJ+KsbuZidywwvgABCL8Apm8iQ7tJ5ls8Bj2tj2MK1g2NRd y6lqEmWX6uf07GzpPgtc/5UE9lO1oYlCycwb/RXqNN1ffUMf0wosAfHCmArOGPtsV6a+ OJ0Q== X-Received: by 10.181.25.225 with SMTP id it1mr21427465wid.8.1428930869433; Mon, 13 Apr 2015 06:14:29 -0700 (PDT) Received: from localhost ([217.14.212.217]) by mx.google.com with ESMTPSA id js3sm11494468wjc.5.2015.04.13.06.14.28 for (version=SSLv3 cipher=RC4-SHA bits=128/128); Mon, 13 Apr 2015 06:14:28 -0700 (PDT) Date: Mon, 13 Apr 2015 15:14:26 +0200 From: To: hackers@freebsd.org Subject: Re: (GSoC 2015) Parallel ports build and installation => paraports Message-ID: <20150413151426.00000262@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 13:14:31 -0000 So, I haven't received a single response to my proposal. Is there something missing or totally wrong with it? Even negative response would be acceptable. I've applied for GSoC as a student and am at step where I should choose organization to which to submit a proposal (but write it first). I would really like to do this project. Domagoj S. From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 13:17:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB42BEA0 for ; Mon, 13 Apr 2015 13:17:14 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F912883 for ; Mon, 13 Apr 2015 13:17:14 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t3DDH9N8027777 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 13 Apr 2015 15:17:10 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t3DDH9XR005554; Mon, 13 Apr 2015 15:17:09 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t3DDH4EG005551; Mon, 13 Apr 2015 15:17:04 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Mon, 13 Apr 2015 15:17:04 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Oliver Pinter Subject: Re: another question - VM mappings In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Mon, 13 Apr 2015 15:17:10 +0200 (CEST) Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 13:17:14 -0000 thanks! On Mon, 13 Apr 2015, Oliver Pinter wrote: > Under amd64 exists an so called shared-page - similar to linux's vdso > mechanism - https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_exec.c#L1045 > . > > This page is double mapped with user-space and with kernel. From > security reason it's only RO, otherwise, when mapped with RW or RWX, > then you could write kernel memory from user-space. > > On Mon, Apr 13, 2015 at 9:31 AM, Wojciech Puchar wrote: >> below is mapping for very simple process (no libc etc) >> >> [wojtek@laptop ~]$ procstat -v 917 >> PID START END PRT RES PRES REF SHD FL TP >> PATH >> 917 0x400000 0x401000 r-x 1 0 1 0 CN-- vn >> /home/wojtek/test/1 >> 917 0x600000 0x601000 rw- 1 0 1 0 ---- df >> 917 0x7ffffffdf000 0x7ffffffff000 rw- 1 0 1 0 ---D df >> 917 0x7ffffffff000 0x800000000000 r-x 0 0 39 0 ---- ph >> >> >> what is "ph" mapping and why read&executable? > > From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 13:18:33 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A05DFF97 for ; Mon, 13 Apr 2015 13:18:33 +0000 (UTC) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 305DA896 for ; Mon, 13 Apr 2015 13:18:33 +0000 (UTC) Received: by wizk4 with SMTP id k4so72068075wiz.1 for ; Mon, 13 Apr 2015 06:18:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=foLvpbPMtNv6ZcXh/FedPV5sBswCHVMwBz/G60xFVVo=; b=me7FQg20CkimpMQYcFVD0i3iZHlABDaq07eVc/fW36g5tx/bqFSiQlluQS6/Dp81Ok pEMe5uO4Iqna+ROUs3VAkcq4rY/mj2WQHPfocPAW6PAu/vlz6xdB5W5BIoRl+pdmYgru Ux5zcdF232cIzAe7elT9KbK+zpQ3rV0/OZ+osE9V70vRmble5Uv8pZNPk+LIaX/hHysk fZo4mmqD4t4GXc9om7qrj2aM17gcxjXmayXx+8no8gMZVDIP3AXsyU8zg6minWPF6vUx 23dInS2vqGhNuAnQOaCwtXrJGxG/0nsma4KHsdKBUp70BKChpXFTA1IZgUkDzM0hJAmw 8pDQ== MIME-Version: 1.0 X-Received: by 10.180.105.136 with SMTP id gm8mr21943515wib.13.1428931111510; Mon, 13 Apr 2015 06:18:31 -0700 (PDT) Received: by 10.180.4.41 with HTTP; Mon, 13 Apr 2015 06:18:31 -0700 (PDT) Received: by 10.180.4.41 with HTTP; Mon, 13 Apr 2015 06:18:31 -0700 (PDT) In-Reply-To: <20150413151426.00000262@gmail.com> References: <20150413151426.00000262@gmail.com> Date: Mon, 13 Apr 2015 15:18:31 +0200 Message-ID: Subject: Re: (GSoC 2015) Parallel ports build and installation => paraports From: =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= To: rank1seeker@gmail.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 13:18:33 -0000 El 13/04/2015 15:14, escribi=C3=B3: > > So, > > I haven't received a single response to my proposal. > Is there something missing or totally wrong with it? > Even negative response would be acceptable. > > I've applied for GSoC as a student and am at step where I should choose > organization to which to submit a proposal (but write it first). > > I would really like to do this project. > Did you send this mail to freebsd-ports@? Maybe you can find some feedback from porters and committers in that list. Cheers > > Domagoj S. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 13 13:30:33 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AE51779F for ; Mon, 13 Apr 2015 13:30:33 +0000 (UTC) Received: from mail-oi0-f43.google.com (mail-oi0-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 763B8A0D for ; Mon, 13 Apr 2015 13:30:33 +0000 (UTC) Received: by oica37 with SMTP id a37so3772321oic.0 for ; Mon, 13 Apr 2015 06:30:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=l98lbAQdIvJ+tTomm4G0b6h/WSyrJ92wj/VeV8J0OZI=; b=RlS/RetTdInuWGKEXvEkqTOkpTI1zQaATxAcsSnWHelABt4PbeViACdBEHjYOU/x5P GI+bbhLMRm9Ja1JCMwNQZqkc5KFs5NlGEJI9rAU7Bel5KxhI5W++1K7F7E2J/h+QUm5v zPdn/wlSjEJ016PwYwapo7xgN7PbXjLtU8mVPAIDXhlTARcnVdh2wq/MBY4Nry5pTxhQ fcoW6O813uyfxEZ9M/p7FSAk0k3+SwdPzTl96oEDrfRP6GurMtUWq4MFjgn6eCYQ86fH I6KMkU4O5lMlUhx8Bhlm0AnHklPMYtmZnKZLm3P86m0+FXDC1Ja0FkPh1ti89fEYjZCI kMrw== X-Gm-Message-State: ALoCoQmsO0M2nrSWA6U47poA08cM3cmP+ewHGTUZlNOy+fuvpAi5Sl4tTThjxNH6kumOhHABjc5P MIME-Version: 1.0 X-Received: by 10.202.194.65 with SMTP id s62mr7477010oif.39.1428931431866; Mon, 13 Apr 2015 06:23:51 -0700 (PDT) Received: by 10.202.80.6 with HTTP; Mon, 13 Apr 2015 06:23:51 -0700 (PDT) In-Reply-To: References: Date: Mon, 13 Apr 2015 15:23:51 +0200 Message-ID: Subject: Re: another question - VM mappings From: Oliver Pinter To: Kimmo Paasiala Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-hackers@freebsd.org" , Wojciech Puchar X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Apr 2015 13:30:33 -0000 On Mon, Apr 13, 2015 at 1:34 PM, Kimmo Paasiala wrote: > On Mon, Apr 13, 2015 at 2:25 PM, Oliver Pinter > wrote: >> Under amd64 exists an so called shared-page - similar to linux's vdso >> mechanism - https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_exec.c#L1045 >> . >> >> This page is double mapped with user-space and with kernel. From >> security reason it's only RO, otherwise, when mapped with RW or RWX, >> then you could write kernel memory from user-space. >> >> On Mon, Apr 13, 2015 at 9:31 AM, Wojciech Puchar wrote: >>> below is mapping for very simple process (no libc etc) >>> >>> [wojtek@laptop ~]$ procstat -v 917 >>> PID START END PRT RES PRES REF SHD FL TP >>> PATH >>> 917 0x400000 0x401000 r-x 1 0 1 0 CN-- vn >>> /home/wojtek/test/1 >>> 917 0x600000 0x601000 rw- 1 0 1 0 ---- df >>> 917 0x7ffffffdf000 0x7ffffffff000 rw- 1 0 1 0 ---D df >>> 917 0x7ffffffff000 0x800000000000 r-x 0 0 39 0 ---- ph >>> >>> >>> what is "ph" mapping and why read&executable? >> _______________________________________________ > > What is the function is this page, why is it there for every process? The rationale behind is to speed up things. With the shared page, the system able to eliminate some syscall overhead especially gettimeofday family of syscalls. The shared-page currently used only to hold time specific informations ( kernel part: https://github.com/freebsd/freebsd/blob/master/sys/kern/kern_sharedpage.c libc part: https://github.com/freebsd/freebsd/blob/master/lib/libc/sys/__vdso_gettimeofday.c ). With these solution the system able to get the current time with a "simple" memory read, rather the a syscall. > > -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 16 15:32:50 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D5FBABC for ; Thu, 16 Apr 2015 15:32:50 +0000 (UTC) Received: from mail-ob0-f170.google.com (mail-ob0-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED534CA5 for ; Thu, 16 Apr 2015 15:32:48 +0000 (UTC) Received: by obbeb7 with SMTP id eb7so45532452obb.3 for ; Thu, 16 Apr 2015 08:32:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=go8u96UaPm8T8degRfLz7sXaUR/1/fDIqZopmdSH27Q=; b=fILDx4lRXc7yoYvhlD+pA61bo1+I6vfKtpMD2WewvvGQx7hzSKjITWRPy9zXTYamjd I2liDpJtCY0IaEJ3B9833PG7RU/UFucTHmgHyy6HhiTQI7TOHloDFOYPHjL9/RHpfzTm KbzPQ2MT1lBn2FTUBP6xGBJJ9Qj0+zI4TjputLArjOvGP6GcRHGgJzgi/IjzoYvKIvtg 2p7oYrxntOal4xj+VcJtFfosr4Z6YgoZfvDy+o137k24qFEwh0QCwnjmCjl3FUdTVQxX 0Rd3n80xSFQqG90aW+6P6I98n+KkyEfWdmz06gHA6YhnB2/Wpn8mmmrCuzX/n/sMbLjV ihAg== X-Gm-Message-State: ALoCoQklHavcbZ9MSU3tnJkcyfZHx8JBp9LR6/SBQVqrLWuZnOLswSul5piFCtF9KsPvbx+tNls3 MIME-Version: 1.0 X-Received: by 10.60.83.233 with SMTP id t9mr26515128oey.73.1429198367711; Thu, 16 Apr 2015 08:32:47 -0700 (PDT) Received: by 10.76.68.68 with HTTP; Thu, 16 Apr 2015 08:32:47 -0700 (PDT) X-Originating-IP: [84.27.222.46] Date: Thu, 16 Apr 2015 17:32:47 +0200 Message-ID: Subject: CloudABI: Taking capability-based security to the next level? From: Ed Schouten To: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Apr 2015 15:32:50 -0000 Hello fellow FreeBSD hackers, If you are planning on attending BSDCan this year, you may have noticed that I am going to give a talk on something mysterious called CloudABI[1]. I thought it would make sense to also announce its availability here before the conference. Before you read the announcement below, I would like to invite you to read a manifesto on capability-based security that I wrote. This document tries to explain the necessity for a system like CloudABI. https://docs.google.com/a/nuxi.nl/document/d/1tW_4CDRuy7HZSkUd6AcDccga_efuIx6ZoyNV9ZLXbJ8/edit # What is CloudABI? CloudABI is an alternative POSIX-like runtime environment that is purely based on the principles behind Capsicum. It can be used to design complex applications that behave correctly in an environment that enforces capability-based security. CloudABI executables can be executed in such a way that the expose as little as possible about the host operating system, making it perfectly suitable as a building block for a safe and secure cluster/cloud computing setup. It could also be used to add support for untrusted plugins and extensions to existing applications (like Google's Native Client, but not tied to a browser). Compared to FreeBSD's binary interface, CloudABI is extremely compact (~60 system calls). The idea behind this is that adding support for CloudABI to existing operating systems should not be hard. An implementation for FreeBSD exists and support for Linux is planned. The intent is that binaries can be executed on multiple operating systems without requiring any recompilation. Support for CloudABI has already been upstreamed to LLVM/Clang and Binutils. It is therefore very easy to build and install a cross compiler for CloudABI. Cross compilation has already been tested to work on Linux, FreeBSD and Mac OS X. CloudABI ships with a C library called cloudlibc. This C library has been designed in such a way that it works reliably in a sandboxed environment. Features that are known to break when using Capsicum on FreeBSD (timezones, locales) still work properly with cloudlibc. cloudlibc has high testing coverage. This high testing coverage will also play a crucial role in ensuring that operating systems implement support for CloudABI consistently. All of CloudABI is and will remain MIT/BSD licensed. The code can be found on GitHub: cloudlibc: https://github.com/NuxiNL/cloudlibc FreeBSD kernel modifications: https://github.com/NuxiNL/freebsd CloudABI has been developed by Nuxi, a company that I founded last year. Nuxi plans on offering commercial support on CloudABI and its components. Interested in hearing how CloudABI can make your product more secure? Please get in touch at info@nuxi.nl to see if there's anything we can do to help out! # Where to go from here? My goal is to present CloudABI at BSDCan and discuss all the fine details with anyone who is interested. Does the idea behind CloudABI sound appealing to you? Can you think of killer use cases? Be sure to talk to me at the conference. If you won't be attending BSDCan this year: no problem! Emails are also appreciated. In my opinion it would make sense to have support for CloudABI integrated into FreeBSD by the time the kernel module becomes more mature. Expect to see more discussions on the mailing lists by the time that happens. In the meantime, be sure to give CloudABI a try and let us know what you think. Instructions on how to obtain a toolchain and patch up your FreeBSD kernel are provided on cloudlibc's GitHub page. We'd love to hear your opinion! Thanks, -- Ed Schouten [1] CloudABI at BSDCan: http://www.bsdcan.org/2015/schedule/events/524.en.html From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 16 20:31:48 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6465DE39; Thu, 16 Apr 2015 20:31:48 +0000 (UTC) Received: from mail-wg0-x22b.google.com (mail-wg0-x22b.google.com [IPv6:2a00:1450:400c:c00::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DC13213B1; Thu, 16 Apr 2015 20:31:47 +0000 (UTC) Received: by wgso17 with SMTP id o17so93538776wgs.1; Thu, 16 Apr 2015 13:31:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=qgFA7qNUXPrLshvGXss7IgcrJa7CArEPqaYrz2nDABk=; b=K4YncPpJ7HbKNP6Uq4arNDFinNqLOKPZfuLXfmP0D479OzoZ9mFOgwo/kGTedSq0Wh GMNhRPE7GVTIKzVQXpO/KBl7qnbCG5EXS2OePxdckkZu+mo93s9/RanXA8XHq2URYekF AA1XO2gU5PZq7qZUt97z8q1eZPwMEB1SHSM05J75e8z2W66myVGtBqE4NWjVMmWZyUEf NlfKTwFaZ4KHNx5J6WAIfD6IAdGcXYq3HHcdMzhlZYEv9pgGi6rlMYFtVufJJp7J+d9X Q+/CmqqJpGXRpffxv532JTcDu1QS0mh0cZdKyY8VD0t4Zkaqq5YzXcWfVMZ/YQqzPeBJ fISw== X-Received: by 10.195.12.48 with SMTP id en16mr49062527wjd.21.1429216306355; Thu, 16 Apr 2015 13:31:46 -0700 (PDT) Received: from [172.29.10.53] (b2b-130-180-35-168.unitymedia.biz. [130.180.35.168]) by mx.google.com with ESMTPSA id qt2sm11812042wjc.7.2015.04.16.13.31.44 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Apr 2015 13:31:45 -0700 (PDT) Message-ID: <55301C30.9010903@gmail.com> Date: Thu, 16 Apr 2015 22:31:44 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Jim Harris CC: Adrian Chadd , Konstantin Belousov , "freebsd-hackers@freebsd.org" , Michael Fuckner , Alan Somers Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <5526EA33.6090004@gmail.com> <5527F554.2030806@gmail.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Apr 2015 20:31:48 -0000 > It's already awesome that Intel has senior engineers working on > FreeBSD driver code! And it would underline Intel's Open-source > commitment and tech leadership if they donated a couple of these > beefy NVMes. > > Intel has agreed to send DC P3700 samples to the FreeBSD Foundation to > put in the cluster for this kind of work - we are working on getting > these through the internal sample distribution process at the moment. This is awesome! Intel rocks. I am spreading the word about this. == OT (slightly): I've done more measurements under Linux, now with all eight cards (md RAID-0 over all 8) under fire from FIO. The results are very good. Here is 1 number (block device level): 2647.4K random 4kB read IOPS @ <800us (95% quantile) The Intel datasheet says 450k IOPS per card, which would be 3600K, so the scaleup is quite good. The random _write_ IOPS I've measured is almost too high to believe (2390.3K) where Intel datasheet suggests 1400k - I am running a longer test overnight now. Let's see if it can be sustained. Anyway: these Intel NVMe's are great tech! We'll now move on testing at filesystem level (XFS) and database level (PostgreSQL). Cheers, /Tobias FIO Ergebnisse 1 NVMe Karte =========================== bvr-sql18:~ # fio --filename /dev/nvme7n1 control.fio random-read-4k: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1 ... random-write-4k: (g=1): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=sync, iodepth=1 ... sequential-read-128k: (g=2): rw=read, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1 ... sequential-write-128k: (g=3): rw=write, bs=128K-128K/128K-128K/128K-128K, ioengine=sync, iodepth=1 ... fio-2.2.6 Starting 256 threads Jobs: 64 (f=64): [_(192),W(64)] [2.4% done] [0KB/1902MB/0KB /s] [0/15.3K/0 iops] [eta 09h:36m:00s] ] random-read-4k: (groupid=0, jobs=64): err= 0: pid=21898: Thu Apr 16 18:56:01 2015 read : io=381205MB, bw=2117.9MB/s, iops=542157, runt=180000msec clat (usec): min=6, max=5568, avg=116.92, stdev=103.46 lat (usec): min=6, max=5568, avg=117.00, stdev=103.47 clat percentiles (usec): | 1.00th=[ 10], 5.00th=[ 14], 10.00th=[ 18], 20.00th=[ 26], | 30.00th=[ 36], 40.00th=[ 54], 50.00th=[ 127], 60.00th=[ 151], | 70.00th=[ 171], 80.00th=[ 193], 90.00th=[ 223], 95.00th=[ 249], | 99.00th=[ 310], 99.50th=[ 346], 99.90th=[ 828], 99.95th=[ 1880], | 99.99th=[ 2864] bw (KB /s): min= 0, max=36016, per=1.56%, avg=33798.82, stdev=1884.10 lat (usec) : 10=0.57%, 20=11.28%, 50=26.41%, 100=5.80%, 250=51.15% lat (usec) : 500=4.64%, 750=0.04%, 1000=0.02% lat (msec) : 2=0.04%, 4=0.05%, 10=0.01% cpu : usr=1.57%, sys=3.76%, ctx=114070395, majf=0, minf=3150 IO depths : 1=116.9%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=97588427/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=1 random-write-4k: (groupid=1, jobs=64): err= 0: pid=21994: Thu Apr 16 18:56:01 2015 write: io=301353MB, bw=1674.2MB/s, iops=428591, runt=180000msec clat (usec): min=7, max=18212, avg=147.90, stdev=202.69 lat (usec): min=8, max=18212, avg=147.99, stdev=202.69 clat percentiles (usec): | 1.00th=[ 14], 5.00th=[ 24], 10.00th=[ 33], 20.00th=[ 52], | 30.00th=[ 76], 40.00th=[ 109], 50.00th=[ 145], 60.00th=[ 173], | 70.00th=[ 195], 80.00th=[ 213], 90.00th=[ 247], 95.00th=[ 290], | 99.00th=[ 454], 99.50th=[ 564], 99.90th=[ 1112], 99.95th=[ 1928], | 99.99th=[13120] bw (KB /s): min= 0, max=29584, per=1.56%, avg=26719.94, stdev=1839.10 lat (usec) : 10=0.03%, 20=3.11%, 50=15.64%, 100=18.69%, 250=53.11% lat (usec) : 500=8.69%, 750=0.52%, 1000=0.10% lat (msec) : 2=0.07%, 4=0.03%, 10=0.01%, 20=0.01% cpu : usr=1.52%, sys=3.20%, ctx=89437032, majf=0, minf=2657 IO depths : 1=115.9%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=0/w=77146473/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=1 sequential-read-128k: (groupid=2, jobs=64): err= 0: pid=22088: Thu Apr 16 18:56:01 2015 read : io=424549MB, bw=2358.6MB/s, iops=18868, runt=180006msec clat (usec): min=213, max=9712, avg=3390.92, stdev=1888.31 lat (usec): min=213, max=9713, avg=3391.02, stdev=1888.31 clat percentiles (usec): | 1.00th=[ 548], 5.00th=[ 764], 10.00th=[ 924], 20.00th=[ 1272], | 30.00th=[ 1832], 40.00th=[ 2608], 50.00th=[ 3376], 60.00th=[ 4128], | 70.00th=[ 4896], 80.00th=[ 5408], 90.00th=[ 5856], 95.00th=[ 6176], | 99.00th=[ 6752], 99.50th=[ 6944], 99.90th=[ 7584], 99.95th=[ 7968], | 99.99th=[ 8640] bw (KB /s): min= 4, max=40031, per=1.56%, avg=37670.97, stdev=2609.28 lat (usec) : 250=0.01%, 500=0.59%, 750=4.10%, 1000=7.60% lat (msec) : 2=20.12%, 4=25.78%, 10=41.81% cpu : usr=0.07%, sys=0.29%, ctx=3980959, majf=0, minf=1920 IO depths : 1=117.2%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=3396390/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=1 sequential-write-128k: (groupid=3, jobs=64): err= 0: pid=22153: Thu Apr 16 18:56:01 2015 write: io=337530MB, bw=1875.9MB/s, iops=15000, runt=180008msec clat (usec): min=52, max=25625, avg=4258.89, stdev=2628.32 lat (usec): min=52, max=25625, avg=4259.05, stdev=2628.31 clat percentiles (usec): | 1.00th=[ 76], 5.00th=[ 314], 10.00th=[ 780], 20.00th=[ 1608], | 30.00th=[ 2576], 40.00th=[ 3440], 50.00th=[ 4256], 60.00th=[ 5024], | 70.00th=[ 5792], 80.00th=[ 6624], 90.00th=[ 7584], 95.00th=[ 8256], | 99.00th=[ 9792], 99.50th=[12352], 99.90th=[18304], 99.95th=[19584], | 99.99th=[21888] bw (KB /s): min= 4, max=32572, per=1.56%, avg=29945.45, stdev=1770.29 lat (usec) : 100=2.65%, 250=1.83%, 500=2.21%, 750=2.89%, 1000=3.12% lat (msec) : 2=11.60%, 4=22.73%, 10=52.03%, 20=0.89%, 50=0.04% cpu : usr=0.25%, sys=0.22%, ctx=3151153, majf=0, minf=0 IO depths : 1=116.7%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=0/w=2700240/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=1 Run status group 0 (all jobs): READ: io=381205MB, aggrb=2117.9MB/s, minb=2117.9MB/s, maxb=2117.9MB/s, mint=180000msec, maxt=180000msec Run status group 1 (all jobs): WRITE: io=301353MB, aggrb=1674.2MB/s, minb=1674.2MB/s, maxb=1674.2MB/s, mint=180000msec, maxt=180000msec Run status group 2 (all jobs): READ: io=424549MB, aggrb=2358.6MB/s, minb=2358.6MB/s, maxb=2358.6MB/s, mint=180006msec, maxt=180006msec Run status group 3 (all jobs): WRITE: io=337530MB, aggrb=1875.9MB/s, minb=1875.9MB/s, maxb=1875.9MB/s, mint=180008msec, maxt=180008msec Disk stats (read/write): nvme7n1: ios=118062872/92590109, merge=0/0, ticks=26066308/26109972, in_queue=55971668, util=100.00% FIO Ergebnisse für RAID-0 über all 8 NVMe Karten ================================================ bvr-sql18:/home/oberstet/scm/scratchbox/freebsd/cruncher # mdadm --detail /dev/md0 /dev/md0: Version : 1.2 Creation Time : Thu Apr 16 19:31:34 2015 Raid Level : raid0 Array Size : 15627067392 (14903.13 GiB 16002.12 GB) Raid Devices : 8 Total Devices : 8 Persistence : Superblock is persistent Update Time : Thu Apr 16 19:31:34 2015 State : clean Active Devices : 8 Working Devices : 8 Failed Devices : 0 Spare Devices : 0 Chunk Size : 256K Name : bvr-sql18:0 (local to host bvr-sql18) UUID : 7280eb72:929d263f:4604a091:3fe38c91 Events : 0 Number Major Minor RaidDevice State 0 259 0 0 active sync /dev/nvme0n1 1 259 1 1 active sync /dev/nvme1n1 2 259 2 2 active sync /dev/nvme2n1 3 259 3 3 active sync /dev/nvme3n1 4 259 4 4 active sync /dev/nvme4n1 5 259 5 5 active sync /dev/nvme5n1 6 259 6 6 active sync /dev/nvme6n1 7 259 7 7 active sync /dev/nvme7n1 bvr-sql18:~ # fio --filename /dev/md0 control2.fio random-read-4k: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32 ... random-write-4k: (g=1): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32 ... sequential-read-128k: (g=2): rw=read, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32 ... sequential-write-128k: (g=3): rw=write, bs=128K-128K/128K-128K/128K-128K, ioengine=libaio, iodepth=32 ... fio-2.2.6 Starting 256 threads random-read-4k: (groupid=0, jobs=64): err= 0: pid=31860: Thu Apr 16 20:22:41 2015 read : io=620514MB, bw=10341MB/s, iops=2647.4K, runt= 60005msec slat (usec): min=1, max=29604K, avg=22.55, stdev=6631.04 clat (usec): min=0, max=37611K, avg=717.16, stdev=34318.22 lat (usec): min=8, max=37611K, avg=739.82, stdev=35080.01 clat percentiles (usec): | 1.00th=[ 318], 5.00th=[ 374], 10.00th=[ 406], 20.00th=[ 442], | 30.00th=[ 466], 40.00th=[ 486], 50.00th=[ 510], 60.00th=[ 532], | 70.00th=[ 556], 80.00th=[ 588], 90.00th=[ 636], 95.00th=[ 692], | 99.00th=[ 1048], 99.50th=[16512], 99.90th=[16768], 99.95th=[24448], | 99.99th=[28544] bw (KB /s): min= 0, max=341456, per=1.78%, avg=188940.35, stdev=43167.55 lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01% lat (usec) : 100=0.01%, 250=0.05%, 500=45.93%, 750=51.16%, 1000=1.81% lat (msec) : 2=0.18%, 4=0.02%, 10=0.04%, 20=0.72%, 50=0.09% lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 2000=0.01%, >=2000=0.01% cpu : usr=3.42%, sys=68.44%, ctx=63458, majf=0, minf=18647 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued : total=r=158851602/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=32 random-write-4k: (groupid=1, jobs=64): err= 0: pid=31935: Thu Apr 16 20:22:41 2015 write: io=560381MB, bw=9336.8MB/s, iops=2390.3K, runt= 60019msec slat (usec): min=1, max=867805, avg=21.12, stdev=515.87 clat (usec): min=0, max=892513, avg=833.67, stdev=5252.13 lat (usec): min=8, max=892527, avg=854.96, stdev=5280.37 clat percentiles (usec): | 1.00th=[ 34], 5.00th=[ 251], 10.00th=[ 414], 20.00th=[ 462], | 30.00th=[ 494], 40.00th=[ 516], 50.00th=[ 532], 60.00th=[ 556], | 70.00th=[ 572], 80.00th=[ 596], 90.00th=[ 660], 95.00th=[ 756], | 99.00th=[16512], 99.50th=[16512], 99.90th=[24448], 99.95th=[29568], | 99.99th=[171008] bw (KB /s): min= 5, max=275816, per=1.59%, avg=151755.38, stdev=43952.51 lat (usec) : 2=0.01%, 4=0.01%, 10=0.03%, 20=0.43%, 50=0.99% lat (usec) : 100=1.12%, 250=2.41%, 500=27.32%, 750=62.61%, 1000=2.59% lat (msec) : 2=0.56%, 4=0.24%, 10=0.18%, 20=1.34%, 50=0.14% lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%, 750=0.01%, 1000=0.01% cpu : usr=5.60%, sys=61.80%, ctx=1940986, majf=0, minf=29148 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued : total=r=0/w=143457578/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=32 sequential-read-128k: (groupid=2, jobs=64): err= 0: pid=32020: Thu Apr 16 20:22:41 2015 read : io=758042MB, bw=12611MB/s, iops=100885, runt= 60111msec slat (usec): min=4, max=9475, avg=25.05, stdev=34.23 clat (usec): min=0, max=1609.2K, avg=20264.21, stdev=88933.36 lat (usec): min=264, max=1609.2K, avg=20289.38, stdev=88933.66 clat percentiles (usec): | 1.00th=[ 442], 5.00th=[ 596], 10.00th=[ 716], 20.00th=[ 924], | 30.00th=[ 1144], 40.00th=[ 1416], 50.00th=[ 1976], 60.00th=[ 4512], | 70.00th=[11328], 80.00th=[21120], 90.00th=[34048], 95.00th=[60672], | 99.00th=[288768], 99.50th=[602112], 99.90th=[1351680], 99.95th=[1449984], | 99.99th=[1548288] bw (KB /s): min=11636, max=1197056, per=1.57%, avg=202645.32, stdev=164421.79 lat (usec) : 2=0.01%, 4=0.01%, 20=0.01%, 50=0.01%, 100=0.01% lat (usec) : 250=0.01%, 500=2.18%, 750=9.42%, 1000=12.10% lat (msec) : 2=26.54%, 4=8.62%, 10=9.60%, 20=10.52%, 50=14.83% lat (msec) : 100=3.39%, 250=1.66%, 500=0.50%, 750=0.16%, 1000=0.15% lat (msec) : 2000=0.32% cpu : usr=0.33%, sys=5.35%, ctx=5444833, majf=0, minf=699080 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued : total=r=6064339/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=32 sequential-write-128k: (groupid=3, jobs=64): err= 0: pid=32084: Thu Apr 16 20:22:41 2015 write: io=841679MB, bw=13993MB/s, iops=111944, runt= 60150msec slat (usec): min=4, max=6855, avg=23.85, stdev=25.17 clat (usec): min=0, max=901229, avg=18237.98, stdev=63454.35 lat (usec): min=53, max=901246, avg=18262.01, stdev=63454.86 clat percentiles (usec): | 1.00th=[ 55], 5.00th=[ 75], 10.00th=[ 98], 20.00th=[ 155], | 30.00th=[ 251], 40.00th=[ 446], 50.00th=[ 852], 60.00th=[ 1656], | 70.00th=[ 4192], 80.00th=[13504], 90.00th=[34560], 95.00th=[86528], | 99.00th=[350208], 99.50th=[432128], 99.90th=[749568], 99.95th=[790528], | 99.99th=[856064] bw (KB /s): min= 2981, max=971776, per=1.57%, avg=224887.01, stdev=155980.09 lat (usec) : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.09% lat (usec) : 100=10.23%, 250=19.57%, 500=11.90%, 750=6.27%, 1000=4.41% lat (msec) : 2=9.95%, 4=7.15%, 10=7.44%, 20=7.82%, 50=7.51% lat (msec) : 100=3.25%, 250=2.55%, 500=1.52%, 750=0.25%, 1000=0.09% cpu : usr=3.89%, sys=5.37%, ctx=5690696, majf=0, minf=336258 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0% issued : total=r=0/w=6733432/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0 latency : target=0, window=0, percentile=100.00%, depth=32 Run status group 0 (all jobs): READ: io=620514MB, aggrb=10341MB/s, minb=10341MB/s, maxb=10341MB/s, mint=60005msec, maxt=60005msec Run status group 1 (all jobs): WRITE: io=560381MB, aggrb=9336.8MB/s, minb=9336.8MB/s, maxb=9336.8MB/s, mint=60019msec, maxt=60019msec Run status group 2 (all jobs): READ: io=758042MB, aggrb=12611MB/s, minb=12611MB/s, maxb=12611MB/s, mint=60111msec, maxt=60111msec Run status group 3 (all jobs): WRITE: io=841679MB, aggrb=13993MB/s, minb=13993MB/s, maxb=13993MB/s, mint=60150msec, maxt=60150msec Disk stats (read/write): md0: ios=164916853/150191138, merge=0/0, ticks=0/0, in_queue=0, util=0.00%, aggrios=20614611/18773876, aggrmerge=0/0, aggrticks=16975295/18070966, aggrin_queue=35215176, aggrutil=99.55% nvme0n1: ios=20613912/18773078, merge=0/0, ticks=85132920/4244536, in_queue=89694924, util=96.14% nvme1n1: ios=20610214/18771214, merge=0/0, ticks=11432236/7672636, in_queue=19248200, util=98.60% nvme2n1: ios=20618968/18778997, merge=0/0, ticks=2527348/3999968, in_queue=6608776, util=96.34% nvme3n1: ios=20616191/18777118, merge=0/0, ticks=8099032/85948268, in_queue=94354244, util=99.55% nvme4n1: ios=20610527/18772284, merge=0/0, ticks=11455980/5902592, in_queue=17468636, util=98.45% nvme5n1: ios=20620340/18777158, merge=0/0, ticks=2224840/3858944, in_queue=6176544, util=98.01% nvme6n1: ios=20615107/18768738, merge=0/0, ticks=5476296/7946824, in_queue=13518332, util=97.72% nvme7n1: ios=20611629/18772423, merge=0/0, ticks=9453712/24993964, in_queue=34651756, util=98.89% bvr-sql18:~ # From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 17 02:50:09 2015 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F37482DA for ; Fri, 17 Apr 2015 02:50:09 +0000 (UTC) Received: from shell1.rawbw.com (shell1.rawbw.com [198.144.192.42]) by mx1.freebsd.org (Postfix) with ESMTP id DE8A2E17 for ; Fri, 17 Apr 2015 02:50:09 +0000 (UTC) Received: from yuri.doctorlan.com (c-50-184-63-128.hsd1.ca.comcast.net [50.184.63.128]) (authenticated bits=0) by shell1.rawbw.com (8.14.9/8.14.9) with ESMTP id t3H2o9ot013486 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 16 Apr 2015 19:50:09 -0700 (PDT) (envelope-from yuri@rawbw.com) X-Authentication-Warning: shell1.rawbw.com: Host c-50-184-63-128.hsd1.ca.comcast.net [50.184.63.128] claimed to be yuri.doctorlan.com Message-ID: <553074DE.4070106@rawbw.com> Date: Thu, 16 Apr 2015 19:50:06 -0700 From: Yuri User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org Subject: Is it possible to check the running kernel signature? Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2015 02:50:10 -0000 I came across this horror story: https://pbs.twimg.com/media/Bd7LUMYCMAAJcqJ.jpg Three letter agencies subverted the BIOS manufacturers to produce BIOSes that were/are able to inject the malicious code right into the FreeBSD kernel during the final BIOS boot stage. This may well be going on with the modern FreeBSD versions. The idea that comes to mind is the ability to verify that the running kernel wasn't tampered with by comparing it with its disk image copy. Same with the kernel modules. Kernel can be verified through the memory mmapped to /dev/mem device. Is this idea feasible, and would it make sense to implement it? Yuri From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 17 04:06:19 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9C1CEF0B for ; Fri, 17 Apr 2015 04:06:19 +0000 (UTC) Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by mx1.freebsd.org (Postfix) with ESMTP id 31FF06A958 for ; Fri, 17 Apr 2015 04:06:18 +0000 (UTC) Received: from ppp121-45-49-75.lns20.adl2.internode.on.net (HELO midget.dons.net.au) ([121.45.49.75]) by ipmail06.adl2.internode.on.net with ESMTP; 17 Apr 2015 13:31:08 +0930 Received: from [10.0.2.26] ([10.0.2.26]) (authenticated bits=0) by midget.dons.net.au (8.15.1/8.14.9) with ESMTPSA id t3H40wV0080406 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 17 Apr 2015 13:31:05 +0930 (CST) (envelope-from darius@dons.net.au) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2098\)) Subject: Re: Is it possible to check the running kernel signature? From: "O'Connor, Daniel" In-Reply-To: <553074DE.4070106@rawbw.com> Date: Fri, 17 Apr 2015 13:30:58 +0930 Cc: freebsd-hackers@FreeBSD.org Content-Transfer-Encoding: quoted-printable Message-Id: <7C5F6DC3-5507-409E-B58A-F9F291D1924A@dons.net.au> References: <553074DE.4070106@rawbw.com> To: Yuri X-Mailer: Apple Mail (2.2098) X-Spam-Score: -2.899 () ALL_TRUSTED,BAYES_00,URIBL_BLOCKED X-Scanned-By: MIMEDefang 2.75 on 10.0.2.1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2015 04:06:19 -0000 > On 17 Apr 2015, at 12:20, Yuri wrote: > The idea that comes to mind is the ability to verify that the running = kernel wasn't tampered with by comparing it with its disk image copy. = Same with the kernel modules. Kernel can be verified through the memory = mmapped to /dev/mem device. > Is this idea feasible, and would it make sense to implement it? If the kernel has been compromised then you can't trust it, since any = userland program has to use the kernel to do its job it is impossible to = validate the kernel because the kernel could just fake up anything it = wants. Also I think when the kernel is loaded it is modified for things like = relocations (although I'm not sure) which would make it tricky to = verify. -- Daniel O'Connor "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 17 08:04:16 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16E74133; Fri, 17 Apr 2015 08:04:16 +0000 (UTC) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C545368A; Fri, 17 Apr 2015 08:04:15 +0000 (UTC) Received: from th-04.cs.huji.ac.il ([132.65.80.125]) by kabab.cs.huji.ac.il with esmtp id 1Yj1Fu-0000b7-5R; Fri, 17 Apr 2015 11:04:06 +0300 From: Daniel Braniss Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: help with coding a loadable kernel module Date: Fri, 17 Apr 2015 11:04:04 +0300 Message-Id: Cc: freeBSD-arm@freebsd.org To: hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) X-Mailer: Apple Mail (2.2070.6) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2015 08:04:16 -0000 Hi, I know I=E2=80=99m embarking on a dangerous trip, but I want to use a = Raspberry Pi and or a BeagleBone to read (and write) RFID cards. Since a driver is needed to use the spibus, I have 2 options while developing: 1- write a conventional driver module means cross compiling, generating a new kernel, making a new image, writing a SD card, etc, etc 2- make it a loadable module cross compile,via NFS load/unload=20 2 seems the logical path, but I=E2=80=99m getting entangled trying to = use the spibus, while with 1 there are several drivers I can use to learn from. So before I give up on option 2, is there some examples/help? I should point out that I have no experience (yet) with arm/spi/fdt and so option 2 will make the trial and error faster :-) thanks danny From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 17 13:24:26 2015 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 30F8C676 for ; Fri, 17 Apr 2015 13:24:26 +0000 (UTC) Received: from trypticon.cs.illinois.edu (trypticon.cs.illinois.edu [128.174.237.181]) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EAAB0FC0 for ; Fri, 17 Apr 2015 13:24:25 +0000 (UTC) Received: from trypticon.cs.illinois.edu (localhost [127.0.0.1]) by trypticon.cs.illinois.edu (8.14.4/8.14.4/Debian-2.1ubuntu2) with ESMTP id t3HDONlc002039; Fri, 17 Apr 2015 08:24:23 -0500 Received: (from dautenh1@localhost) by trypticon.cs.illinois.edu (8.14.4/8.14.4/Submit) id t3HDONUT002038; Fri, 17 Apr 2015 08:24:23 -0500 Date: Fri, 17 Apr 2015 08:24:23 -0500 From: Nathan Dautenhahn To: Yuri Cc: freebsd-hackers@FreeBSD.org Subject: Re: Is it possible to check the running kernel signature? Message-ID: <20150417132423.GA65136@trypticon.cs.illinois.edu> References: <553074DE.4070106@rawbw.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <553074DE.4070106@rawbw.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2015 13:24:26 -0000 On Thu, Apr 16, 2015 at 07:50:06PM -0700, Yuri wrote: > I came across this horror story: > https://pbs.twimg.com/media/Bd7LUMYCMAAJcqJ.jpg > Three letter agencies subverted the BIOS manufacturers to produce > BIOSes that were/are able to inject the malicious code right into > the FreeBSD kernel during the final BIOS boot stage. This may well > be going on with the modern FreeBSD versions. > > The idea that comes to mind is the ability to verify that the > running kernel wasn't tampered with by comparing it with its disk > image copy. Same with the kernel modules. Kernel can be verified > through the memory mmapped to /dev/mem device. The challenge is that the SMM handler operates as firmware, operating at a higher privilege level than the kernel. However, the kernel could do some type of measurement after each invocation of the SMM handler to ensure that all malicious modifications are detected and patched. This does assume that the attacker doesn't interpose on the system in any other way than SMM interrupts (e.g. DMA). If you want to trust the kernel (which might not be that trustworthy where an attacker could inject surreptitious code more easily than BIOS in my opinion) then the kernel can just do a scan. If you don't trust the kernel you could use a thin hypervisor to measure the memory. Although there you have the practical challenges of measurement, keys, etc. I have been considering ideas along the direction of an isolated measurement component as a use case for the nested kernel (nestedkernel.org). Very interesting direction. ::nathan:: > > Is this idea feasible, and would it make sense to implement it? > > Yuri > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 17 14:50:18 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E22C81C9 for ; Fri, 17 Apr 2015 14:50:17 +0000 (UTC) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::103]) by mx1.freebsd.org (Postfix) with ESMTP id 80D13CAF for ; Fri, 17 Apr 2015 14:50:17 +0000 (UTC) Received: from [172.16.1.102] (unknown [172.16.1.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 37B882FB27 for ; Fri, 17 Apr 2015 14:50:16 +0000 (UTC) Message-ID: <55311DA7.8080209@metricspace.net> Date: Fri, 17 Apr 2015 10:50:15 -0400 From: Eric McCorkle User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Re: ZFS support for EFI References: <55189CBA.9040107@metricspace.net> In-Reply-To: <55189CBA.9040107@metricspace.net> Content-Type: multipart/mixed; boundary="------------030702070506070109040607" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Apr 2015 14:50:18 -0000 This is a multi-part message in MIME format. --------------030702070506070109040607 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit I did some work on this last weekend. I've got the zfs-enabled boot1 loading the ZFS uberblock, but it stops somewhere in the vdev_probe code, believing the block to be a log. I've attached a patch if anyone wants to play around with it. Also, if someone with a UFS system could test that the modularization didn't break UFS functionality, that'd be helpful. On 03/29/2015 08:45 PM, Eric McCorkle wrote: > Hi folks, > > I've been messing around off and on for a while with adding ZFS support > to the EFI boot. It's been mostly exploratory and self-contained up to > this point, but I've gotten to a point that warrants some discussion. > > > First, I've converted boot1.c (the EFI boot block) to use an FS module > framework. This facilitates the addition of ZFS, and should also come > in handy if someone wants to add other functionality later (ie. crypto, > netboot, etc.) > > > More importantly, the EFI loader doesn't seem to make use of its > command-line arguments at all. But a ZFS-enabled loader would really > need the ability to take arguments from boot1 (or grub, or whatever > else). On the boot1 side, with ZFS you need to load and parse > /boot/loader.conf (which may cause you to switch pools), then hand off > the information to loader. In the BIOS loader, that's done through a > binary data object that gets passed in. Command-line strings seem like > the most sensible way to do it with EFI. > > Would this be the right way to go, and if so, what ought these > command-line strings look like? > > > Thanks, > Eric > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > --------------030702070506070109040607 Content-Type: text/x-patch; name="zfsefi.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="zfsefi.diff" Index: sys/boot/efi/boot1/Makefile =================================================================== --- sys/boot/efi/boot1/Makefile (revision 281381) +++ sys/boot/efi/boot1/Makefile (working copy) @@ -13,7 +13,7 @@ INTERNALPROG= # architecture-specific loader code -SRCS= boot1.c reloc.c start.S +SRCS= boot1.c reloc.c start.S ufs_module.c zfs_module.c CFLAGS+= -I. CFLAGS+= -I${.CURDIR}/../include @@ -20,6 +20,8 @@ CFLAGS+= -I${.CURDIR}/../include/${MACHINE_CPUARCH} CFLAGS+= -I${.CURDIR}/../../../contrib/dev/acpica/include CFLAGS+= -I${.CURDIR}/../../.. +CFLAGS+= -I${.CURDIR}/../../zfs/ +CFLAGS+= -I${.CURDIR}/../../../cddl/boot/zfs/ # Always add MI sources and REGULAR efi loader bits .PATH: ${.CURDIR}/../loader/arch/${MACHINE_CPUARCH} Index: sys/boot/efi/boot1/boot1.c =================================================================== --- sys/boot/efi/boot1/boot1.c (revision 281381) +++ sys/boot/efi/boot1/boot1.c (working copy) @@ -5,6 +5,8 @@ * All rights reserved. * Copyright (c) 2014 Nathan Whitehorn * All rights reserved. + * Copyright (c) 2014 Eric McCorkle + * All rights reverved. * * Redistribution and use in source and binary forms are freely * permitted provided that the above copyright notice and this @@ -21,7 +23,6 @@ __FBSDID("$FreeBSD$"); #include -#include #include #include @@ -28,6 +29,8 @@ #include #include +#include "boot_module.h" + #define _PATH_LOADER "/boot/loader.efi" #define _PATH_KERNEL "/boot/kernel/kernel" @@ -41,14 +44,20 @@ u_int sp_size; }; +static const boot_module_t* const boot_modules[] = +{ +#ifdef ZFS_EFI_BOOT + &zfs_module, +#endif +#ifdef UFS_EFI_BOOT + &ufs_module +#endif +}; + +#define NUM_BOOT_MODULES (sizeof(boot_modules) / sizeof(boot_module_t*)) + static const char digits[] = "0123456789abcdef"; -static void panic(const char *fmt, ...) __dead2; -static int printf(const char *fmt, ...); -static int putchar(char c, void *arg); -static int vprintf(const char *fmt, va_list ap); -static int vsnprintf(char *str, size_t sz, const char *fmt, va_list ap); - static int __printf(const char *fmt, putc_func_t *putc, void *arg, va_list ap); static int __putc(char c, void *arg); static int __puts(const char *s, putc_func_t *putc, void *arg); @@ -62,9 +71,67 @@ static EFI_SYSTEM_TABLE *systab; static EFI_HANDLE *image; -static void -bcopy(const void *src, void *dst, size_t len) + +void* Malloc(size_t len, const char* file, int line) { + void* out; + printf("Allocating %lu bytes at %s:%d\n", len, file, line); + if (systab->BootServices->AllocatePool(EfiLoaderData, + len, &out) != + EFI_SUCCESS) { + printf("Can't allocate memory pool\n"); + return NULL; + } + printf("Allocated pool at %p\n", out); + return out; +} + +int strncmp(const char *a, const char *b, size_t len) +{ + for (int i = 0; i < len; i++) + if(a[i] == '\0' && b[i] == '\0') { + return 0; + } else if(a[i] < b[i]) { + return -1; + } else if(a[i] > b[i]) { + return 1; + } + + return 0; +} + +char* strdup(const char* s) { + int len; + + for(len = 1; s[len]; len++); + + char* out = malloc(len); + + for(int i = 0; i < len; i++) + out[i] = s[i]; + + return out; +} + +int bcmp(const void *a, const void *b, size_t len) +{ + const char *sa; + const char *sb; + + for (int i = 0; i < len; i++) + if(sa[i] != sb[i]) + return 1; + + return 0; +} + +int memcmp(const void *a, const void *b, size_t len) +{ + return bcmp(a, b, len); +} + +void bcopy(const void *src, void *dst, size_t len) +{ const char *s = src; char *d = dst; @@ -72,23 +139,24 @@ *d++ = *s++; } -static void -memcpy(void *dst, const void *src, size_t len) +void* memcpy(void *dst, const void *src, size_t len) { bcopy(src, dst, len); + return dst; } -static void -bzero(void *b, size_t len) + +void* memset(void *b, int val, size_t len) { char *p = b; while (len-- != 0) - *p++ = 0; + *p++ = val; + + return b; } -static int -strcmp(const char *s1, const char *s2) +int strcmp(const char *s1, const char *s2) { for (; *s1 == *s2 && *s1; s1++, s2++) ; @@ -95,30 +163,50 @@ return ((u_char)*s1 - (u_char)*s2); } +int putchr(char c, void *arg) +{ + CHAR16 buf[2]; + + if (c == '\n') { + buf[0] = '\r'; + buf[1] = 0; + systab->ConOut->OutputString(systab->ConOut, buf); + } + buf[0] = c; + buf[1] = 0; + systab->ConOut->OutputString(systab->ConOut, buf); + return (1); +} + static EFI_GUID BlockIoProtocolGUID = BLOCK_IO_PROTOCOL; static EFI_GUID DevicePathGUID = DEVICE_PATH_PROTOCOL; -static EFI_GUID LoadedImageGUID = LOADED_IMAGE_PROTOCOL; static EFI_GUID ConsoleControlGUID = EFI_CONSOLE_CONTROL_PROTOCOL_GUID; -static EFI_BLOCK_IO *bootdev; -static EFI_DEVICE_PATH *bootdevpath; -static EFI_HANDLE *bootdevhandle; +#define MAX_DEVS 128 -EFI_STATUS efi_main(EFI_HANDLE Ximage, EFI_SYSTEM_TABLE* Xsystab) +void efi_main(EFI_HANDLE Ximage, EFI_SYSTEM_TABLE* Xsystab) { - EFI_HANDLE handles[128]; + EFI_HANDLE handles[MAX_DEVS]; + dev_info_t module_devs[NUM_BOOT_MODULES][MAX_DEVS]; + size_t dev_offsets[NUM_BOOT_MODULES]; EFI_BLOCK_IO *blkio; - UINTN i, nparts = sizeof(handles), cols, rows, max_dim, best_mode; + UINTN nparts = sizeof(handles); EFI_STATUS status; EFI_DEVICE_PATH *devpath; EFI_BOOT_SERVICES *BS; EFI_CONSOLE_CONTROL_PROTOCOL *ConsoleControl = NULL; SIMPLE_TEXT_OUTPUT_INTERFACE *conout = NULL; - char *path = _PATH_LOADER; + // Basic initialization systab = Xsystab; image = Ximage; + for(int i = 0; i < NUM_BOOT_MODULES; i++) + { + dev_offsets[i] = 0; + } + + // Set up the console, so printf works. BS = systab->BootServices; status = BS->LocateProtocol(&ConsoleControlGUID, NULL, (VOID **)&ConsoleControl); @@ -128,10 +216,14 @@ /* * Reset the console and find the best text mode. */ + UINTN max_dim; + UINTN best_mode; + UINTN cols; + UINTN rows; conout = systab->ConOut; conout->Reset(conout, TRUE); max_dim = best_mode = 0; - for (i = 0; ; i++) { + for (int i = 0; ; i++) { status = conout->QueryMode(conout, i, &cols, &rows); if (EFI_ERROR(status)) @@ -141,6 +233,7 @@ best_mode = i; } } + if (max_dim > 0) conout->SetMode(conout, best_mode); conout->EnableCursor(conout, TRUE); @@ -147,206 +240,93 @@ conout->ClearScreen(conout); printf("\n" - ">> FreeBSD EFI boot block\n"); - printf(" Loader path: %s\n", path); + ">> FreeBSD ZFS-enabled EFI boot block\n"); + printf(" Loader path: %s\n\n", _PATH_LOADER); + printf(" Initializing modules:"); + for(int i = 0; i < NUM_BOOT_MODULES; i++) + { + if (NULL != boot_modules[i]) + { + printf(" %s", boot_modules[i]->name); + boot_modules[i]->init(image, systab, BS); + } + } + putchr('\n', NULL); + + // Get all the device handles status = systab->BootServices->LocateHandle(ByProtocol, &BlockIoProtocolGUID, NULL, &nparts, handles); nparts /= sizeof(handles[0]); + //printf(" Scanning %lu device handles\n", nparts); - for (i = 0; i < nparts; i++) { + // Scan all partitions, probing with all modules. + for (int i = 0; i < nparts; i++) { + dev_info_t devinfo; + + // Figure out if we're dealing with an actual partition status = systab->BootServices->HandleProtocol(handles[i], &DevicePathGUID, (void **)&devpath); - if (EFI_ERROR(status)) + if (EFI_ERROR(status)) { + //printf(" Not a device path protocol\n"); continue; + } - while (!IsDevicePathEnd(NextDevicePathNode(devpath))) + while (!IsDevicePathEnd(NextDevicePathNode(devpath))) { + //printf(" Advancing to next device\n"); devpath = NextDevicePathNode(devpath); + } status = systab->BootServices->HandleProtocol(handles[i], &BlockIoProtocolGUID, (void **)&blkio); - if (EFI_ERROR(status)) + if (EFI_ERROR(status)) { + //printf(" Not a block device\n"); continue; + } - if (!blkio->Media->LogicalPartition) + if (!blkio->Media->LogicalPartition) { + //printf(" Logical partition\n"); continue; + } - if (domount(devpath, blkio, 1) >= 0) - break; - } + // Setup devinfo + devinfo.dev = blkio; + devinfo.devpath = devpath; + devinfo.devhandle = handles[i]; - if (i == nparts) - panic("No bootable partition found"); - - bootdevhandle = handles[i]; - load(path); - - panic("Load failed"); - - return EFI_SUCCESS; -} - -static int -dskread(void *buf, u_int64_t lba, int nblk) -{ - EFI_STATUS status; - int size; - - lba = lba / (bootdev->Media->BlockSize / DEV_BSIZE); - size = nblk * DEV_BSIZE; - status = bootdev->ReadBlocks(bootdev, bootdev->Media->MediaId, lba, - size, buf); - - if (EFI_ERROR(status)) - return (-1); - - return (0); -} - -#include "ufsread.c" - -static ssize_t -fsstat(ufs_ino_t inode) -{ -#ifndef UFS2_ONLY - static struct ufs1_dinode dp1; - ufs1_daddr_t addr1; -#endif -#ifndef UFS1_ONLY - static struct ufs2_dinode dp2; -#endif - static struct fs fs; - static ufs_ino_t inomap; - char *blkbuf; - void *indbuf; - size_t n, nb, size, off, vboff; - ufs_lbn_t lbn; - ufs2_daddr_t addr2, vbaddr; - static ufs2_daddr_t blkmap, indmap; - u_int u; - - blkbuf = dmadat->blkbuf; - indbuf = dmadat->indbuf; - if (!dsk_meta) { - inomap = 0; - for (n = 0; sblock_try[n] != -1; n++) { - if (dskread(dmadat->sbbuf, sblock_try[n] / DEV_BSIZE, - SBLOCKSIZE / DEV_BSIZE)) - return -1; - memcpy(&fs, dmadat->sbbuf, sizeof(struct fs)); - if (( -#if defined(UFS1_ONLY) - fs.fs_magic == FS_UFS1_MAGIC -#elif defined(UFS2_ONLY) - (fs.fs_magic == FS_UFS2_MAGIC && - fs.fs_sblockloc == sblock_try[n]) -#else - fs.fs_magic == FS_UFS1_MAGIC || - (fs.fs_magic == FS_UFS2_MAGIC && - fs.fs_sblockloc == sblock_try[n]) -#endif - ) && - fs.fs_bsize <= MAXBSIZE && - fs.fs_bsize >= sizeof(struct fs)) - break; - } - if (sblock_try[n] == -1) { - printf("Not ufs\n"); - return -1; - } - dsk_meta++; - } else - memcpy(&fs, dmadat->sbbuf, sizeof(struct fs)); - if (!inode) - return 0; - if (inomap != inode) { - n = IPERVBLK(&fs); - if (dskread(blkbuf, INO_TO_VBA(&fs, n, inode), DBPERVBLK)) - return -1; - n = INO_TO_VBO(n, inode); -#if defined(UFS1_ONLY) - memcpy(&dp1, (struct ufs1_dinode *)blkbuf + n, - sizeof(struct ufs1_dinode)); -#elif defined(UFS2_ONLY) - memcpy(&dp2, (struct ufs2_dinode *)blkbuf + n, - sizeof(struct ufs2_dinode)); -#else - if (fs.fs_magic == FS_UFS1_MAGIC) - memcpy(&dp1, (struct ufs1_dinode *)blkbuf + n, - sizeof(struct ufs1_dinode)); - else - memcpy(&dp2, (struct ufs2_dinode *)blkbuf + n, - sizeof(struct ufs2_dinode)); -#endif - inomap = inode; - fs_off = 0; - blkmap = indmap = 0; + // Run through each module, see if it can load this partition + for (int j = 0; j < NUM_BOOT_MODULES; j++ ) + { + if (NULL != boot_modules[j] && + boot_modules[j]->probe(devinfo)) + { + // If it can, save it to the device list for + // that module + module_devs[j][dev_offsets[j]++] = devinfo; + } + } } - size = DIP(di_size); - n = size - fs_off; - return (n); -} -static struct dmadat __dmadat; + // Select a partition to boot. We do this by trying each + // module in order. + for (int i = 0; i < NUM_BOOT_MODULES; i++) + { + if (NULL != boot_modules[i]) + { + printf(" Trying to load from %lu %s partitions\n", + dev_offsets[i], boot_modules[i]->name); + boot_modules[i]->load(module_devs[i], dev_offsets[i], + _PATH_LOADER); + printf(" Failed\n"); + } + } -static int -domount(EFI_DEVICE_PATH *device, EFI_BLOCK_IO *blkio, int quiet) -{ - - dmadat = &__dmadat; - bootdev = blkio; - bootdevpath = device; - if (fsread(0, NULL, 0)) { - if (!quiet) - printf("domount: can't read superblock\n"); - return (-1); - } - if (!quiet) - printf("Succesfully mounted UFS filesystem\n"); - return (0); + // If we get here, we're out of luck... + panic("No bootable partitions found!"); } -static void -load(const char *fname) +void panic(const char *fmt, ...) { - ufs_ino_t ino; - EFI_STATUS status; - EFI_HANDLE loaderhandle; - EFI_LOADED_IMAGE *loaded_image; - void *buffer; - size_t bufsize; - - if ((ino = lookup(fname)) == 0) { - printf("File %s not found\n", fname); - return; - } - - bufsize = fsstat(ino); - status = systab->BootServices->AllocatePool(EfiLoaderData, - bufsize, &buffer); - fsread(ino, buffer, bufsize); - - /* XXX: For secure boot, we need our own loader here */ - status = systab->BootServices->LoadImage(TRUE, image, bootdevpath, - buffer, bufsize, &loaderhandle); - if (EFI_ERROR(status)) - printf("LoadImage failed with error %lx\n", status); - - status = systab->BootServices->HandleProtocol(loaderhandle, - &LoadedImageGUID, (VOID**)&loaded_image); - if (EFI_ERROR(status)) - printf("HandleProtocol failed with error %lx\n", status); - - loaded_image->DeviceHandle = bootdevhandle; - - status = systab->BootServices->StartImage(loaderhandle, NULL, NULL); - if (EFI_ERROR(status)) - printf("StartImage failed with error %lx\n", status); -} - -static void -panic(const char *fmt, ...) -{ char buf[128]; va_list ap; @@ -358,50 +338,25 @@ while (1) {} } -static int -printf(const char *fmt, ...) +int printf(const char *fmt, ...) { va_list ap; int ret; - /* Don't annoy the user as we probe for partitions */ - if (strcmp(fmt,"Not ufs\n") == 0) - return 0; va_start(ap, fmt); - ret = vprintf(fmt, ap); + ret = __printf(fmt, putchr, 0, ap); va_end(ap); return (ret); } -static int -putchar(char c, void *arg) +void vprintf(const char *fmt, va_list ap) { - CHAR16 buf[2]; - - if (c == '\n') { - buf[0] = '\r'; - buf[1] = 0; - systab->ConOut->OutputString(systab->ConOut, buf); - } - buf[0] = c; - buf[1] = 0; - systab->ConOut->OutputString(systab->ConOut, buf); - return (1); + __printf(fmt, putchr, 0, ap); } -static int -vprintf(const char *fmt, va_list ap) +int vsnprintf(char *str, size_t sz, const char *fmt, va_list ap) { - int ret; - - ret = __printf(fmt, putchar, 0, ap); - return (ret); -} - -static int -vsnprintf(char *str, size_t sz, const char *fmt, va_list ap) -{ struct sp_data sp; int ret; Index: sys/boot/efi/boot1/boot_module.h =================================================================== --- sys/boot/efi/boot1/boot_module.h (revision 0) +++ sys/boot/efi/boot1/boot_module.h (working copy) @@ -0,0 +1,57 @@ +#ifndef _BOOT_MODULE_H_ +#define _BOOT_MODULE_H_ + +#include + +#include +#include +#include + +#define UFS_EFI_BOOT 1 +#define ZFS_EFI_BOOT 1 + +// EFI device info +typedef struct dev_info_t +{ + EFI_BLOCK_IO *dev; + EFI_DEVICE_PATH *devpath; + EFI_HANDLE *devhandle; +} dev_info_t; + +// A boot loader module. This is a standard interface for filesystem +// modules in the EFI system. +typedef struct boot_module_t +{ + const char* const name; + + // Initialize the module. + void (* const init)(EFI_HANDLE image, + EFI_SYSTEM_TABLE* systab, + EFI_BOOT_SERVICES *bootsrv); + + // Check to see if curr_dev is a device that this module can handle. + bool (* const probe)(dev_info_t dev); + + // Select the best out of a set of devices that probe indicated were + // loadable, and load it. + void (* const load)(const dev_info_t devs[], + size_t ndevs, + const char* loader_path); +} boot_module_t; + +// Standard boot modules +#ifdef UFS_EFI_BOOT +extern const boot_module_t ufs_module; +#endif +#ifdef ZFS_EFI_BOOT +extern const boot_module_t zfs_module; +#endif + +// Functions available to modules +extern int strcmp(const char *s1, const char *s2); +extern void bcopy(const void *src, void *dst, size_t len); +extern void panic(const char *fmt, ...) __dead2; +extern int printf(const char *fmt, ...); +extern int vsnprintf(char *str, size_t sz, const char *fmt, va_list ap); + +#endif Property changes on: sys/boot/efi/boot1/boot_module.h ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: sys/boot/efi/boot1/ufs_module.c =================================================================== --- sys/boot/efi/boot1/ufs_module.c (revision 0) +++ sys/boot/efi/boot1/ufs_module.c (working copy) @@ -0,0 +1,211 @@ +/*- + * Copyright (c) 1998 Robert Nordier + * All rights reserved. + * Copyright (c) 2001 Robert Drehmel + * All rights reserved. + * Copyright (c) 2014 Nathan Whitehorn + * All rights reserved. + * Copyright (c) 2015 Eric McCorkle + * All rights reverved. + * + * Redistribution and use in source and binary forms are freely + * permitted provided that the above copyright notice and this + * paragraph and the following disclaimer are duplicated in all + * such forms. + * + * This software is provided "AS IS" and without any express or + * implied warranties, including, without limitation, the implied + * warranties of merchantability and fitness for a particular + * purpose. + */ +#include +#include + +#include +#include + +#include + +#include "boot_module.h" + +static EFI_HANDLE image; +static EFI_SYSTEM_TABLE* systab; +static EFI_BOOT_SERVICES *bootsrv; +static dev_info_t devinfo; +static EFI_GUID LoadedImageGUID = LOADED_IMAGE_PROTOCOL; + +static int +dskread(void *buf, u_int64_t lba, int nblk) +{ + EFI_STATUS status; + int size; + + lba = lba / (devinfo.dev->Media->BlockSize / DEV_BSIZE); + size = nblk * DEV_BSIZE; + status = devinfo.dev->ReadBlocks(devinfo.dev, + devinfo.dev->Media->MediaId, lba, + size, buf); + + if (EFI_ERROR(status)) + return (-1); + + return (0); +} + +#include "ufsread.c" + +static ssize_t +fsstat(ufs_ino_t inode) +{ +#ifndef UFS2_ONLY + static struct ufs1_dinode dp1; + ufs1_daddr_t addr1; +#endif +#ifndef UFS1_ONLY + static struct ufs2_dinode dp2; +#endif + static struct fs fs; + static ufs_ino_t inomap; + char *blkbuf; + void *indbuf; + size_t n, nb, size, off, vboff; + ufs_lbn_t lbn; + ufs2_daddr_t addr2, vbaddr; + static ufs2_daddr_t blkmap, indmap; + u_int u; + + blkbuf = dmadat->blkbuf; + indbuf = dmadat->indbuf; + if (!dsk_meta) { + inomap = 0; + for (n = 0; sblock_try[n] != -1; n++) { + if (dskread(dmadat->sbbuf, sblock_try[n] / DEV_BSIZE, + SBLOCKSIZE / DEV_BSIZE)) + return -1; + memcpy(&fs, dmadat->sbbuf, sizeof(struct fs)); + if (( +#if defined(UFS1_ONLY) + fs.fs_magic == FS_UFS1_MAGIC +#elif defined(UFS2_ONLY) + (fs.fs_magic == FS_UFS2_MAGIC && + fs.fs_sblockloc == sblock_try[n]) +#else + fs.fs_magic == FS_UFS1_MAGIC || + (fs.fs_magic == FS_UFS2_MAGIC && + fs.fs_sblockloc == sblock_try[n]) +#endif + ) && + fs.fs_bsize <= MAXBSIZE && + fs.fs_bsize >= sizeof(struct fs)) + break; + } + if (sblock_try[n] == -1) { + printf("Not ufs\n"); + return -1; + } + dsk_meta++; + } else + memcpy(&fs, dmadat->sbbuf, sizeof(struct fs)); + if (!inode) + return 0; + if (inomap != inode) { + n = IPERVBLK(&fs); + if (dskread(blkbuf, INO_TO_VBA(&fs, n, inode), DBPERVBLK)) + return -1; + n = INO_TO_VBO(n, inode); +#if defined(UFS1_ONLY) + memcpy(&dp1, (struct ufs1_dinode *)blkbuf + n, + sizeof(struct ufs1_dinode)); +#elif defined(UFS2_ONLY) + memcpy(&dp2, (struct ufs2_dinode *)blkbuf + n, + sizeof(struct ufs2_dinode)); +#else + if (fs.fs_magic == FS_UFS1_MAGIC) + memcpy(&dp1, (struct ufs1_dinode *)blkbuf + n, + sizeof(struct ufs1_dinode)); + else + memcpy(&dp2, (struct ufs2_dinode *)blkbuf + n, + sizeof(struct ufs2_dinode)); +#endif + inomap = inode; + fs_off = 0; + blkmap = indmap = 0; + } + size = DIP(di_size); + n = size - fs_off; + return (n); +} + +static struct dmadat __dmadat; + +static bool +probe(const dev_info_t dev) +{ + devinfo = dev; + dmadat = &__dmadat; + if (fsread(0, NULL, 0)) { + return 0; + } + return 1; +} + +static void +try_load(const dev_info_t dev, + const char* const loader_path) +{ + ufs_ino_t ino; + EFI_STATUS status; + EFI_HANDLE loaderhandle; + EFI_LOADED_IMAGE *loaded_image; + void *buffer; + size_t bufsize; + + devinfo = dev; + if ((ino = lookup(loader_path)) == 0) { + printf("File %s not found\n", loader_path); + return; + } + + bufsize = fsstat(ino); + status = systab->BootServices->AllocatePool(EfiLoaderData, + bufsize, &buffer); + fsread(ino, buffer, bufsize); + + status = systab->BootServices->LoadImage(TRUE, image, devinfo.devpath, + buffer, bufsize, &loaderhandle); + + status = systab->BootServices->HandleProtocol(loaderhandle, + &LoadedImageGUID, (VOID**)&loaded_image); + loaded_image->DeviceHandle = devinfo.devhandle; + + status = systab->BootServices->StartImage(loaderhandle, NULL, NULL); +} + +static void +load(const dev_info_t devs[], + const size_t ndevs, + const char* const loader_path) +{ + for(int i = 0; i < ndevs; i++) + { + try_load(devs[i], loader_path); + } +} + + +static void init(EFI_HANDLE xImage, + EFI_SYSTEM_TABLE* xSystab, + EFI_BOOT_SERVICES * xBootsrv) +{ + image = xImage; + systab = xSystab; + bootsrv = xBootsrv; +} + +const boot_module_t ufs_module = +{ + .name = "UFS", + .init = init, + .probe = probe, + .load = load +}; Property changes on: sys/boot/efi/boot1/ufs_module.c ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property Index: sys/boot/efi/boot1/zfs_module.c =================================================================== --- sys/boot/efi/boot1/zfs_module.c (revision 0) +++ sys/boot/efi/boot1/zfs_module.c (working copy) @@ -0,0 +1,237 @@ +/* Copyright (c) 2015 Eric McCorkle. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * 2. Redistributions in binary form must reproduce the above + * copyright notice, this list of conditions and the following + * disclaimer in the documentation and/or other materials provided + * with the distribution. + * + * 3. Neither the name of the author nor the names of any contributors + * may be used to endorse or promote products derived from this + * software without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHORS AND CONTRIBUTORS ``AS IS'' + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED + * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ +#include +#include +#include + +#include +#include +#include + +#include + +#include "boot_module.h" + +#include "libzfs.h" +#include "zfsimpl.c" + +#define PATH_CONFIG "/boot/config" +#define PATH_DOTCONFIG "/boot/.config" + +static EFI_HANDLE image; +static EFI_SYSTEM_TABLE* systab; +static EFI_BOOT_SERVICES *bootsrv; +static EFI_GUID LoadedImageGUID = LOADED_IMAGE_PROTOCOL; + +static int +vdev_read(vdev_t * const vdev, + void * const priv, + const off_t off, + void * const buf, + const size_t bytes) +{ + const dev_info_t* const devinfo = (const dev_info_t*) priv; + const off_t lba = off / devinfo->dev->Media->BlockSize; + printf(" Request to read %lu bytes at offset %lu, lba %lu to %p\n", + bytes, off, lba, buf); + const EFI_STATUS status = + devinfo->dev->ReadBlocks(devinfo->dev, + devinfo->dev->Media->MediaId, + lba, bytes, buf); + printf(" Read completed, status %lu\n", status); + if (EFI_ERROR(status)) + return (-1); + + return (0); +} + +static bool probe(dev_info_t dev) +{ + printf(" Probing device for ZFS filesystem\n"); + int result = vdev_probe(vdev_read, &dev, NULL); + + if (result) { + printf(" ZFS probe failed\n"); + } + + return result == 0; +} + +static void try_load(const dev_info_t devinfo, + const char* const loader_path) +{ + spa_t *spa; + struct zfsmount zfsmount; + dnode_phys_t dn; + bool autoboot = true; + + printf(" Attempting ZFS mount..."); + // First, try mounting the ZFS volume + if (zfs_spa_init(spa) != 0 || zfs_mount(spa, 0, &zfsmount) != 0) { + printf("failed\n"); + // Mount failed. Don't report this loudly + return; + } + printf("success\n"); + + //vdev_t * const primary_vdev = spa_get_primary_vdev(spa); + + printf(" Attempting lookup of %s...", loader_path); + if (zfs_lookup(&zfsmount, loader_path, &dn) == 0) { + printf("failed\n"); + return; + } + printf("success\n"); + + printf(" Attempting to stat..."); + struct stat st; + if (zfs_dnode_stat(spa, &dn, &st)) { + printf("Unable to get file statistics\n"); + return; + } + printf("success\n"); + + const size_t bufsize = st.st_size; + void* buffer; + EFI_STATUS status; + EFI_HANDLE loaderhandle; + EFI_LOADED_IMAGE *loaded_image; + + printf(" Attempting to allocate pool of %zu bytes...", bufsize); + if (systab->BootServices->AllocatePool(EfiLoaderData, + bufsize, &buffer) != + EFI_SUCCESS) { + printf("Can't allocate loader pool\n"); + return; + } + printf("success\n"); + + printf(" Attempting to read %zu bytes...", bufsize); + if (dnode_read(spa, &dn, 0, buffer, bufsize) < 0) { + printf("Can't read image\n"); + return; + } + printf("success\n"); + + printf(" Attempting to load image..."); + if (systab->BootServices->LoadImage(TRUE, image, devinfo.devpath, + buffer, bufsize, &loaderhandle) != + EFI_SUCCESS) { + printf("Bad loader image\n"); + return; + } + printf("success\n"); + + printf(" Attempting to prepare image for execution..."); + if (systab->BootServices->HandleProtocol(loaderhandle, + &LoadedImageGUID, + (VOID**)&loaded_image) != + EFI_SUCCESS) { + printf("Failed to prepare loader\n"); + return; + } + printf("success\n"); + + loaded_image->DeviceHandle = devinfo.devhandle; + + printf(" Attempting to execute next stage..."); + // XXX Set up command args first + if (systab->BootServices->StartImage(loaderhandle, NULL, NULL) != + EFI_SUCCESS) { + printf("Failed to execute loader\n"); + return; + } + +} + +static int zfs_mount_ds(const char * const dsname, + struct zfsmount * const zfsmount, + spa_t ** const spa) +{ + uint64_t newroot; + spa_t *newspa; + char *q; + + q = strchr(dsname, '/'); + if (q) + *q++ = '\0'; + newspa = spa_find_by_name(dsname); + if (newspa == NULL) { + printf("\nCan't find ZFS pool %s\n", dsname); + return -1; + } + + if (zfs_spa_init(newspa)) + return -1; + + newroot = 0; + if (q) { + if (zfs_lookup_dataset(newspa, q, &newroot)) { + printf("\nCan't find dataset %s in ZFS pool %s\n", + q, newspa->spa_name); + return -1; + } + } + if (zfs_mount(newspa, newroot, zfsmount)) { + printf("\nCan't mount ZFS dataset\n"); + return -1; + } + *spa = newspa; + return (0); +} + +static void load(const dev_info_t devs[], + const size_t ndevs, + const char* const loader_path) +{ + for(int i = 0; i < ndevs; i++) { + try_load(devs[i], loader_path); + } +} + +static void init(EFI_HANDLE xImage, + EFI_SYSTEM_TABLE* xSystab, + EFI_BOOT_SERVICES * xBootsrv) +{ + image = xImage; + systab = xSystab; + bootsrv = xBootsrv; + zfs_init(); +} + +const boot_module_t zfs_module = +{ + .name = "ZFS", + .init = init, + .probe = probe, + .load = load +}; Property changes on: sys/boot/efi/boot1/zfs_module.c ___________________________________________________________________ Added: svn:eol-style ## -0,0 +1 ## +native \ No newline at end of property Added: svn:keywords ## -0,0 +1 ## +FreeBSD=%H \ No newline at end of property Added: svn:mime-type ## -0,0 +1 ## +text/plain \ No newline at end of property --------------030702070506070109040607-- From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 18 03:00:15 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6223D35D for ; Sat, 18 Apr 2015 03:00:15 +0000 (UTC) Received: from mail-pd0-x236.google.com (mail-pd0-x236.google.com [IPv6:2607:f8b0:400e:c02::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 32EA6677 for ; Sat, 18 Apr 2015 03:00:15 +0000 (UTC) Received: by pdea3 with SMTP id a3so146251545pde.3 for ; Fri, 17 Apr 2015 20:00:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=acoJklz7BGCPRwwovLN3XzHwljMlWYBVmTx5gF8Ik0o=; b=0sIrBV96a1mWV0MZmQL9IQtyLwohZ9yErA33i5WV6jHO0nKxfhTHNx9UOQODuvel/z /wp1gDtlIGtG0gHRqKtwncn2jDf+QqHLHK7EXn6xLJiC++6yklZBPYOo+O8vr/QJbB6B oBm/SMds9vZoX8x/gSj4+wdVq6L+cYbyABewh7oFzs21zc7di6xJUdm627XbDmj7of+X d9BzYcGu7y13DlOcheVcjE20j2ERY/OZz1UO0hdbK+xWhjPPL8Zja/eV3TKiqBBsN0XS 3fQqNRuwThrobsGDhtjSoiv3zrdQLChRAg7cnLBBSbxfcb7EPYQxHCClCxV+1F3AgRRC O9zA== MIME-Version: 1.0 X-Received: by 10.70.102.11 with SMTP id fk11mr10384402pdb.144.1429326014587; Fri, 17 Apr 2015 20:00:14 -0700 (PDT) Received: by 10.70.82.68 with HTTP; Fri, 17 Apr 2015 20:00:14 -0700 (PDT) In-Reply-To: References: Date: Sat, 18 Apr 2015 13:00:14 +1000 Message-ID: Subject: Re: CloudABI: Taking capability-based security to the next level? From: Outback Dingo To: Ed Schouten Cc: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Apr 2015 03:00:15 -0000 On Fri, Apr 17, 2015 at 1:32 AM, Ed Schouten wrote: > Hello fellow FreeBSD hackers, > > If you are planning on attending BSDCan this year, you may have > noticed that I am going to give a talk on something mysterious called > CloudABI[1]. I thought it would make sense to also announce its > availability here before the conference. > > Before you read the announcement below, I would like to invite you to > read a manifesto on capability-based security that I wrote. This > document tries to explain the necessity for a system like CloudABI. > > > https://docs.google.com/a/nuxi.nl/document/d/1tW_4CDRuy7HZSkUd6AcDccga_efuIx6ZoyNV9ZLXbJ8/edit > > # What is CloudABI? > > CloudABI is an alternative POSIX-like runtime environment that is > purely based on the principles behind Capsicum. It can be used to > design complex applications that behave correctly in an environment > that enforces capability-based security. CloudABI executables can be > executed in such a way that the expose as little as possible about the > host operating system, making it perfectly suitable as a building > block for a safe and secure cluster/cloud computing setup. It could > also be used to add support for untrusted plugins and extensions to > existing applications (like Google's Native Client, but not tied to a > browser). > > Compared to FreeBSD's binary interface, CloudABI is extremely compact > (~60 system calls). The idea behind this is that adding support for > CloudABI to existing operating systems should not be hard. An > implementation for FreeBSD exists and support for Linux is planned. > The intent is that binaries can be executed on multiple operating > systems without requiring any recompilation. > > Support for CloudABI has already been upstreamed to LLVM/Clang and > Binutils. It is therefore very easy to build and install a cross > compiler for CloudABI. Cross compilation has already been tested to > work on Linux, FreeBSD and Mac OS X. > > CloudABI ships with a C library called cloudlibc. This C library has > been designed in such a way that it works reliably in a sandboxed > environment. Features that are known to break when using Capsicum on > FreeBSD (timezones, locales) still work properly with cloudlibc. > cloudlibc has high testing coverage. This high testing coverage will > also play a crucial role in ensuring that operating systems implement > support for CloudABI consistently. > > All of CloudABI is and will remain MIT/BSD licensed. The code can be > found on GitHub: > > cloudlibc: https://github.com/NuxiNL/cloudlibc > FreeBSD kernel modifications: https://github.com/NuxiNL/freebsd > > CloudABI has been developed by Nuxi, a company that I founded last > year. Nuxi plans on offering commercial support on CloudABI and its > components. Interested in hearing how CloudABI can make your product > more secure? Please get in touch at info@nuxi.nl to see if there's > anything we can do to help out! > > # Where to go from here? > > My goal is to present CloudABI at BSDCan and discuss all the fine > details with anyone who is interested. Does the idea behind CloudABI > sound appealing to you? Can you think of killer use cases? Be sure to > talk to me at the conference. If you won't be attending BSDCan this > year: no problem! Emails are also appreciated. > > In my opinion it would make sense to have support for CloudABI > integrated into FreeBSD by the time the kernel module becomes more > mature. Expect to see more discussions on the mailing lists by the > time that happens. > > In the meantime, be sure to give CloudABI a try and let us know what > you think. Instructions on how to obtain a toolchain and patch up your > FreeBSD kernel are provided on cloudlibc's GitHub page. We'd love to > hear your opinion! > > Thanks, > Looks good but a patch would have probably been better for users looking to backport to say 10.1, or apply to a more recent current and help to track progress. In the meantime, Ill take a look at whats there. > -- > Ed Schouten > > [1] CloudABI at BSDCan: > http://www.bsdcan.org/2015/schedule/events/524.en.html > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 18 09:48:43 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA34DA7 for ; Sat, 18 Apr 2015 09:48:43 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA85A74B for ; Sat, 18 Apr 2015 09:48:43 +0000 (UTC) Received: by pdbnk13 with SMTP id nk13so153317535pdb.0 for ; Sat, 18 Apr 2015 02:48:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Svl85J1dPvw9PGaOlMyfP7ilCaOi8GCDqcABeG1w0zw=; b=rl/XWkZgMHN0izR+unoM2TZNyYwWUyD66VCh0g6BV24Yqe4oTxt5dqRbwk4GE5cpGY 6b3HqGkZBau7oMVBNxa0gA+3b2N+JrU2KC1BJAJQzka6vNYGWustxDmCDphLHP7Z90hu BpZCuzJe7/TLBKTuVvongu26ua4AOFUrbl5vYXg3xPehT5+2Ld/iJdbWoJ3luOepefL5 G1wknDHrNSX0ZVvFV4YmgO10z0D6pR15rMwSOsbesz5OhSlLUolTNfyL68rddVgBh8xh WxCFuRqvBigYN9nOuK/DmOthfVHSL8vfL0/+ru+hE7rOESMzeWQROibvy6fj3tMfYuf9 /I2A== MIME-Version: 1.0 X-Received: by 10.68.69.105 with SMTP id d9mr12323998pbu.144.1429350523157; Sat, 18 Apr 2015 02:48:43 -0700 (PDT) Received: by 10.70.82.68 with HTTP; Sat, 18 Apr 2015 02:48:43 -0700 (PDT) In-Reply-To: References: Date: Sat, 18 Apr 2015 19:48:43 +1000 Message-ID: Subject: Re: CloudABI: Taking capability-based security to the next level? From: Outback Dingo To: Ed Schouten Cc: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Apr 2015 09:48:43 -0000 On Sat, Apr 18, 2015 at 1:00 PM, Outback Dingo wrote: > > On Fri, Apr 17, 2015 at 1:32 AM, Ed Schouten wrote: > >> Hello fellow FreeBSD hackers, >> >> If you are planning on attending BSDCan this year, you may have >> noticed that I am going to give a talk on something mysterious called >> CloudABI[1]. I thought it would make sense to also announce its >> availability here before the conference. >> >> Before you read the announcement below, I would like to invite you to >> read a manifesto on capability-based security that I wrote. This >> document tries to explain the necessity for a system like CloudABI. >> >> >> https://docs.google.com/a/nuxi.nl/document/d/1tW_4CDRuy7HZSkUd6AcDccga_efuIx6ZoyNV9ZLXbJ8/edit >> >> # What is CloudABI? >> >> CloudABI is an alternative POSIX-like runtime environment that is >> purely based on the principles behind Capsicum. It can be used to >> design complex applications that behave correctly in an environment >> that enforces capability-based security. CloudABI executables can be >> executed in such a way that the expose as little as possible about the >> host operating system, making it perfectly suitable as a building >> block for a safe and secure cluster/cloud computing setup. It could >> also be used to add support for untrusted plugins and extensions to >> existing applications (like Google's Native Client, but not tied to a >> browser). >> >> Compared to FreeBSD's binary interface, CloudABI is extremely compact >> (~60 system calls). The idea behind this is that adding support for >> CloudABI to existing operating systems should not be hard. An >> implementation for FreeBSD exists and support for Linux is planned. >> The intent is that binaries can be executed on multiple operating >> systems without requiring any recompilation. >> >> Support for CloudABI has already been upstreamed to LLVM/Clang and >> Binutils. It is therefore very easy to build and install a cross >> compiler for CloudABI. Cross compilation has already been tested to >> work on Linux, FreeBSD and Mac OS X. >> >> CloudABI ships with a C library called cloudlibc. This C library has >> been designed in such a way that it works reliably in a sandboxed >> environment. Features that are known to break when using Capsicum on >> FreeBSD (timezones, locales) still work properly with cloudlibc. >> cloudlibc has high testing coverage. This high testing coverage will >> also play a crucial role in ensuring that operating systems implement >> support for CloudABI consistently. >> >> All of CloudABI is and will remain MIT/BSD licensed. The code can be >> found on GitHub: >> >> cloudlibc: https://github.com/NuxiNL/cloudlibc >> FreeBSD kernel modifications: https://github.com/NuxiNL/freebsd >> >> CloudABI has been developed by Nuxi, a company that I founded last >> year. Nuxi plans on offering commercial support on CloudABI and its >> components. Interested in hearing how CloudABI can make your product >> more secure? Please get in touch at info@nuxi.nl to see if there's >> anything we can do to help out! >> >> # Where to go from here? >> >> My goal is to present CloudABI at BSDCan and discuss all the fine >> details with anyone who is interested. Does the idea behind CloudABI >> sound appealing to you? Can you think of killer use cases? Be sure to >> talk to me at the conference. If you won't be attending BSDCan this >> year: no problem! Emails are also appreciated. >> >> In my opinion it would make sense to have support for CloudABI >> integrated into FreeBSD by the time the kernel module becomes more >> mature. Expect to see more discussions on the mailing lists by the >> time that happens. >> >> In the meantime, be sure to give CloudABI a try and let us know what >> you think. Instructions on how to obtain a toolchain and patch up your >> FreeBSD kernel are provided on cloudlibc's GitHub page. We'd love to >> hear your opinion! >> >> Thanks, >> > > Looks good but a patch would have probably been better for users looking > to backport to say 10.1, or apply to a more recent current and help to > track progress. > In the meantime, Ill take a look at whats there. > > > though in the meantime it appears your modified FreeBSD tree is broken at the kernel level, merging and catching up to HEAD / CURRENT should resolve that. ===> cryptodev (all) cc -O2 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -nostdinc -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/home/dingo/freebsd/sys/GENERIC/opt_global.h -I. -I/usr/home/dingo/freebsd/sys -I/usr/home/dingo/freebsd/s ys/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/obj/usr/home/dingo/freebsd/sys/GENERIC -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestan ding -fwrapv -fstack-protector -gdwarf-2 -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wall -Wredundant-decls -Wnested-externs -Wstrict-proto types -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error-tautological-compare -Wno- error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -mno-aes -mno-avx -std=iso9899:1999 -c /usr/home/dingo/freebsd/sys/modules/cryptodev/../../opencrypto/cryptodev.c -o cryptodev.o /usr/home/dingo/freebsd/sys/modules/cryptodev/../../opencrypto/cryptodev.c:1309:32: error: too few arguments to function call, expected 5, have 4 error = falloc(td, &f, &fd, 0); ~~~~~~ ^ /usr/home/dingo/freebsd/sys/sys/filedesc.h:147:1: note: 'falloc' declared here int falloc(struct thread *td, struct file **resultfp, int *resultfd, ^ 1 error generated. *** Error code 1 Stop. make[4]: stopped in /usr/home/dingo/freebsd/sys/modules/cryptodev *** Error code 1 > -- >> Ed Schouten >> >> [1] CloudABI at BSDCan: >> http://www.bsdcan.org/2015/schedule/events/524.en.html >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >> " >> > > From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 18 20:28:43 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8C6F7E95 for ; Sat, 18 Apr 2015 20:28:43 +0000 (UTC) Received: from mail-ob0-f177.google.com (mail-ob0-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5AF40D43 for ; Sat, 18 Apr 2015 20:28:42 +0000 (UTC) Received: by obfe9 with SMTP id e9so92777132obf.1 for ; Sat, 18 Apr 2015 13:28:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=vrzuP+wFenPDX53/L+aQgeF4IfJ559XN8NLdU1u8l+0=; b=PIsj5VlP6tjG000rJW4Vjmqmp9MwSkjp6UfOxgY6bdoQM2ozfiRLyeEuU7KwA2s3jy +xCKWXCRo+9pgWUMUDYpOkoJ/DfNjzWXVkP7BpbSHvRh60oS38PQ5P9V5DLfYagiL60p 9II4hURjTqkVJfmN0spyOup+Iy6N+I4s5s0ZDvPV8vYmBu/hsFcfoKzaA+R9sZQt0JBr c+WwTDmcZz4vTeYVPZnWpWvfMNmt8WLxGHRRRky/KWT3fOK9t+fFjsmoCx39DPGGs+bu YWXJp5sbsJ/0SwxtFJgw3Tr4encetOh1T1SSwEhnfd7lxnP/1hKa11kpjxc2fU/nmYN5 F1hw== X-Gm-Message-State: ALoCoQnevhmxLszNd7Lo7p0da81dyDF2SjhZMClzzHPxto5ocnXFipJ+q6FpKc3ivCaleBTFAker MIME-Version: 1.0 X-Received: by 10.60.35.42 with SMTP id e10mr8088211oej.26.1429388916093; Sat, 18 Apr 2015 13:28:36 -0700 (PDT) Received: by 10.76.68.68 with HTTP; Sat, 18 Apr 2015 13:28:36 -0700 (PDT) X-Originating-IP: [217.91.38.185] In-Reply-To: References: Date: Sat, 18 Apr 2015 22:28:36 +0200 Message-ID: Subject: Re: CloudABI: Taking capability-based security to the next level? From: Ed Schouten To: Outback Dingo Cc: hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Apr 2015 20:28:43 -0000 Hi there, 2015-04-18 11:48 GMT+02:00 Outback Dingo : > though in the meantime it appears your modified FreeBSD tree is broken at > the kernel level, merging and catching up to HEAD / CURRENT should resolve > that. > > ... > /usr/home/dingo/freebsd/sys/modules/cryptodev/../../opencrypto/cryptodev.c:1309:32: > error: too few arguments to function call, expected 5, have 4 > error = falloc(td, &f, &fd, 0); > ~~~~~~ ^ I had to add an additional argument to falloc(). I thought I updated all of the callers, but it looks like I forgot this in a couple of places. I didn't notice this, because I usually build my development kernels without the modules. This should be fixed now, but I'm running a `make universe' to double-check. Thanks for reporting this! -- Ed Schouten