From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 06:11:46 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DB6FD342 for ; Sun, 14 Apr 2013 06:11:46 +0000 (UTC) (envelope-from justin.muniz@maine.edu) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 68259262 for ; Sun, 14 Apr 2013 06:11:45 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id z13so3625071lbh.41 for ; Sat, 13 Apr 2013 23:11:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=+/FJXXlLjGHXzf6HkUmSCZd5I7/MwsID9mp8W0vhj/Y=; b=DQDMc/FRgbewlSII8H23eS46sQS2b0jKs8I33fJMKjrh8iqSTAOqWXj9n0xtPHiouf 7SVbxxngv+Bt5AojuVeGVTk47NsVOM9U42gHtg5+c6ZrFzVf5SkAEaH+GBPusajPFwiz uYYIJbNtjTbbo0pmZyVLETcwFstu/ipz9sQSNQzBg14IdZXNyU8VcVlpOcm97+ToE9gL Pcccj5RQggIRcK3avb8UXk5FayhKY274CREdX65ERZorfe1dBQt7DjBRQT09yoUclw2v bLHTRjdF8/KG2Wnq4Mcnva5PlsECLY9cQBAsnR7W6Oc8DJMAVJbakBLRAXBbgcSzqdQx FWGQ== MIME-Version: 1.0 X-Received: by 10.112.147.170 with SMTP id tl10mr8026068lbb.100.1365919904721; Sat, 13 Apr 2013 23:11:44 -0700 (PDT) Received: by 10.112.135.229 with HTTP; Sat, 13 Apr 2013 23:11:44 -0700 (PDT) Date: Sun, 14 Apr 2013 02:11:44 -0400 Message-ID: Subject: GSOC: Qt front-end for freebsd-update From: Justin Edward Muniz To: freebsd-hackers@freebsd.org X-Gm-Message-State: ALoCoQmrcCNHexurHZWYD9sztkTbjMg1znR+jP1BylxXeCocr6HgzTR3YI3pj9KPwbmhKasp6xal Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 06:11:46 -0000 I am excited for this year's Google Summer of Code, and I have a project in mind that I am working to propose. I am a CS major and have experience with Qt, C++ and shell scripting. I have been developing on FreeBSD for several years, and I am looking to tackle developing a new Qt front-end for the freebsd-update command. I am thinking a rather straight-forward user interface with more advanced options available (such as selecting a specific server). I am looking for constructive feedback, and also a developer interested in mentoring me this Summer. Thanks everyone for your time, and for your hard work. Justin Muniz From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 11:05:15 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A96938DA; Sun, 14 Apr 2013 11:05:15 +0000 (UTC) (envelope-from utisoft@gmail.com) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) by mx1.freebsd.org (Postfix) with ESMTP id 76B6AA9F; Sun, 14 Apr 2013 11:05:15 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id c11so4856846ieb.15 for ; Sun, 14 Apr 2013 04:05:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=WuahHaH0IvBX4xtGfBqY3EUjwSZKA1jRIy8WfbG1rD8=; b=IEwDQLfyTRE4xWENTnZP1vjLsiKQyvjdni7FCmJbKlG1cvP5jrSj5ivA6GeLbFB45+ 9KBLe1KeZoy9ZM2ydiPF+BR/twA5jePnyIVWl9EpPviFKell+fzM8c8Z/qPqkr1y3P5d HqKYLQY4kvKKaZt9hiuNFK3GteFPAbC1/qGxEKPYijoD5zqtluwnJWb/AUbM3k3jOwFH 4EPmzmd9osep3HLXdgJrc/vxhtM7bf1QOytK1OTSij3YCbmCc3UGNMLw5cflgOLoNpK5 h2HGbbs9Tm0gZ6kPalzVIHTryG6y4oU8f6+RMWIOAPI+RirTRnOZi6y5PtpURW0RsqY9 BSEA== X-Received: by 10.50.60.6 with SMTP id d6mr2991063igr.62.1365937515167; Sun, 14 Apr 2013 04:05:15 -0700 (PDT) MIME-Version: 1.0 Sender: utisoft@gmail.com Received: by 10.64.58.52 with HTTP; Sun, 14 Apr 2013 04:04:45 -0700 (PDT) In-Reply-To: References: From: Chris Rees Date: Sun, 14 Apr 2013 12:04:45 +0100 X-Google-Sender-Auth: Zti4AwW_VO7lJpZKmbQfzmz84NA Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update To: Justin Edward Muniz , Colin Percival Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 11:05:15 -0000 On 14 April 2013 07:11, Justin Edward Muniz wrote: > I am excited for this year's Google Summer of Code, and I have a > project in mind that I am working to propose. > > I am a CS major and have experience with Qt, C++ and shell scripting. > I have been developing on FreeBSD for several years, and I am looking to > tackle developing a new Qt front-end for the freebsd-update command. > > I am thinking a rather straight-forward user interface with more > advanced options available (such as selecting a specific server). I am > looking for constructive feedback, and also a developer interested in > mentoring me this Summer. > > Thanks everyone for your time, and for your hard work. Are you referring to the suggestion below? https://wiki.freebsd.org/IdeasPage#KDE_front-ends_to_the_freebsd-update.288.29utility Colin Percival (CCd) mentions he would be the technical contact for this project, though that was five years ago-- perhaps he'd like to comment. Chris From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 11:59:53 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 29094228 for ; Sun, 14 Apr 2013 11:59:53 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (unknown [IPv6:2001:610:1108:5012::107]) by mx1.freebsd.org (Postfix) with ESMTP id E6734CCD for ; Sun, 14 Apr 2013 11:59:52 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 3F1E41203CD; Sun, 14 Apr 2013 13:59:37 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id C88322848C; Sun, 14 Apr 2013 13:59:30 +0200 (CEST) Date: Sun, 14 Apr 2013 13:59:30 +0200 From: Jilles Tjoelker To: Justin Edward Muniz Subject: Re: GSOC: Qt front-end for freebsd-update Message-ID: <20130414115930.GA37723@stack.nl> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 11:59:53 -0000 On Sun, Apr 14, 2013 at 02:11:44AM -0400, Justin Edward Muniz wrote: > I am excited for this year's Google Summer of Code, and I have a > project in mind that I am working to propose. > I am a CS major and have experience with Qt, C++ and shell scripting. > I have been developing on FreeBSD for several years, and I am looking to > tackle developing a new Qt front-end for the freebsd-update command. > I am thinking a rather straight-forward user interface with more > advanced options available (such as selecting a specific server). I am > looking for constructive feedback, and also a developer interested in > mentoring me this Summer. If you want to work on freebsd-update, I think it will be more useful to improve freebsd-update itself. Some issues are: * It is not particularly robust in the face of errors (such as a full disk). From reading the documentation, you might get the impression that it is better than it actually is. The state for the rollback command is only set up after installation of the update so that command is not useful for backing out a partially installed update. * The upgrade feature takes large amounts of time and network bandwidth. * Less required user interaction would be nice. Think of upgrading many machines. * freebsd-update and pkgng are two upgrade systems. Perhaps freebsd-upgrade should be scrapped altogether in favour of a pkgng-based solution. If you want to write GUIs, perhaps a pkgng GUI is more interesting. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 15:08:59 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2278437D for ; Sun, 14 Apr 2013 15:08:59 +0000 (UTC) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from wojtek.tensor.gdynia.pl (wojtek.tensor.gdynia.pl [188.252.31.196]) by mx1.freebsd.org (Postfix) with ESMTP id A242D24D for ; Sun, 14 Apr 2013 15:08:58 +0000 (UTC) Received: from wojtek.tensor.gdynia.pl (localhost [127.0.0.1]) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5) with ESMTP id r3EF8ejw046101; Sun, 14 Apr 2013 17:08:40 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Received: from localhost (wojtek@localhost) by wojtek.tensor.gdynia.pl (8.14.6/8.14.5/Submit) with ESMTP id r3EF8eue046098; Sun, 14 Apr 2013 17:08:40 +0200 (CEST) (envelope-from wojtek@wojtek.tensor.gdynia.pl) Date: Sun, 14 Apr 2013 17:08:39 +0200 (CEST) From: Wojciech Puchar To: Justin Edward Muniz Subject: Re: GSOC: Qt front-end for freebsd-update In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (wojtek.tensor.gdynia.pl [127.0.0.1]); Sun, 14 Apr 2013 17:08:40 +0200 (CEST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 15:08:59 -0000 > I am a CS major and have experience with Qt, C++ and shell scripting. > I have been developing on FreeBSD for several years, and I am looking to > tackle developing a new Qt front-end for the freebsd-update command. spend your time for something more useful :) From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 15:42:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7E88E7F7 for ; Sun, 14 Apr 2013 15:42:29 +0000 (UTC) (envelope-from justin.muniz@maine.edu) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) by mx1.freebsd.org (Postfix) with ESMTP id 08D5031F for ; Sun, 14 Apr 2013 15:42:28 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id fo12so3651818lab.24 for ; Sun, 14 Apr 2013 08:42:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=Xk+uF+DaTN2GH5ATA2aNYml27yu0P+mLXqCOYx0ybMQ=; b=OABvo+iJIuJATK82Ub9LTReewxbBvYlZVSjEQYjEL8Vagr97HT4fD9behRXeMg1Z7M ColfEV5KaKgmfnHxA/LcGN2QF5zCrDboqlAPQFPNxEIWfNZg/cwa4MUogrzRfY7GRDY1 UbOHDGZDDupiie/Jv3yjV+GC+HAzFtkFKe4ZFVk+x+MsDUhE+7ACsPFLlHD3i19vIdLC 1GwelFGtd6Nv6E5PCeeRLJsIaiShxaTX/FwUlL9BUdG4U8hBReya8uxDvaTFat0ojkoQ +dL0HGTfXdBxlzN9Uo8mmprIkQ4G+ZEGVkvkOKl/pqB5vibxPj16qrcaMK6oJj7VBHWw 1pLQ== MIME-Version: 1.0 X-Received: by 10.112.59.103 with SMTP id y7mr4171468lbq.16.1365954147891; Sun, 14 Apr 2013 08:42:27 -0700 (PDT) Received: by 10.112.135.229 with HTTP; Sun, 14 Apr 2013 08:42:27 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 11:42:27 -0400 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update From: Justin Edward Muniz To: freebsd-hackers@freebsd.org X-Gm-Message-State: ALoCoQnPbqLhUGtwHi3RdFHRZkwITwotr7IjnK+E6vkbpKPFWpEXShNI2shLqtYdrNEvc0JKp3GO Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 15:42:29 -0000 Thank you for your advice! I have already sent an email to Colin, and I did indeed take the idea from that page. Justin Muniz On Sun, Apr 14, 2013 at 7:04 AM, Chris Rees wrote: > On 14 April 2013 07:11, Justin Edward Muniz > wrote: > > I am excited for this year's Google Summer of Code, and I have a > > project in mind that I am working to propose. > > > > I am a CS major and have experience with Qt, C++ and shell > scripting. > > I have been developing on FreeBSD for several years, and I am looking to > > tackle developing a new Qt front-end for the freebsd-update command. > > > > I am thinking a rather straight-forward user interface with more > > advanced options available (such as selecting a specific server). I am > > looking for constructive feedback, and also a developer interested in > > mentoring me this Summer. > > > > Thanks everyone for your time, and for your hard work. > > Are you referring to the suggestion below? > > > https://wiki.freebsd.org/IdeasPage#KDE_front-ends_to_the_freebsd-update.288.29utility > > Colin Percival (CCd) mentions he would be the technical contact for > this project, though that was five years ago-- perhaps he'd like to > comment. > > Chris > From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 15:46:31 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 97850914 for ; Sun, 14 Apr 2013 15:46:31 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pa0-f50.google.com (mail-pa0-f50.google.com [209.85.220.50]) by mx1.freebsd.org (Postfix) with ESMTP id 7268C344 for ; Sun, 14 Apr 2013 15:46:31 +0000 (UTC) Received: by mail-pa0-f50.google.com with SMTP id bg2so2132451pad.23 for ; Sun, 14 Apr 2013 08:46:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=BqnHMiz6Svoqf8mNpK+V2eBG/boK9dCw9bB5HqF/dUg=; b=hb3UrI8SyadPP1YYuFXN7nWwsEMiQnppfQ9sX3oKODlN3IlGLnLFUZX0USr8FV2FCD UORn9/tLnFIJsWzhlXYAa8vZR6aW0XQMNI/U0sCXdoRy0Q0MBED32ljFvqi2F4Wq6ydU nL5Fb1N9Bgf57xNhORpNh58ePS6r8EEbxmxIQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:x-gm-message-state; bh=BqnHMiz6Svoqf8mNpK+V2eBG/boK9dCw9bB5HqF/dUg=; b=oCQV7OFrah49YZVGLGFvjZ8WzKK1N/DUiE4XZXd/3d9a95wcbsSSPhcgxja5xBlKZl tCiNgoxr824Bf80HbOhQiJodA7e/t63fvDSJetoUkIMaI8VRjXIa9R58lnukAdxGAjge 6toiLu9J/H6LtPcefhsqDr228u8q84VmQIYphsHI4PeqB2JXnrheZ4Iskx/1hEXGddOH oNQYp9NSgDOIt4wiX+Q9sus2T4yfli4nZ0xeNeZSa6Dr0cgPRJwqbXLpL21q87NrkbsQ 1J/GNz343yCIfGxqFdPrCrI9NzSf/eaBcSuEkRhTSZQImiV2IvMT5eYkWXiMhXLTj7Zj oZKg== X-Received: by 10.68.229.163 with SMTP id sr3mr24480008pbc.54.1365954385144; Sun, 14 Apr 2013 08:46:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.86.34 with HTTP; Sun, 14 Apr 2013 08:45:55 -0700 (PDT) In-Reply-To: References: From: Eitan Adler Date: Sun, 14 Apr 2013 11:45:55 -0400 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update To: Justin Edward Muniz Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQnbAy4OOfc0vCr2w/oq0Z0IrnR2cfCUm7352vc2mbkABfp5tp4GJwckwsd11tXuCEhOf/cR Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 15:46:31 -0000 On 14 April 2013 11:42, Justin Edward Muniz wrote: > Thank you for your advice! I have already sent an email to Colin, and I did > indeed take the idea from that page. I think GUI front ends to freebsd-update, portsnap, or pkgng would all be useful. One thing I would look into though, is what PC-BSD offers. They may already have similar things. -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 16:02:37 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BB59126D for ; Sun, 14 Apr 2013 16:02:37 +0000 (UTC) (envelope-from justin.muniz@maine.edu) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) by mx1.freebsd.org (Postfix) with ESMTP id 43F85636 for ; Sun, 14 Apr 2013 16:02:36 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id ec20so3743377lab.27 for ; Sun, 14 Apr 2013 09:02:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=EvTB7ahf0DE5UmHykCpe5V7T2k+2j3YgYSsEL4tQwSw=; b=nDoC7BYKSmCfixAQ8IHm/H9GFK+rcIyUASWdzypUA75njpfMxb+uA5FhVhT0dI0I/6 EVAS/RjUG+RE9d4a3h1OR2r0+nBfTx5o4jhEOcbmeJIcNs8QFPfm7bdEwBODgR70bs+i UngUve9POaW12uCJfJkOpMHIJXS/P43qTsWBiAY3PjH6/TW+h41vYpMY/FfH+jfukA63 EJxBoiZdRERzpKh+AoFx2Yr3n4kVJx+iO/48bjPQfs6Bm8rmBKZvXOo8Eghb5JN8eBFw D5USN9qzEFx2a/CfsnKIH/jwJqi1hZfWPQDBWwD3+mF9a0G+2GkarDUf5bQsxPut9PZZ EEqQ== MIME-Version: 1.0 X-Received: by 10.112.138.198 with SMTP id qs6mr8695407lbb.68.1365955356081; Sun, 14 Apr 2013 09:02:36 -0700 (PDT) Received: by 10.112.135.229 with HTTP; Sun, 14 Apr 2013 09:02:35 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 12:02:35 -0400 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update From: Justin Edward Muniz To: Eitan Adler , freebsd-hackers@freebsd.org X-Gm-Message-State: ALoCoQkszurftEy1yFYsecTEEcIXmTnL4HP+IPXEYmnTBut+Mia8aI5VltGjamVFWSBX/7RG/v4p Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 16:02:37 -0000 > > I think GUI front ends to freebsd-update, portsnap, or pkgng would all be > useful. > > One thing I would look into though, is what PC-BSD offers. They may > already have similar things. > > Very interesting, I am checking out the source for PC-BSD's updater to study it. Portsnap and pkgng seem like interesting projects, would it be bizarre to combine functionality into one GUI? I need to do more research on pkgng, I am more familiar with the other commands. From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 16:15:20 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DE1BC61D for ; Sun, 14 Apr 2013 16:15:20 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-we0-x22d.google.com (mail-we0-x22d.google.com [IPv6:2a00:1450:400c:c03::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 77A6A6C7 for ; Sun, 14 Apr 2013 16:15:20 +0000 (UTC) Received: by mail-we0-f173.google.com with SMTP id t57so3038179wey.4 for ; Sun, 14 Apr 2013 09:15:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=MUdjoyPhTvJuArTcYB6j38gi7ge1EprFP92443fTjjU=; b=yGmOHlR6kFs3Mc2VfBbfLsb0qzQo9CSAkjHJs8yk70WzOA5sM6V/+17WvzzFm3z55P rgN2T8P3/czezhriIPKTY18j9VOoFZ2yH2iJCIr9eA8x0rpdAHR3V7VpN19NiIfMHxuE ad9iUmy6GX7+vMEB+EWkx+W+9LtuTvRMdMuExNESFu+S8WWrOu3X6XbjtQQpEgpe5lBr pt7M5uoDg8xp/z4GUBKcI7EqFN4H6r97JHcW4UDbLFdmowm5+gEU7QCzn0i4Tn+l5MdZ fTLhT/ktIopiy8lWNVw4UZMIhQreGlOaIbPnYIHzhKjbYNrizR+KMp4tHwSa0ABvWriD iQiw== MIME-Version: 1.0 X-Received: by 10.194.60.195 with SMTP id j3mr27312123wjr.33.1365956119509; Sun, 14 Apr 2013 09:15:19 -0700 (PDT) Received: by 10.216.139.72 with HTTP; Sun, 14 Apr 2013 09:15:19 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 19:15:19 +0300 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update From: Kimmo Paasiala To: Justin Edward Muniz Content-Type: text/plain; charset=UTF-8 Cc: Eitan Adler , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 16:15:20 -0000 On Sun, Apr 14, 2013 at 7:02 PM, Justin Edward Muniz wrote: >> >> I think GUI front ends to freebsd-update, portsnap, or pkgng would all be >> useful. >> >> One thing I would look into though, is what PC-BSD offers. They may >> already have similar things. >> >> Very interesting, I am checking out the source for PC-BSD's updater to > study it. > Portsnap and pkgng seem like interesting projects, would it be bizarre to > combine > functionality into one GUI? I need to do more research on pkgng, I am more > familiar > with the other commands. Please don't mix the two, they are related but their usages do not really overlap. portsnap(8) only deals with keeping the ports(7) tree and the /usr/ports/INDEX file up to date. PKGNG (like the old pkg_* tools) is mostly concerned with registering built ports as packages or installing pre-built packages in the system. Some functionality of it does use the ports tree but it does not depend on it. I have to also ask, what would a GUI offer that the command line tools do not offer at the moment? -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 16:17:36 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 73F76721 for ; Sun, 14 Apr 2013 16:17:36 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-pb0-f43.google.com (mail-pb0-f43.google.com [209.85.160.43]) by mx1.freebsd.org (Postfix) with ESMTP id 4B4396DD for ; Sun, 14 Apr 2013 16:17:36 +0000 (UTC) Received: by mail-pb0-f43.google.com with SMTP id md4so2133451pbc.30 for ; Sun, 14 Apr 2013 09:17:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=9d4He9fuR/yV5Uhrv8yVLFwGNjzppbqEu3mNndWdESs=; b=Gut5QvR9k8Vfvx371lOnSb9Akp2/DfUJYh0sBisIDgiYU5zGN22csevolSSr9mSpWu SuU6tTL7HYgRLplg7SOrvmUiq5zmn+5BuDD03VXhBS0YVgh6h+YguNY/VtwYL/stGSdD x5RNF8P/8yuF+6xmwfRzFEbYS66Stz4VQpNNI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:x-gm-message-state; bh=9d4He9fuR/yV5Uhrv8yVLFwGNjzppbqEu3mNndWdESs=; b=YpJc1Wo3XFVkRUl6ll+1z9Zdbht2HyG/Xkd9WqeoTLAmj9Nj2eiHXhlgQ5uplclPkh 7818XRFTn7HXG7a50oEJiT0aGMEfA2Wdj2j5zjjySCbZbLDHRWrNYsKeWbo23coW6S47 mHLPUbuwqQRQKtbtIUZtb0G4yM60YUBqTEr7Hw0umCPOVRny94fz1pZTW1ekhP9vFaGw um/9gJgTBkRPkQl2guTbn24zjWH4KX/MTsdFmxfFqbQzwbZc1iTRHx7fIQfqDPTnlwj6 bdBkqeTmcmNW2Kzr0+xDIzUhUqMHCzM48EqjVjwCGyTp5mmMOVEGCvjiVPts7IRSTMeW 6yJQ== X-Received: by 10.66.118.140 with SMTP id km12mr17161095pab.129.1365956255838; Sun, 14 Apr 2013 09:17:35 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.86.34 with HTTP; Sun, 14 Apr 2013 09:17:04 -0700 (PDT) In-Reply-To: References: From: Eitan Adler Date: Sun, 14 Apr 2013 12:17:04 -0400 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update To: Kimmo Paasiala Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQnrEOZC5t/IHVVWR8vyWbXmVDUNrhF7N0AdYxYaCZLsIngpeNvgpE+I5e61ZiyvKLTiRyRK Cc: freebsd-hackers@freebsd.org, Justin Edward Muniz X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 16:17:36 -0000 On 14 April 2013 12:15, Kimmo Paasiala wrote: > I have to also ask, what would a GUI offer that the command line tools > do not offer at the moment? A GUI. -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 16:22:12 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5E805A16 for ; Sun, 14 Apr 2013 16:22:12 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) by mx1.freebsd.org (Postfix) with ESMTP id EBE9B79A for ; Sun, 14 Apr 2013 16:22:11 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hn17so856599wib.16 for ; Sun, 14 Apr 2013 09:22:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=DzyANhOdfUrD7PpGH5+C+18Yrzl1i0Ooeri6IVzM5+I=; b=Ji9hLmWFAsZ1emKOWKpcJF66OlXsaHee+6RCAoydJHK4jkgeLUi7/ycHgElLj0VFYJ HzrqzksVb7rchpj2lEGcRSRAzAo9OSezYtWzqYNb7RWJGr+Ta5Clv577/2Kwb486jErR 6FE6amJhfN1S4nfpImkvFTDuqSSSR1X3EZ8fVzTcPmgD8mu5RByJnZg3h+v314N7+Tb3 TsGwFGmQd+Q8+AnUcgZCfwp2UIyiz/MVIVVfRse+OagY/f558xRMX/cI1LTKACVSkIpw suR2mBuTbJeKWdf3oQ9f1bxid5u0rTg1r3jkJPnImZtkbbhCiZuFNVhLrdQhse/1HQlc ghcw== MIME-Version: 1.0 X-Received: by 10.180.37.73 with SMTP id w9mr3896808wij.32.1365956531139; Sun, 14 Apr 2013 09:22:11 -0700 (PDT) Received: by 10.216.139.72 with HTTP; Sun, 14 Apr 2013 09:22:11 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 19:22:11 +0300 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update From: Kimmo Paasiala To: Eitan Adler Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, Justin Edward Muniz X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 16:22:12 -0000 On Sun, Apr 14, 2013 at 7:17 PM, Eitan Adler wrote: > On 14 April 2013 12:15, Kimmo Paasiala wrote: >> I have to also ask, what would a GUI offer that the command line tools >> do not offer at the moment? > > A GUI. > > That's kind of given :D But does FreeBSD lack a GUI for ports/packages management? I don't see many requests for such things on the FreeBSD forums or on the mailing lists. -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 16:27:34 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 51ED3D0A for ; Sun, 14 Apr 2013 16:27:34 +0000 (UTC) (envelope-from fernando.apesteguia@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) by mx1.freebsd.org (Postfix) with ESMTP id DC63E7E5 for ; Sun, 14 Apr 2013 16:27:33 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id hm11so881614wib.5 for ; Sun, 14 Apr 2013 09:27:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=I4wGZhGure6P7ClvAnyvBr1Gf0rR6ZguIHQ89nlhPxk=; b=giYMYQKHCQTPzU6uyQqb7MC+W+BfDjIwbac6Ni0EIPRf8J/pNH32VysbaAHc3fE4be IfZ1JfweGoEIguubqhFwz80mKhPoQn5cJViRFHC/Fq8U1X9/O5cvq6USarJ0Vco2ztfT KmZx2bLncTagDaE2W+Ur5fspUV1X8iTJC82J2h58/alRpB+jC6avLISxNP1NK3Wixq20 MTU0aLln/XSginbU/ak8cZit2+t/sXZpsfuHniVyDHcZtzW0Ka7SFOeoPOvsXP0NXQ/b jiToZRGCtxQVQu3CM8K7+85INTs/LY7R3Lw2l0ymHeu+gk+qapMI21YuDeeuJToU8vv1 sf4w== MIME-Version: 1.0 X-Received: by 10.180.90.41 with SMTP id bt9mr7350310wib.19.1365956852977; Sun, 14 Apr 2013 09:27:32 -0700 (PDT) Received: by 10.180.125.77 with HTTP; Sun, 14 Apr 2013 09:27:32 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 18:27:32 +0200 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update From: =?ISO-8859-1?Q?Fernando_Apestegu=EDa?= To: Kimmo Paasiala Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Eitan Adler , Justin Edward Muniz , freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 16:27:34 -0000 On Sun, Apr 14, 2013 at 6:22 PM, Kimmo Paasiala wrote: > On Sun, Apr 14, 2013 at 7:17 PM, Eitan Adler wrote: > > On 14 April 2013 12:15, Kimmo Paasiala wrote: > >> I have to also ask, what would a GUI offer that the command line tools > >> do not offer at the moment? > > > > A GUI. > > > > > > That's kind of given :D > > But does FreeBSD lack a GUI for ports/packages management? I don't see > many requests for such things on the FreeBSD forums or on the mailing > lists. > It seems we already have something similar in the ports[1] collection. There is also a newer version[2] using Qt4 but it seems more limited. It might be worth a look at those first. [1] ports-mgmt/kports [2] ports-mgmt/kports-qt4 > > -Kimmo > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 16:41:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 27ECCF7 for ; Sun, 14 Apr 2013 16:41:29 +0000 (UTC) (envelope-from justin.muniz@maine.edu) Received: from mail-lb0-f173.google.com (mail-lb0-f173.google.com [209.85.217.173]) by mx1.freebsd.org (Postfix) with ESMTP id A3EE2881 for ; Sun, 14 Apr 2013 16:41:28 +0000 (UTC) Received: by mail-lb0-f173.google.com with SMTP id w20so3865236lbh.4 for ; Sun, 14 Apr 2013 09:41:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=nvJq/kkC8mk5yPHfHocim7qdG4z/WFJl2CBQiMPfYRg=; b=Tnjxw3IIQnmgYZQbzroVwEWFCd3ptT5DFkPJMn/KYFLHzCI/O03lvIMPx0oenN9Qal MoG36Vm4ynQmO3Fd8crtvCllTQhJhYV97CV1+yvuj8IfsqG08LBQ8HLUIFSTwd5II8UI eX1Ts3Ldc78OrG34HiHV4F5n0Q5Vt0h/FK9d9AnMhFkO+e9fJ+2C2nRXhgWBG70t0Qyn n5N+vQnksuHLWJLj/TI3RHgDmagcJNgGOez4NPa0vv/vjLp4CXcJtm0c2yqi3Boo+DIw yZ+IFCd2TUgeNpi0B7T9uxXx8ltPsGqYjjPK3Nd7FK2v7CdcRAvipSa/pmBf71PmtXf7 OrxQ== MIME-Version: 1.0 X-Received: by 10.112.59.103 with SMTP id y7mr4241556lbq.16.1365957681171; Sun, 14 Apr 2013 09:41:21 -0700 (PDT) Received: by 10.112.135.229 with HTTP; Sun, 14 Apr 2013 09:41:21 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 12:41:21 -0400 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update From: Justin Edward Muniz To: Kimmo Paasiala , freebsd-hackers@freebsd.org X-Gm-Message-State: ALoCoQlOpfv1oCCfYOpOtBsOlGMPpGpkf8dE8wRC/+6REUq3I31A43uTIUDM7WGyDx48rK9eyfVv Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 16:41:29 -0000 > > Please don't mix the two, they are related but their usages do not really > overlap. > > portsnap(8) only deals with keeping the ports(7) tree and the > /usr/ports/INDEX file up to date. > > PKGNG (like the old pkg_* tools) is mostly concerned with registering > built ports as packages or installing pre-built packages in the > system. Some functionality of it does use the ports tree but it does > not depend on it. Yes, that does make sense; thank you for your input. A set of GUIs seems like the way to go, the reason why I considered combining them is for convenience. I am still learning about pkgng, but I do understand that it does not replace ports package management tools. A front-end could certainly add or combine functionality for convenience, but some admins run multiple operating systems and may have to reference notes before updates. And some newer users might prefer a GUI for some of these common commands. Justin Muniz From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 16:50:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 679D857F for ; Sun, 14 Apr 2013 16:50:48 +0000 (UTC) (envelope-from justin.muniz@maine.edu) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) by mx1.freebsd.org (Postfix) with ESMTP id E0FBB8FA for ; Sun, 14 Apr 2013 16:50:47 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id u10so3906932lbi.3 for ; Sun, 14 Apr 2013 09:50:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type:x-gm-message-state; bh=sGVbioRMcIWjzeUFXBzo8fMpR/mrmSzhxYrrfeUjcYw=; b=Zrt7HcBxI3izIdjk2xw/mjEReOrPpUW3y45NLvpO5e0vpBWVPyskB1NzmGws/R7oo4 mrEmZhL7hv1PPAngztXbfE3+YcqDguD1s4WmtgNCgZBr8Qki3psNywrfsbwAZ3WRgnGE a9AewL9GgkaPytkba2P5qM8UV1BJt48plkTDujZSPxnH60kG4UOuN/fwfEm99Tv/Ieiv YnOMCR74UcWZ8o82gjJ+DrixGITzSeh9cA04KFsujwjYh2hhW+RuRUlV4xcgNsrN+h43 fvlMQAriV6v5ZFhelGENpKJeCSrEL5N4E2eHWwcSUZWYi/9gHuLZo0At51rx1inKwz89 B8NA== MIME-Version: 1.0 X-Received: by 10.112.125.167 with SMTP id mr7mr8608221lbb.19.1365958246214; Sun, 14 Apr 2013 09:50:46 -0700 (PDT) Received: by 10.112.135.229 with HTTP; Sun, 14 Apr 2013 09:50:46 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 12:50:46 -0400 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update From: Justin Edward Muniz To: =?ISO-8859-1?Q?Fernando_Apestegu=EDa?= , freebsd-hackers@freebsd.org X-Gm-Message-State: ALoCoQkNvRPB8AFkt953z8lQ7xarBF4BABvCK0Keh5GF/UMc/pExeLm1+RukOgaff5sRY8VUKioN Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 16:50:48 -0000 > > It seems we already have something similar in the ports[1] collection. > There is also a newer version[2] using Qt4 but it seems more limited. It > might be worth a look at those first. > > [1] ports-mgmt/kports > [2] ports-mgmt/kports-qt4 > > Yes, I just found those GUI programs myself. No sense reinventing the wheel; I will check out the functionality and make sure I don't overlap. I am most interested in a GUI front-end for updating binary updates to the base system. Thank you for taking your time to help! I will keep researching current GUI projects and make sure this is worthwhile. Everyone has given me much to work with, I appreciate it all. Justin Muniz From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 16:54:30 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 74D1179F for ; Sun, 14 Apr 2013 16:54:30 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 0C02F92A for ; Sun, 14 Apr 2013 16:54:29 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id l13so293903wie.12 for ; Sun, 14 Apr 2013 09:54:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=R6XdPkEFwIqyIDxC6/wkhiXxna5DO3JNeDjwhRZ3V7E=; b=wYzDTMfhXYsegS/etFxTPegCpXyKYT9S7TibVbdDT/F6eE4g4Ch6jKG30ocEQ3EOqC YX2fCEiiQKPkqpaE542l/E7tnRrxT+ZxKCjr7PEKv1H5tkt57FoMWzLirMlkQ8C2qbwk JINaMLJ+tEqSceuQPGqKFx+uQiuXOb/UKE2aWk/9R34YPDVPA8b9x6H+C7cXQDcLKAe4 WMe/mG3cYBRrFy364xJrZPz18FETvM/vLcI6XFyuf5+YpyG3hf3FfwLCMeeby0/O3Wmx 1r5iQveZoeILPxCl0CohXHWNlA0mCjqpxKoVgDDZgddAHTHNmi99Wc0aTUFQopzqNJBn TP7g== MIME-Version: 1.0 X-Received: by 10.180.97.233 with SMTP id ed9mr7317850wib.32.1365958469277; Sun, 14 Apr 2013 09:54:29 -0700 (PDT) Received: by 10.216.139.72 with HTTP; Sun, 14 Apr 2013 09:54:29 -0700 (PDT) In-Reply-To: References: Date: Sun, 14 Apr 2013 19:54:29 +0300 Message-ID: Subject: Re: GSOC: Qt front-end for freebsd-update From: Kimmo Paasiala To: Justin Edward Muniz Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, =?UTF-8?Q?Fernando_Apestegu=C3=ADa?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 16:54:30 -0000 On Sun, Apr 14, 2013 at 7:50 PM, Justin Edward Muniz wrote: >> >> It seems we already have something similar in the ports[1] collection. >> There is also a newer version[2] using Qt4 but it seems more limited. It >> might be worth a look at those first. >> >> [1] ports-mgmt/kports >> [2] ports-mgmt/kports-qt4 >> >> Yes, I just found those GUI programs myself. No sense reinventing the > wheel; > I will check out the functionality and make sure I don't overlap. I am most > interested > in a GUI front-end for updating binary updates to the base system. Thank > you for taking > your time to help! I will keep researching current GUI projects and make > sure this is > worthwhile. > > Everyone has given me much to work with, I appreciate it all. > > Justin Muniz > Could you also take a look at how security/pinentry-* ports offer multiple ways to implement a GUI for a simple service, it offers a curses, QT and GTK frontends to pinentry. -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 14 20:04:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EC6A831C for ; Sun, 14 Apr 2013 20:04:29 +0000 (UTC) (envelope-from opensrcsan3523@gmail.com) Received: from mail-pd0-f196.google.com (mail-pd0-f196.google.com [209.85.192.196]) by mx1.freebsd.org (Postfix) with ESMTP id CED52FC6 for ; Sun, 14 Apr 2013 20:04:29 +0000 (UTC) Received: by mail-pd0-f196.google.com with SMTP id 10so1258294pdc.7 for ; Sun, 14 Apr 2013 13:04:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=2y4PoUSuYdMaYsNH/bErA6MSDu78seBH1RMb31ae4i0=; b=IBLudCGTe2WCkbuW/oXDLd1xiCejzvKB4g4ozsKj0F5f/39rUZF/lAwTou+ykuoMDm mjjFqdTUa4oE/j0oIUnDCM6+352jrdUN0ZO6/8NKEAhc9NYKJSFM3cTivA4LBAKg3Yl7 MhLg9axMdkP8OiDHsP0+Kj1kTALLiZAwxYXUiPR/KvHzdYdZaLcjKTZmj5/hxLWHhI48 bnlmp0ULBjDmgkgGcDkDs1bQoX1yT4WF6RDRXZIaHexwQlmGIJqaOIUa8J1TgP5txCKJ L+Tv3yv3jkZLnbJxbZJ0Q/9nG6BqGa1OYxdCDmqrGyXet55j+ta9+hN2YhfuMxDSkLBe lwuw== MIME-Version: 1.0 X-Received: by 10.68.180.194 with SMTP id dq2mr25184603pbc.175.1365969869293; Sun, 14 Apr 2013 13:04:29 -0700 (PDT) Received: by 10.66.157.162 with HTTP; Sun, 14 Apr 2013 13:04:29 -0700 (PDT) Date: Mon, 15 Apr 2013 01:34:29 +0530 Message-ID: Subject: GSOC and Contribution to open source. From: santosh hosamani To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Apr 2013 20:04:30 -0000 Hi , I am interested in participating for GSOC -2013 . I am interested in following ideas CPU online/offline project BHyVe BIOS emulation to boot legacy systems I have two years of work exp in linux kernel device driver,C now I have selected master thesis as virtualization so I would be interested in contributing to this field . and I have no prior experience in open source so I would be really thankful if anyone could guide me how to start contributing to the world of open source. Regards, santosh. From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 10:40:02 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9476844F for ; Mon, 15 Apr 2013 10:40:02 +0000 (UTC) (envelope-from hps@bitfrost.no) Received: from mta.bitpro.no (mta.bitpro.no [92.42.64.202]) by mx1.freebsd.org (Postfix) with ESMTP id 5B489297 for ; Mon, 15 Apr 2013 10:40:02 +0000 (UTC) Received: from mail.bitfrost.no (mail.bitfrost.no [46.29.221.36]) by mta.bitpro.no (Postfix) with ESMTP id D50607A0FA; Mon, 15 Apr 2013 12:39:54 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at bitfrost.no Received: from laptop015.hselasky.homeunix.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: hanspetter) by mail.bitfrost.no (Postfix) with ESMTPSA id BCC0D2057B; Mon, 15 Apr 2013 12:39:49 +0200 (CEST) Message-ID: <516BD94F.7030507@bitfrost.no> Date: Mon, 15 Apr 2013 12:41:19 +0200 From: Hans Petter Selasky Organization: Bitfrost A/S MIME-Version: 1.0 To: Joshua Isom Subject: Re: USB Printer quirks References: <5169DFFB.4030101@gmail.com> In-Reply-To: <5169DFFB.4030101@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 10:40:02 -0000 On 04/14/13 00:45, Joshua Isom wrote: > I've got my printer to finally work, now that I got around to hooking it > up again and trying. It's a Kodak AiO, and there's a driver that's been > around for a few years and works under Linux. It doesn't work properly > under FreeBSD, or a Debian/kFreeBSD jail, but does work under a Linux > jail. With FreeBSD, the best I'd get is one document and then need to > power cycle the printer to get it to do anything. From what I remember > of the logs, the communication wasn't coming back properly. Cups is > reporting it's not getting bidirectional communication, but it is mostly > printing properly. > > Since it works under the Linux jail, and not the Debian jail, I'm > assuming it has to be a FreeBSD kernel issue that's masked by the Linux > overhead. But where would I begin to troubleshoot the underlying issue? > Right now I'm running a -CURRENT kernel: > > FreeBSD jri.homeunix.com 10.0-CURRENT FreeBSD 10.0-CURRENT #15 r249169M: > Fri Apr 5 16:12:04 CDT 2013 > root@jri.homeunix.com:/usr/obj/root/ATH/head/sys/ATH amd64 > > Any help would be appreciated. Hi, What does dmesg say about your printer. Is cups hooked up the correct /dev/uxxx device ? --HPS From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 17:10:30 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BA1ED678; Mon, 15 Apr 2013 17:10:30 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from mail.ipfw.ru (unknown [IPv6:2a01:4f8:120:6141::2]) by mx1.freebsd.org (Postfix) with ESMTP id 61546EEC; Mon, 15 Apr 2013 17:10:30 +0000 (UTC) Received: from dhcp170-36-red.yandex.net ([95.108.170.36]) by mail.ipfw.ru with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.76 (FreeBSD)) (envelope-from ) id 1URmyb-000DO2-RQ; Mon, 15 Apr 2013 21:13:58 +0400 Message-ID: <516C3462.30501@FreeBSD.org> Date: Mon, 15 Apr 2013 21:09:54 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: "freebsd-net@freebsd.org" , hackers@freebsd.org Subject: VLANHWFILTER "upgrade" X-Enigmail-Version: 1.4.6 Content-Type: multipart/mixed; boundary="------------000509050108030808090809" Cc: Jack Vogel X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 17:10:30 -0000 This is a multi-part message in MIME format. --------------000509050108030808090809 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Hello list. We currently have VLAHWFILTER functionality allowing underlying physical/virtual interfaces to be aware of vlans stacked on them. However, this knowledge is only used to program NIC hw filter (or to broadcast to member ifaces in lagg case). Proposed idea is to save vlan ifp pointer inside the driver and push packet to given vlan directly. This changes removes 1 read lock on RX fast path. Additionally, we can do the same in more popular case of ix -> lagg [ -> lagg -> lagg ] -> vlan if we solve 1) lagg interface counters issue (trivial) 2) IFF_MONITOR on lagg interface issue (not so trivial, unfortunately). Patch to ixgbe driver attached (maybe it is better to put ixgbe_vlan_get() and struct ifvlans directly to if_vlan.[ch]). -- WBR, Alexander --------------000509050108030808090809 Content-Type: text/plain; charset=UTF-8; name="ixgbe_vlans2.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="ixgbe_vlans2.diff" Index: sys/dev/ixgbe/ixgbe.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/dev/ixgbe/ixgbe.c (revision 248704) +++ sys/dev/ixgbe/ixgbe.c (working copy) @@ -2880,6 +2880,14 @@ ixgbe_allocate_queues(struct adapter *adapter) error =3D ENOMEM; goto err_rx_desc; } + + if ((rxr->vlans =3D malloc(sizeof(struct ifvlans), M_DEVBUF, + M_NOWAIT | M_ZERO)) =3D=3D NULL) { + device_printf(dev, + "Critical Failure setting up vlan index\n"); + error =3D ENOMEM; + goto err_rx_desc; + } } =20 /* @@ -4271,6 +4279,11 @@ ixgbe_free_receive_buffers(struct rx_ring *rxr) rxr->ptag =3D NULL; } =20 + if (rxr->vlans !=3D NULL) { + free(rxr->vlans, M_DEVBUF); + rxr->vlans =3D NULL; + } + return; } =20 @@ -4303,7 +4316,7 @@ ixgbe_rx_input(struct rx_ring *rxr, struct ifnet * return; } IXGBE_RX_UNLOCK(rxr); - (*ifp->if_input)(ifp, m); + (*ifp->if_input)(m->m_pkthdr.rcvif, m); IXGBE_RX_LOCK(rxr); } =20 @@ -4360,6 +4373,7 @@ ixgbe_rxeof(struct ix_queue *que) u16 count =3D rxr->process_limit; union ixgbe_adv_rx_desc *cur; struct ixgbe_rx_buf *rbuf, *nbuf; + struct ifnet *ifp_dst; =20 IXGBE_RX_LOCK(rxr); =20 @@ -4522,9 +4536,19 @@ ixgbe_rxeof(struct ix_queue *que) (staterr & IXGBE_RXD_STAT_VP)) vtag =3D le16toh(cur->wb.upper.vlan); if (vtag) { - sendmp->m_pkthdr.ether_vtag =3D vtag; - sendmp->m_flags |=3D M_VLANTAG; - } + ifp_dst =3D rxr->vlans->idx[EVL_VLANOFTAG(vtag)]; + + if (ifp_dst !=3D NULL) { + ifp_dst->if_ipackets++; + sendmp->m_pkthdr.rcvif =3D ifp_dst; + } else { + sendmp->m_pkthdr.ether_vtag =3D vtag; + sendmp->m_flags |=3D M_VLANTAG; + sendmp->m_pkthdr.rcvif =3D ifp; + } + } else + sendmp->m_pkthdr.rcvif =3D ifp; + if ((ifp->if_capenable & IFCAP_RXCSUM) !=3D 0) ixgbe_rx_checksum(staterr, sendmp, ptype); #if __FreeBSD_version >=3D 800000 @@ -4625,7 +4649,32 @@ ixgbe_rx_checksum(u32 staterr, struct mbuf * mp, u= return; } =20 +/* + * This routine gets real vlan ifp based on + * underlying ifp and vlan tag. + */ +static struct ifnet * +ixgbe_get_vlan(struct ifnet *ifp, uint16_t vtag) +{ =20 + /* XXX: IFF_MONITOR */ +#if 0 + struct lagg_port *lp =3D ifp->if_lagg; + struct lagg_softc *sc =3D lp->lp_softc; + + /* Skip lagg nesting */ + while (ifp->if_type =3D=3D IFT_IEEE8023ADLAG) { + lp =3D ifp->if_lagg; + sc =3D lp->lp_softc; + ifp =3D sc->sc_ifp; + } +#endif + /* Get vlan interface based on tag */ + ifp =3D VLAN_DEVAT(ifp, vtag); + + return (ifp); +} + /* ** This routine is run via an vlan config EVENT, ** it enables us to use the HW Filter table since @@ -4637,7 +4686,9 @@ static void ixgbe_register_vlan(void *arg, struct ifnet *ifp, u16 vtag) { struct adapter *adapter =3D ifp->if_softc; - u16 index, bit; + u16 index, bit, j; + struct rx_ring *rxr; + struct ifnet *ifv; =20 if (ifp->if_softc !=3D arg) /* Not our event */ return; @@ -4645,7 +4696,20 @@ ixgbe_register_vlan(void *arg, struct ifnet *ifp, if ((vtag =3D=3D 0) || (vtag > 4095)) /* Invalid */ return; =20 + ifv =3D ixgbe_get_vlan(ifp, vtag); + IXGBE_CORE_LOCK(adapter); + + if (ifp->if_capenable & IFCAP_VLAN_HWFILTER) { + rxr =3D adapter->rx_rings; + + for (j =3D 0; j < adapter->num_queues; j++, rxr++) { + IXGBE_RX_LOCK(rxr); + rxr->vlans->idx[vtag] =3D ifv; + IXGBE_RX_UNLOCK(rxr); + } + } + index =3D (vtag >> 5) & 0x7F; bit =3D vtag & 0x1F; adapter->shadow_vfta[index] |=3D (1 << bit); @@ -4663,7 +4727,8 @@ static void ixgbe_unregister_vlan(void *arg, struct ifnet *ifp, u16 vtag) { struct adapter *adapter =3D ifp->if_softc; - u16 index, bit; + u16 index, bit, j; + struct rx_ring *rxr; =20 if (ifp->if_softc !=3D arg) return; @@ -4672,6 +4737,15 @@ ixgbe_unregister_vlan(void *arg, struct ifnet *ifp= return; =20 IXGBE_CORE_LOCK(adapter); + + rxr =3D adapter->rx_rings; + + for (j =3D 0; j < adapter->num_queues; j++, rxr++) { + IXGBE_RX_LOCK(rxr); + rxr->vlans->idx[vtag] =3D NULL; + IXGBE_RX_UNLOCK(rxr); + } + index =3D (vtag >> 5) & 0x7F; bit =3D vtag & 0x1F; adapter->shadow_vfta[index] &=3D ~(1 << bit); @@ -4686,8 +4760,8 @@ ixgbe_setup_vlan_hw_support(struct adapter *adapte { struct ifnet *ifp =3D adapter->ifp; struct ixgbe_hw *hw =3D &adapter->hw; + u32 ctrl, j; struct rx_ring *rxr; - u32 ctrl; =20 =20 /* @@ -4713,6 +4787,15 @@ ixgbe_setup_vlan_hw_support(struct adapter *adapte= if (ifp->if_capenable & IFCAP_VLAN_HWFILTER) { ctrl &=3D ~IXGBE_VLNCTRL_CFIEN; ctrl |=3D IXGBE_VLNCTRL_VFE; + } else { + /* Zero vlan table */ + rxr =3D adapter->rx_rings; + + for (j =3D 0; j < adapter->num_queues; j++, rxr++) { + IXGBE_RX_LOCK(rxr); + memset(rxr->vlans->idx, 0, sizeof(struct ifvlans)); + IXGBE_RX_UNLOCK(rxr); + } } if (hw->mac.type =3D=3D ixgbe_mac_82598EB) ctrl |=3D IXGBE_VLNCTRL_VME; Index: sys/dev/ixgbe/ixgbe.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- sys/dev/ixgbe/ixgbe.h (revision 248704) +++ sys/dev/ixgbe/ixgbe.h (working copy) @@ -284,6 +284,11 @@ struct ix_queue { u64 irqs; }; =20 +struct ifvlans { + struct ifnet *idx[4096]; +}; + + /* * The transmit ring, one per queue */ @@ -307,7 +312,6 @@ struct tx_ring { } queue_status; u32 txd_cmd; bus_dma_tag_t txtag; - char mtx_name[16]; #ifndef IXGBE_LEGACY_TX struct buf_ring *br; struct task txq_task; @@ -324,6 +328,7 @@ struct tx_ring { unsigned long no_tx_dma_setup; u64 no_desc_avail; u64 total_packets; + char mtx_name[16]; }; =20 =20 @@ -346,8 +351,8 @@ struct rx_ring { u16 num_desc; u16 mbuf_sz; u16 process_limit; - char mtx_name[16]; struct ixgbe_rx_buf *rx_buffers; + struct ifvlans *vlans; bus_dma_tag_t ptag; =20 u32 bytes; /* Used for AIM calc */ @@ -363,6 +368,7 @@ struct rx_ring { #ifdef IXGBE_FDIR u64 flm; #endif + char mtx_name[16]; }; =20 /* Our adapter structure */ --------------000509050108030808090809-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 18:39:58 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 27851A85; Mon, 15 Apr 2013 18:39:58 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id E825212D1; Mon, 15 Apr 2013 18:39:54 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 15D891E0; Mon, 15 Apr 2013 20:36:14 +0200 (CEST) Date: Mon, 15 Apr 2013 20:42:03 +0200 From: Pawel Jakub Dawidek To: Alexander Motin Subject: Re: devstat overhead VS precision Message-ID: <20130415184203.GA1839@garage.freebsd.pl> References: <51692C95.3010901@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="jRHKVT23PllUwdXP" Content-Disposition: inline In-Reply-To: <51692C95.3010901@FreeBSD.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org, freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 18:39:58 -0000 --jRHKVT23PllUwdXP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Apr 13, 2013 at 12:59:49PM +0300, Alexander Motin wrote: > Hi. >=20 > It is long known that collecting disk and GEOM statistics may cause=20 > significant processing overhead under high IOPS. On my recent high-IOPS= =20 > benchmarks performance difference was reaching three times! Last time=20 > situation improved a lot by more active use of TSC, but there are still= =20 > many systems where TSCs are not synchronized. I propose to switch that=20 > statistics from using binuptime() to getbinuptime() to solve the problem= =20 > globally. >=20 > From one side getbinuptime() resolution is limited by 1ms, but since=20 > time is usually averaged over the many I/Os, additional sub-millisecond= =20 > precision will come from sampling. Since most of tools now show request= =20 > processing times up to 0.1ms, that precision should be sufficient. I=20 > believe real disk performance is more important that n-th digit in some= =20 > statistics. >=20 > The following patch does the change and makes disk performance=20 > irrelevant to the timecounter performance: > http://people.freebsd.org/~mav/devstat_time.patch >=20 > Are there any objections against it? No objections here, but I wonder if you were able to compare the results somehow before and after the change so we have some hard numbers to show that we don't lose much by applying the change. On a mostly unrelated note when two threads (T0 and T1) call get*time() on two different cores, but T0 does that a bit earlier is it possible that T0 can get later time than T1? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --jRHKVT23PllUwdXP Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFsSfsACgkQForvXbEpPzSVlgCgtgrIp5Wpi0WYlOFXuCZr8BZY x6IAoOUXv9oinJ61u126VKsqgPeUmWGM =noxd -----END PGP SIGNATURE----- --jRHKVT23PllUwdXP-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 19:13:36 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id BE39E603; Mon, 15 Apr 2013 19:13:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ea0-x22d.google.com (mail-ea0-x22d.google.com [IPv6:2a00:1450:4013:c01::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 04E35153D; Mon, 15 Apr 2013 19:13:35 +0000 (UTC) Received: by mail-ea0-f173.google.com with SMTP id k11so2433517eaj.32 for ; Mon, 15 Apr 2013 12:13:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=dL9CFkwr3uaohHKnHkOWW2VUUtAo0Mt/3ByA90QWg5A=; b=UUIAVJT6Skg4Lcx3ZeTkzaaKLDoXvPDz9cfCD0px8LrHCxTTkB4YbHr6Wjv6ZnxWaz BsV6TsNHyZtT2vwlbxUZfxxm4TBpmF4M7KoDZma9qm1z8TYD/7yqmARMrwJm66AM5Dg/ /DLTK+0Pe1JNInQXUSj7xNlYhhS29LXvqDKdQgl8P16xcXlZfRf0EqlfhAIyBPMTIFOJ 3YMVauUHtyGLu/m+qv1ufz/r0xROkjtKspgWfmoFWb4CPaxpyslPD+W776r7vc7U6rh7 U1Pm6XZp3NB8p6xxCyT5awByXMQ6xtvYZ70MY+Z/k6VeG39ZxcoC6eR5S9TiAVcZWkpq r8oA== X-Received: by 10.14.149.141 with SMTP id x13mr32974188eej.31.1366053214378; Mon, 15 Apr 2013 12:13:34 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPS id u44sm28373499eel.7.2013.04.15.12.13.32 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 15 Apr 2013 12:13:33 -0700 (PDT) Sender: Alexander Motin Message-ID: <516C515A.9090602@FreeBSD.org> Date: Mon, 15 Apr 2013 22:13:30 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Pawel Jakub Dawidek Subject: Re: devstat overhead VS precision References: <51692C95.3010901@FreeBSD.org> <20130415184203.GA1839@garage.freebsd.pl> In-Reply-To: <20130415184203.GA1839@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org, freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 19:13:36 -0000 On 15.04.2013 21:42, Pawel Jakub Dawidek wrote: > On Sat, Apr 13, 2013 at 12:59:49PM +0300, Alexander Motin wrote: >> Hi. >> >> It is long known that collecting disk and GEOM statistics may cause >> significant processing overhead under high IOPS. On my recent high-IOPS >> benchmarks performance difference was reaching three times! Last time >> situation improved a lot by more active use of TSC, but there are still >> many systems where TSCs are not synchronized. I propose to switch that >> statistics from using binuptime() to getbinuptime() to solve the problem >> globally. >> >> From one side getbinuptime() resolution is limited by 1ms, but since >> time is usually averaged over the many I/Os, additional sub-millisecond >> precision will come from sampling. Since most of tools now show request >> processing times up to 0.1ms, that precision should be sufficient. I >> believe real disk performance is more important that n-th digit in some >> statistics. >> >> The following patch does the change and makes disk performance >> irrelevant to the timecounter performance: >> http://people.freebsd.org/~mav/devstat_time.patch >> >> Are there any objections against it? > > No objections here, but I wonder if you were able to compare the results > somehow before and after the change so we have some hard numbers to show > that we don't lose much by applying the change. I haven't tested it statistically, but I haven't noticed any visual difference in gstat output with its 0.1ms displayed resolution. > On a mostly unrelated note when two threads (T0 and T1) call get*time() > on two different cores, but T0 does that a bit earlier is it possible > that T0 can get later time than T1? There is no any locking or memory barriers in get*time(), so CPU is free to do some prefetching and out-of-order execution, slightly blurring "a bit earlier" concept. Except such close cases I believe it should not happen. -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 19:18:19 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6A1AB754; Mon, 15 Apr 2013 19:18:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id D4451157B; Mon, 15 Apr 2013 19:18:18 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r3FJIFng018857; Mon, 15 Apr 2013 22:18:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.2 kib.kiev.ua r3FJIFng018857 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r3FJIFrR018856; Mon, 15 Apr 2013 22:18:15 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 15 Apr 2013 22:18:15 +0300 From: Konstantin Belousov To: Pawel Jakub Dawidek Subject: Re: devstat overhead VS precision Message-ID: <20130415191815.GR2930@kib.kiev.ua> References: <51692C95.3010901@FreeBSD.org> <20130415184203.GA1839@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6cExhHXXDEBW2NKZ" Content-Disposition: inline In-Reply-To: <20130415184203.GA1839@garage.freebsd.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@FreeBSD.org, Alexander Motin , freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 19:18:19 -0000 --6cExhHXXDEBW2NKZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Mon, Apr 15, 2013 at 08:42:03PM +0200, Pawel Jakub Dawidek wrote: > On a mostly unrelated note when two threads (T0 and T1) call get*time() > on two different cores, but T0 does that a bit earlier is it possible > that T0 can get later time than T1? Define earlier first. If you have taken sufficient measures to prevent preemption and interruption, e.g. by entering spinlock before the fragment that calls get*, then no, it is impossible, at least not with any x86 timekeeping hardware we use. On the other hand, if interrupts are allowed, all bets are off. --6cExhHXXDEBW2NKZ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRbFJ2AAoJEJDCuSvBvK1B23MQAJEv7S5Z6HpuFZZEPrvvoFAJ uoYC+eIVzmQ0tcf6BsCjhd0/hqwjzF34be8wtaAvJdzBoMgJ44RJci2bCl0NHM6L Onwt6L8o4Tdx5gc6LzrERH78aCrpIBtOrg5y/oOr+9ecv4EKLuCm8xQf4SDQyY4F qcteyKQOaWDyZkRZQ7FMH9FpRPHQ+1yVEWsyxoqLck4hASoE87/ogieWPr2/4RGg jMzH8UFgdmnv3FgMZc+9ksHcZZctoOZt613O+LyVX6/vQwHubeefXcRbuGxnbE2h 1LAIlzIUWwm2H6LkWrqBM6TABu17vb1YPRgGtg8hE7Gfktd7QT31IZ3NL5IO9DRE 9esF9Ya32zrCcFOXZb/TkmC9RsdcjCwQHhK/VR1rUe6ZP3C5sq1yTWwMFG2GtXIv zdMrqOlJO69Cw3efrMcPBtRY0U5b48KtOmLqK/ntuRjo7I1np4c+tXEdRpRgSJy3 +aVBJHijtId4jahvwN2+DswXw8gxhYRs692IuR+P5VSg+d5Il/NSve8RFj0SfCO1 SP4eY+MfDBbxQOoxGTjDVNNK9uRKRDAF/6Hx0zB/nvySXLDBMG9whmVXj4pYSFfL dF310fGGxKiUGZ1vr/ESAPB220TNg1O0PS1nb0r9caJHWj+FK/7b4FdrCZi5Dspy p7uuueUvPU+Ja9a5IF/V =XPWU -----END PGP SIGNATURE----- --6cExhHXXDEBW2NKZ-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 19:35:22 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 5F446D7D; Mon, 15 Apr 2013 19:35:22 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 2B1981625; Mon, 15 Apr 2013 19:35:21 +0000 (UTC) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) by mail.dawidek.net (Postfix) with ESMTPSA id 7AA01229; Mon, 15 Apr 2013 21:31:41 +0200 (CEST) Date: Mon, 15 Apr 2013 21:37:30 +0200 From: Pawel Jakub Dawidek To: Konstantin Belousov Subject: Re: devstat overhead VS precision Message-ID: <20130415193730.GB1839@garage.freebsd.pl> References: <51692C95.3010901@FreeBSD.org> <20130415184203.GA1839@garage.freebsd.pl> <20130415191815.GR2930@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4SFOXa2GPu3tIq4H" Content-Disposition: inline In-Reply-To: <20130415191815.GR2930@kib.kiev.ua> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@FreeBSD.org, Alexander Motin , freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 19:35:22 -0000 --4SFOXa2GPu3tIq4H Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 15, 2013 at 10:18:15PM +0300, Konstantin Belousov wrote: > On Mon, Apr 15, 2013 at 08:42:03PM +0200, Pawel Jakub Dawidek wrote: > > On a mostly unrelated note when two threads (T0 and T1) call get*time() > > on two different cores, but T0 does that a bit earlier is it possible > > that T0 can get later time than T1? >=20 > Define earlier first. >=20 > If you have taken sufficient measures to prevent preemption and interrupt= ion, > e.g. by entering spinlock before the fragment that calls get*, then no, > it is impossible, at least not with any x86 timekeeping hardware we use. >=20 > On the other hand, if interrupts are allowed, all bets are off. So if we consider only one thread, it is not possible for it to obtain time t0, be scheduled to different CPU and obtain t1 where t1 < t0? --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --4SFOXa2GPu3tIq4H Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlFsVvoACgkQForvXbEpPzQzzQCeOMrSkgTMbYMjl8If37USSBdT c7wAn1w8SL3vUVYHUMHseuDWom7+klgb =/CA5 -----END PGP SIGNATURE----- --4SFOXa2GPu3tIq4H-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 19:39:17 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 18DA7EC3; Mon, 15 Apr 2013 19:39:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 9DAD5164D; Mon, 15 Apr 2013 19:39:16 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r3FJdD1Y022713; Mon, 15 Apr 2013 22:39:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.2 kib.kiev.ua r3FJdD1Y022713 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r3FJdD2n022712; Mon, 15 Apr 2013 22:39:13 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 15 Apr 2013 22:39:12 +0300 From: Konstantin Belousov To: Pawel Jakub Dawidek Subject: Re: devstat overhead VS precision Message-ID: <20130415193912.GS2930@kib.kiev.ua> References: <51692C95.3010901@FreeBSD.org> <20130415184203.GA1839@garage.freebsd.pl> <20130415191815.GR2930@kib.kiev.ua> <20130415193730.GB1839@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qxmd2aOE9n9J83EO" Content-Disposition: inline In-Reply-To: <20130415193730.GB1839@garage.freebsd.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@FreeBSD.org, Alexander Motin , freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 19:39:17 -0000 --qxmd2aOE9n9J83EO Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 15, 2013 at 09:37:30PM +0200, Pawel Jakub Dawidek wrote: > On Mon, Apr 15, 2013 at 10:18:15PM +0300, Konstantin Belousov wrote: > > On Mon, Apr 15, 2013 at 08:42:03PM +0200, Pawel Jakub Dawidek wrote: > > > On a mostly unrelated note when two threads (T0 and T1) call get*time= () > > > on two different cores, but T0 does that a bit earlier is it possible > > > that T0 can get later time than T1? > >=20 > > Define earlier first. > >=20 > > If you have taken sufficient measures to prevent preemption and interru= ption, > > e.g. by entering spinlock before the fragment that calls get*, then no, > > it is impossible, at least not with any x86 timekeeping hardware we use. > >=20 > > On the other hand, if interrupts are allowed, all bets are off. >=20 > So if we consider only one thread, it is not possible for it to obtain > time t0, be scheduled to different CPU and obtain t1 where t1 < t0? Yes, I believe this scenario is also not possible. The context switching ensures that thread's view on the global memory is consistent. At least it is so on x86, and I think it must be the same on all other architectures. Otherwise the compiler emited code would not work. --qxmd2aOE9n9J83EO Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRbFdgAAoJEJDCuSvBvK1BHHwP/3jWyITU2jLV7rr4PL9byXPW 5It47gDGungmtlGIiPj7s5t9Apq4DvZ9G7kBNvR/1FjATVPbXa9TPvo6t8qqGrEm 7CIq72pt6B0BcxwBLD3cLDU6qQfm5hMu5VkJqKFGg/LpWzKkHOvGt2ostDEY5N79 WQldd1wJsRp25SsFChsb/z9RXBKIcCJvc3ukUxTQAblaIIoQ3dThUpVv+pN5/pl/ yj2URn44dQTJkNNQw4mUN72PHf66wFeYpw3I2U8Lr1LMdl/2PGuM4gbancjeGSp3 md7EoKrGll6/8lH58X4aVgHDjbqD7/ccDCn8yNok2FwaDSyiuUqUPyYTGIiCExgm Ni+71fyyL9wFd79n93ovC6LrqwaB8Lvn8fOydGp8ZPIRDRl0d+dEbhD8FUaaCaUk wl0TmhOZVarWJBstiOunWHlVeLkltcSdfV9X8gyp0ThFJZfM9nbgPdMdawRaIQ+v H1lWbXBhAFA+kwwM2zZSf1YiOvUGVbjf/Y0XSR6i6rSmhjJl2QA9BYX/Gr60I1Fg hQr9JlBsLOCZbqNX7f4KdjucVOjKLsdDyk10on4pGYIQXHCmNO28bhNiKxiqYMkP aYervPFifouuO9LvcGkQqD1XIJH4mQfSKLQtkTpKd44Sf1avoqHFu37SYp3+Fu51 eq5VSCa48ydDx69Wujek =KrTY -----END PGP SIGNATURE----- --qxmd2aOE9n9J83EO-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 20:15:03 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 07B21C97 for ; Mon, 15 Apr 2013 20:15:03 +0000 (UTC) (envelope-from jrisom@gmail.com) Received: from mail-ie0-x234.google.com (mail-ie0-x234.google.com [IPv6:2607:f8b0:4001:c03::234]) by mx1.freebsd.org (Postfix) with ESMTP id D166817D6 for ; Mon, 15 Apr 2013 20:15:02 +0000 (UTC) Received: by mail-ie0-f180.google.com with SMTP id qd14so571462ieb.11 for ; Mon, 15 Apr 2013 13:15:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=dEnI7VsGE+WYAe5GH8BzI6zd7aZ5bJ3GfOf94ossP4M=; b=qMdeH1r6nxVdX23JRdSZYv6HXN96fev6Fy9lLOGP96q5Q57VcqYcXL0pfJeOYFCGA3 H3zmSM7krFSo1vz/3R6HsMR5D9juQHvO79ZrxbWdWXeFG8jYMo7ttdZVedB80HBYDAla AFKCABwa+Q6cA4RjqCv0+z+wkUHYSvv1hheAW43k0LxZG/T21TCPQSyq8OiZbp2iYORS Q/By4A7BMO6Vd4X0qvgr/hE3EqgqDmQbB6v6ZCXWTaTPBSXdlnznrf8jTK2iFLUt2KZ1 Kps4Nsw3NCzhoAEfgN43oKUBZoqw3W+kWIjfQMHQHNSwXiynkdLM+kcAS/HtcwKaqHO5 UxJQ== X-Received: by 10.50.136.130 with SMTP id qa2mr6126771igb.1.1366056902502; Mon, 15 Apr 2013 13:15:02 -0700 (PDT) Received: from [192.168.1.34] (c-98-212-197-211.hsd1.il.comcast.net. [98.212.197.211]) by mx.google.com with ESMTPS id ua6sm12443684igb.0.2013.04.15.13.15.01 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 15 Apr 2013 13:15:01 -0700 (PDT) Message-ID: <516C5FC0.9060507@gmail.com> Date: Mon, 15 Apr 2013 15:14:56 -0500 From: Joshua Isom User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Hans Petter Selasky Subject: Re: USB Printer quirks References: <5169DFFB.4030101@gmail.com> <516BD94F.7030507@bitfrost.no> In-Reply-To: <516BD94F.7030507@bitfrost.no> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 20:15:03 -0000 On 4/15/2013 5:41 AM, Hans Petter Selasky wrote: > Hi, > > What does dmesg say about your printer. > > Is cups hooked up the correct /dev/uxxx device ? > > --HPS Here's what I got the last time I plugged it in. Apr 13 07:38:17 jri root: Unknown USB device: vendor 0x040a product 0x4043 bus uhub7 Apr 13 07:38:17 jri kernel: ugen8.2: at usbus8 Apr 13 07:38:17 jri kernel: ulpt0: on usbus8 Apr 13 07:38:17 jri kernel: ulpt0: using bi-directional mode Apr 13 07:38:17 jri kernel: umass0: on usbus8 Apr 13 07:38:17 jri kernel: umass0: SCSI over Bulk-Only; quirks = 0x0000 Apr 13 07:38:17 jri kernel: umass0:14:0:-1: Attached to scbus14 Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): got CAM status 0x50 Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): fatal error, failed to attach to device Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): lost device - 0 outstanding, 4 refs Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): removing device entry The umass is probably for the card reader, I've never tried it with FreeBSD. Cups would sometimes work, at most I'd be able to print one document/test and then it'd hang. I would actually have to power cycle the printer to try again. I think it was a problem reading from usb. What I'm really curious about is why it would work properly in a linux jail and not on FreeBSD. The closest hint I have is a kernel message "linux: pid 8781 (usb): ioctl fd=5, cmd=0x604 ('^F',4) is not implemented". From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 19:18:42 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 639EF865 for ; Mon, 15 Apr 2013 19:18:42 +0000 (UTC) (envelope-from vagner@bsdway.ru) Received: from bsdway.ru (bsdway.ru [62.109.17.46]) by mx1.freebsd.org (Postfix) with ESMTP id 75B2B1587 for ; Mon, 15 Apr 2013 19:18:41 +0000 (UTC) Received: from localhost ([188.134.95.3]) (authenticated bits=0) by bsdway.ru (8.14.4/8.14.5) with ESMTP id r3FJ6eM4054855 for ; Mon, 15 Apr 2013 23:06:41 +0400 (MSD) (envelope-from vagner@bsdway.ru) Date: Mon, 15 Apr 2013 23:06:35 +0400 From: Vagner To: FreeBSD hackers Mail List Subject: SystemV IPC. Segment info Message-ID: <20130415190635.GA22063@vagner-wrk.bsdway.ru> Mail-Followup-To: FreeBSD hackers Mail List MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="xHFwDpU9dbj6ez1V" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bsdway.ru [62.109.17.46]); Mon, 15 Apr 2013 23:06:41 +0400 (MSD) X-Mailman-Approved-At: Mon, 15 Apr 2013 20:38:50 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 19:18:42 -0000 --xHFwDpU9dbj6ez1V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello! A few weeks ago, I ran into problem, which related to SystemV IPC. More than 20 processes attached to a segment shared queue. Process-initiator for create segment was killed, as process which was accessed to segment last. Segment didn't free memory, but tagged it as SHMSEG_REMOVED as the result. This is a reason of memory overflow (memory assotiated as shm). Moreover, processes, which was attached to this segment did't get a new data. I have one resolve. I need to restarted all process, which still attached to segment. But this reason have a problem. We haven't list of this processes at system. Moreover, struct shmid_ds, which described segment, haven't this info too. This patch is a resolve of problem. It: - added a linked list of structures shmid_pi in struct shmid_ds. PID (and last access time) recorded to this struct consistently. Memory allocates with ident 'shminfo' for this list of struct shmid_pi. - added syscall shminf for get all elements from list shmid_ds. - added option [-P] in ipcs(1) for system call shminf. Thanks. -- Respectfully, Stanislav Putrya System administrator FotoStrana.Ru Ltd. ICQ IM: 328585847 Jabber-GoogleTalk: root.vagner mob.phone SPB: +79215788755 mob.phone RND: +79525600664 email: vagner@bsdway.ru email: putrya@playform.ru email: root.vagner@gmail.com site: bsdway.ru site: fotostrana.ru ---------------------------------------- ( ) ASCII ribbon campaign X - against HTML, vCards and / \ - proprietary attachments in e-mail --xHFwDpU9dbj6ez1V Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="shminf.patch" Index: usr.bin/ipcs/ipcs.c =================================================================== --- usr.bin/ipcs/ipcs.c (revision 249257) +++ usr.bin/ipcs/ipcs.c (working copy) @@ -30,6 +30,7 @@ #include #include +#include #define _KERNEL #include #include @@ -37,6 +38,7 @@ #undef _KERNEL #include +#include #include #include #include @@ -47,6 +49,9 @@ #include #include +#include +#include + #include "ipc.h" char *fmt_perm(u_short); @@ -58,12 +63,15 @@ void print_kmsqheader(int option); void print_kmsqptr(int i, int option, struct msqid_kernel *kmsqptr); void print_kshmtotal(struct shminfo shminfo); +void print_kshminfoptr(int i, int option, struct shmid_kernel *kshmptr); void print_kshmheader(int option); void print_kshmptr(int i, int option, struct shmid_kernel *kshmptr); void print_ksemtotal(struct seminfo seminfo); void print_ksemheader(int option); void print_ksemptr(int i, int option, struct semid_kernel *ksemaptr); +int shminf(int, struct shmid_pi *); + char * fmt_perm(u_short mode) { @@ -103,6 +111,7 @@ #define OUTSTANDING 4 #define PID 8 #define TIME 16 +#define PIDLIST 32 int main(int argc, char *argv[]) @@ -114,7 +123,7 @@ int i; uid_t uid = 0; - while ((i = getopt(argc, argv, "MmQqSsabC:cN:optTu:y")) != -1) + while ((i = getopt(argc, argv, "MmQqSsabPC:cN:optTu:y")) != -1) switch (i) { case 'a': option |= BIGGEST | CREATOR | OUTSTANDING | PID | TIME; @@ -165,6 +174,9 @@ user = optarg; uid = user2uid(user); break; + case 'P': + option |= PIDLIST; + break; case 'y': use_sysctl = 0; break; @@ -409,20 +421,28 @@ void print_kshmheader(int option) { - - printf("Shared Memory:\n"); - printf("T %12s %12s %-11s %-8s %-8s", - "ID", "KEY", "MODE", "OWNER", "GROUP"); - if (option & CREATOR) - printf(" %-8s %-8s", "CREATOR", "CGROUP"); - if (option & OUTSTANDING) - printf(" %12s", "NATTCH"); - if (option & BIGGEST) - printf(" %12s", "SEGSZ"); - if (option & PID) - printf(" %12s %12s", "CPID", "LPID"); - if (option & TIME) - printf(" %-8s %-8s %-8s", "ATIME", "DTIME", "CTIME"); + if (option & PIDLIST) + { + printf("Shared Memory pids info:\n"); + printf("T %12s %12s %-11s %-8s %-8s", + "ID", "KEY", "NATTCH", "PID", "ATIME"); + } + else + { + printf("Shared Memory:\n"); + printf("T %12s %12s %-11s %-8s %-8s", + "ID", "KEY", "MODE", "OWNER", "GROUP"); + if (option & CREATOR) + printf(" %-8s %-8s", "CREATOR", "CGROUP"); + if (option & OUTSTANDING) + printf(" %12s", "NATTCH"); + if (option & BIGGEST) + printf(" %12s", "SEGSZ"); + if (option & PID) + printf(" %12s %12s", "CPID", "LPID"); + if (option & TIME) + printf(" %-8s %-8s %-8s", "ATIME", "DTIME", "CTIME"); + } printf("\n"); } @@ -430,18 +450,44 @@ print_kshmptr(int i, int option, struct shmid_kernel *kshmptr) { char atime_buf[100], dtime_buf[100], ctime_buf[100]; + int sysc_num, pid, error, cnt; cvt_time(kshmptr->u.shm_atime, atime_buf); cvt_time(kshmptr->u.shm_dtime, dtime_buf); cvt_time(kshmptr->u.shm_ctime, ctime_buf); - printf("m %12d %12d %s %-8s %-8s", - IXSEQ_TO_IPCID(i, kshmptr->u.shm_perm), - (int)kshmptr->u.shm_perm.key, - fmt_perm(kshmptr->u.shm_perm.mode), - user_from_uid(kshmptr->u.shm_perm.uid, 0), - group_from_gid(kshmptr->u.shm_perm.gid, 0)); + if (option & PIDLIST) + { + struct shmid_pi shmid_pi_entry[kshmptr->u.shm_nattch]; + bzero(shmid_pi_entry, sizeof(struct shmid_pi)*(kshmptr->u.shm_nattch)); + error = shminf(IXSEQ_TO_IPCID(i, kshmptr->u.shm_perm), shmid_pi_entry); + if (error) + printf("ERROR:%d,%s ", error, strerror(errno)); + + for (cnt=0; cnt<(kshmptr->u.shm_nattch) && shmid_pi_entry[cnt].shm_pid; cnt++) + { + bzero(atime_buf, sizeof(atime_buf)); + cvt_time(shmid_pi_entry[cnt].shm_atime, atime_buf); + printf("m %12d %12d %6d", + IXSEQ_TO_IPCID(i, kshmptr->u.shm_perm), + (int)kshmptr->u.shm_perm.key, + kshmptr->u.shm_nattch); + printf("%9d %10s\n", shmid_pi_entry[cnt].shm_pid, atime_buf); + }; + return; + } + else + { + printf("m %12d %12d %s %-8s %-8s", + IXSEQ_TO_IPCID(i, kshmptr->u.shm_perm), + (int)kshmptr->u.shm_perm.key, + fmt_perm(kshmptr->u.shm_perm.mode), + user_from_uid(kshmptr->u.shm_perm.uid, 0), + group_from_gid(kshmptr->u.shm_perm.gid, 0)); + }; + kd=NULL; + if (option & CREATOR) printf(" %-8s %-8s", user_from_uid(kshmptr->u.shm_perm.cuid, 0), @@ -456,9 +502,11 @@ kshmptr->u.shm_segsz); if (option & PID) + { printf(" %12d %12d", kshmptr->u.shm_cpid, kshmptr->u.shm_lpid); + } if (option & TIME) printf(" %s %s %s", @@ -466,7 +514,8 @@ dtime_buf, ctime_buf); - printf("\n"); + if (!(option & PIDLIST)) + printf("\n"); } void Index: lib/libc/sys/Symbol.map =================================================================== --- lib/libc/sys/Symbol.map (revision 249257) +++ lib/libc/sys/Symbol.map (working copy) @@ -268,6 +268,7 @@ shmat; shmdt; shmget; + shminf; shmsys; shutdown; sigaction; @@ -921,6 +922,8 @@ __sys_shmdt; _shmget; __sys_shmget; + _shminf; + __sys_shminf; _shmsys; __sys_shmsys; _shutdown; Index: sys/fs/procfs/procfs_map.c =================================================================== --- sys/fs/procfs/procfs_map.c (revision 249257) +++ sys/fs/procfs/procfs_map.c (working copy) @@ -241,6 +241,6 @@ } } vm_map_unlock_read(map); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); return (error); } Index: sys/bsm/audit_kevents.h =================================================================== --- sys/bsm/audit_kevents.h (revision 249257) +++ sys/bsm/audit_kevents.h (working copy) @@ -383,6 +383,7 @@ #define AUE_DARWIN_PIDFORTASK 359 /* Darwin-specific. */ #define AUE_DARWIN_SYSCTL_NONADMIN 360 #define AUE_DARWIN_COPYFILE 361 /* Darwin-specific. */ +#define AUE_SHMINF 362 /* * Audit event identifiers added as part of OpenBSM, generally corresponding Index: sys/vm/vm_pageout.c =================================================================== --- sys/vm/vm_pageout.c (revision 249257) +++ sys/vm/vm_pageout.c (working copy) @@ -1364,7 +1364,7 @@ continue; } if (!vm_map_trylock_read(&vm->vm_map)) { - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); PROC_UNLOCK(p); continue; } @@ -1372,7 +1372,7 @@ vm_map_unlock_read(&vm->vm_map); if (shortage == VM_OOM_MEM) size += vmspace_resident_count(vm); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); /* * if the this process is bigger than the biggest one * remember it. @@ -1788,7 +1788,7 @@ tryagain = 1; } #endif - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); } sx_sunlock(&allproc_lock); if (tryagain != 0 && attempts <= 10) Index: sys/vm/vm_map.c =================================================================== --- sys/vm/vm_map.c (revision 249257) +++ sys/vm/vm_map.c (working copy) @@ -331,7 +331,7 @@ } static inline void -vmspace_dofree(struct vmspace *vm) +vmspace_dofree(struct vmspace *vm, int *pid) { CTR1(KTR_VM, "vmspace_free: %p", vm); @@ -340,7 +340,7 @@ * Make sure any SysV shm is freed, it might not have been in * exit1(). */ - shmexit(vm); + shmexit(vm, pid); /* * Lock the map, to wait out all other references to it. @@ -356,14 +356,14 @@ } void -vmspace_free(struct vmspace *vm) +vmspace_free(struct vmspace *vm, int *pid) { if (vm->vm_refcnt == 0) panic("vmspace_free: attempt to free already freed vmspace"); if (atomic_fetchadd_int(&vm->vm_refcnt, -1) == 1) - vmspace_dofree(vm); + vmspace_dofree(vm, pid); } void @@ -376,7 +376,7 @@ p->p_vmspace = NULL; PROC_VMSPACE_UNLOCK(p); KASSERT(vm == &vmspace0, ("vmspace_exitfree: wrong vmspace")); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); } void @@ -425,7 +425,7 @@ p->p_vmspace = &vmspace0; PROC_VMSPACE_UNLOCK(p); pmap_activate(td); - vmspace_dofree(vm); + vmspace_dofree(vm, &p->p_pid); } vmspace_container_reset(p); } @@ -453,7 +453,7 @@ } while (!atomic_cmpset_int(&vm->vm_refcnt, refcnt, refcnt + 1)); if (vm != p->p_vmspace) { PROC_VMSPACE_UNLOCK(p); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); return (NULL); } PROC_VMSPACE_UNLOCK(p); @@ -3657,6 +3657,7 @@ vmspace_exec(struct proc *p, vm_offset_t minuser, vm_offset_t maxuser) { struct vmspace *oldvmspace = p->p_vmspace; + int *oldpid = &p->p_pid; struct vmspace *newvmspace; newvmspace = vmspace_alloc(minuser, maxuser); @@ -3675,7 +3676,7 @@ PROC_VMSPACE_UNLOCK(p); if (p == curthread->td_proc) pmap_activate(curthread); - vmspace_free(oldvmspace); + vmspace_free(oldvmspace, oldpid); return (0); } @@ -3687,9 +3688,11 @@ vmspace_unshare(struct proc *p) { struct vmspace *oldvmspace = p->p_vmspace; + int *oldpid; struct vmspace *newvmspace; vm_ooffset_t fork_charge; + oldpid = &p->p_pid; if (oldvmspace->vm_refcnt == 1) return (0); fork_charge = 0; @@ -3697,7 +3700,7 @@ if (newvmspace == NULL) return (ENOMEM); if (!swap_reserve_by_cred(fork_charge, p->p_ucred)) { - vmspace_free(newvmspace); + vmspace_free(newvmspace, NULL); return (ENOMEM); } PROC_VMSPACE_LOCK(p); @@ -3705,7 +3708,7 @@ PROC_VMSPACE_UNLOCK(p); if (p == curthread->td_proc) pmap_activate(curthread); - vmspace_free(oldvmspace); + vmspace_free(oldvmspace, oldpid); return (0); } Index: sys/vm/vm_meter.c =================================================================== --- sys/vm/vm_meter.c (revision 249257) +++ sys/vm/vm_meter.c (working copy) @@ -184,7 +184,7 @@ VM_OBJECT_UNLOCK(object); } vm_map_unlock_read(map); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); if (paging) total.t_pw++; } Index: sys/vm/vm_glue.c =================================================================== --- sys/vm/vm_glue.c (revision 249257) +++ sys/vm/vm_glue.c (working copy) @@ -945,7 +945,7 @@ didswap++; PROC_UNLOCK(p); vm_map_unlock(&vm->vm_map); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); sx_sunlock(&allproc_lock); goto retry; } @@ -954,7 +954,7 @@ PROC_UNLOCK(p); vm_map_unlock(&vm->vm_map); nextproc1: - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); continue; } sx_sunlock(&allproc_lock); Index: sys/vm/vm_extern.h =================================================================== --- sys/vm/vm_extern.h (revision 249257) +++ sys/vm/vm_extern.h (working copy) @@ -81,7 +81,7 @@ int vmspace_unshare(struct proc *); void vmspace_exit(struct thread *); struct vmspace *vmspace_acquire_ref(struct proc *); -void vmspace_free(struct vmspace *); +void vmspace_free(struct vmspace *, int *); void vmspace_exitfree(struct proc *); void vnode_pager_setsize(struct vnode *, vm_ooffset_t); int vslock(void *, size_t); Index: sys/security/audit/audit_bsm.c =================================================================== --- sys/security/audit/audit_bsm.c (revision 249257) +++ sys/security/audit/audit_bsm.c (working copy) @@ -1403,7 +1403,7 @@ } break; - /* AUE_SHMAT, AUE_SHMCTL, AUE_SHMDT and AUE_SHMGET are SysV IPC */ + /* AUE_SHMAT, AUE_SHMCTL, AUE_SHMDT, AUE_SHMINF and AUE_SHMGET are SysV IPC */ case AUE_SHMAT: if (ARG_IS_VALID(kar, ARG_SVIPC_ID)) { tok = au_to_arg32(1, "shmid", ar->ar_arg_svipc_id); @@ -1458,6 +1458,15 @@ } break; + case AUE_SHMINF: + if (ARG_IS_VALID(kar, ARG_SVIPC_ID)) { + tok = au_to_arg32(1, "shmid", ar->ar_arg_svipc_id); + kau_write(rec, tok); + /* XXXAUDIT: Does having the ipc token make sense? */ + tok = au_to_ipc(AT_IPC_SHM, ar->ar_arg_svipc_id); + kau_write(rec, tok); + } + case AUE_SHMGET: /* This is unusual; the return value is in an argument token */ if (ARG_IS_VALID(kar, ARG_SVIPC_ID)) { Index: sys/sys/syscall.mk =================================================================== --- sys/sys/syscall.mk (revision 249257) +++ sys/sys/syscall.mk (working copy) @@ -396,4 +396,5 @@ rctl_remove_rule.o \ posix_fallocate.o \ posix_fadvise.o \ - wait6.o + wait6.o \ + shminf.o Index: sys/sys/syscall.h =================================================================== --- sys/sys/syscall.h (revision 249257) +++ sys/sys/syscall.h (working copy) @@ -448,4 +448,5 @@ #define SYS_posix_fallocate 530 #define SYS_posix_fadvise 531 #define SYS_wait6 532 -#define SYS_MAXSYSCALL 533 +#define SYS_shminf 533 +#define SYS_MAXSYSCALL 534 Index: sys/sys/ipc.h =================================================================== --- sys/sys/ipc.h (revision 249257) +++ sys/sys/ipc.h (working copy) @@ -135,7 +135,7 @@ int ipcperm(struct thread *, struct ipc_perm *, int); extern void (*shmfork_hook)(struct proc *, struct proc *); -extern void (*shmexit_hook)(struct vmspace *); +extern void (*shmexit_hook)(struct vmspace *, int *); #else /* ! _KERNEL */ Index: sys/sys/sysproto.h =================================================================== --- sys/sys/sysproto.h (revision 249257) +++ sys/sys/sysproto.h (working copy) @@ -1747,6 +1747,10 @@ char wrusage_l_[PADL_(struct __wrusage *)]; struct __wrusage * wrusage; char wrusage_r_[PADR_(struct __wrusage *)]; char info_l_[PADL_(siginfo_t *)]; siginfo_t * info; char info_r_[PADR_(siginfo_t *)]; }; +struct shminf_args { + char shmid_l_[PADL_(int)]; int shmid; char shmid_r_[PADR_(int)]; + char buf_l_[PADL_(struct shmid_pi *)]; struct shmid_pi * buf; char buf_r_[PADR_(struct shmid_pi *)]; +}; int nosys(struct thread *, struct nosys_args *); void sys_sys_exit(struct thread *, struct sys_exit_args *); int sys_fork(struct thread *, struct fork_args *); @@ -2125,6 +2129,7 @@ int sys_posix_fallocate(struct thread *, struct posix_fallocate_args *); int sys_posix_fadvise(struct thread *, struct posix_fadvise_args *); int sys_wait6(struct thread *, struct wait6_args *); +int sys_shminf(struct thread *, struct shminf_args *); #ifdef COMPAT_43 @@ -2817,6 +2822,7 @@ #define SYS_AUE_posix_fallocate AUE_NULL #define SYS_AUE_posix_fadvise AUE_NULL #define SYS_AUE_wait6 AUE_WAIT6 +#define SYS_AUE_shminf AUE_SHMINF #undef PAD_ #undef PADL_ Index: sys/sys/shm.h =================================================================== --- sys/sys/shm.h (revision 249257) +++ sys/sys/shm.h (working copy) @@ -42,6 +42,7 @@ #include #include #include +#include #define SHM_RDONLY 010000 /* Attach read-only (else read-write) */ #define SHM_RND 020000 /* Round attach address to SHMLBA */ @@ -90,6 +91,15 @@ }; #endif + + +struct shmid_pi { + pid_t shm_pid; + time_t shm_atime; + LIST_ENTRY(shmid_pi) shm_pi_list; +}; + + struct shmid_ds { struct ipc_perm shm_perm; /* operation permission structure */ size_t shm_segsz; /* size of segment in bytes */ @@ -99,8 +109,11 @@ time_t shm_atime; /* time of last shmat() */ time_t shm_dtime; /* time of last shmdt() */ time_t shm_ctime; /* time of last change by shmctl() */ + LIST_HEAD(shmid_pi_head, shmid_pi) shm_pi; /* processes info who is attached. !EDIT this in /usr/include/sys/shm.h */ + }; + #ifdef _KERNEL #include @@ -142,7 +155,7 @@ struct proc; struct vmspace; -void shmexit(struct vmspace *); +void shmexit(struct vmspace *, int *); void shmfork(struct proc *, struct proc *); #else /* !_KERNEL */ @@ -159,6 +172,7 @@ #endif void *shmat(int, const void *, int); int shmget(key_t, size_t, int); +int shminf(int, struct shmid_pi *); int shmctl(int, int, struct shmid_ds *); int shmdt(const void *); __END_DECLS Index: sys/kern/kern_proc.c =================================================================== --- sys/kern/kern_proc.c (revision 249257) +++ sys/kern/kern_proc.c (working copy) @@ -2114,7 +2114,7 @@ } } vm_map_unlock_read(map); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); PRELE(p); free(kve, M_TEMP); return (error); @@ -2302,7 +2302,7 @@ } } vm_map_unlock_read(map); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); PRELE(p); free(kve, M_TEMP); return (error); Index: sys/kern/sysv_shm.c =================================================================== --- sys/kern/sysv_shm.c (revision 249257) +++ sys/kern/sysv_shm.c (working copy) @@ -99,7 +99,9 @@ FEATURE(sysv_shm, "System V shared memory segments support"); static MALLOC_DEFINE(M_SHM, "shm", "SVID compatible shared memory segments"); +static MALLOC_DEFINE(M_SHMINF, "shminfo", "Struct for pids info in shared memory segments"); + static int shmget_allocate_segment(struct thread *td, struct shmget_args *uap, int mode); static int shmget_existing(struct thread *td, struct shmget_args *uap, @@ -123,12 +125,12 @@ static int shm_find_segment_by_key(key_t); static struct shmid_kernel *shm_find_segment_by_shmid(int); static struct shmid_kernel *shm_find_segment_by_shmidx(int); -static int shm_delete_mapping(struct vmspace *vm, struct shmmap_state *); +static int shm_delete_mapping(struct vmspace *vm, struct shmmap_state *, int *); static void shmrealloc(void); static int shminit(void); static int sysvshm_modload(struct module *, int, void *); static int shmunload(void); -static void shmexit_myhook(struct vmspace *vm); +static void shmexit_myhook(struct vmspace *vm, int *pid); static void shmfork_myhook(struct proc *p1, struct proc *p2); static int sysctl_shmsegs(SYSCTL_HANDLER_ARGS); @@ -247,6 +249,7 @@ #ifdef MAC mac_sysvshm_cleanup(shmseg); #endif + racct_sub_cred(shmseg->cred, RACCT_NSHM, 1); racct_sub_cred(shmseg->cred, RACCT_SHMSIZE, size); crfree(shmseg->cred); @@ -254,16 +257,19 @@ } static int -shm_delete_mapping(struct vmspace *vm, struct shmmap_state *shmmap_s) +shm_delete_mapping(struct vmspace *vm, struct shmmap_state *shmmap_s, int *pid) { struct shmid_kernel *shmseg; + struct shmid_pi *shm_pi_entry; int segnum, result; vm_size_t size; GIANT_REQUIRED; segnum = IPCID_TO_IX(shmmap_s->shmid); + shmseg = &shmsegs[segnum]; + size = round_page(shmseg->u.shm_segsz); result = vm_map_remove(&vm->vm_map, shmmap_s->va, shmmap_s->va + size); if (result != KERN_SUCCESS) @@ -272,8 +278,26 @@ shmseg->u.shm_dtime = time_second; if ((--shmseg->u.shm_nattch <= 0) && (shmseg->u.shm_perm.mode & SHMSEG_REMOVED)) { + LIST_FOREACH(shm_pi_entry, &shmseg->u.shm_pi, shm_pi_list) + { + LIST_REMOVE(shm_pi_entry, shm_pi_list); + free(shm_pi_entry, M_SHMINF); + break; + }; + shm_deallocate_segment(shmseg); shm_last_free = segnum; + } else if (pid) + { + LIST_FOREACH(shm_pi_entry, &shmseg->u.shm_pi, shm_pi_list) + { + if (shm_pi_entry->shm_pid == (*pid)) + { + LIST_REMOVE(shm_pi_entry, shm_pi_list); + free(shm_pi_entry, M_SHMINF); + break; + } + }; } return (0); } @@ -290,6 +314,7 @@ { struct proc *p = td->td_proc; struct shmmap_state *shmmap_s; + #ifdef MAC struct shmid_kernel *shmsegptr; #endif @@ -320,7 +345,9 @@ if (error != 0) goto done2; #endif - error = shm_delete_mapping(p->p_vmspace, shmmap_s); + + error = shm_delete_mapping(p->p_vmspace, shmmap_s, &p->p_pid); + done2: mtx_unlock(&Giant); return (error); @@ -344,6 +371,7 @@ int i, flags; struct shmid_kernel *shmseg; struct shmmap_state *shmmap_s = NULL; + struct shmid_pi *shmseg_pi; vm_offset_t attach_va; vm_prot_t prot; vm_size_t size; @@ -420,6 +448,15 @@ goto done2; } + if (!shmseg->u.shm_nattch) + LIST_INIT(&shmseg->u.shm_pi); + + shmseg_pi = malloc(sizeof(struct shmid_pi), + M_SHMINF, M_WAITOK); + shmseg_pi->shm_pid = p->p_pid; + shmseg_pi->shm_atime = time_second; + LIST_INSERT_HEAD(&shmseg->u.shm_pi, shmseg_pi, shm_pi_list); + shmmap_s->va = attach_va; shmmap_s->shmid = shmid; shmseg->u.shm_lpid = p->p_pid; @@ -439,7 +476,34 @@ return kern_shmat(td, uap->shmid, uap->shmaddr, uap->shmflg); } +#ifndef _SYS_SYSPROTO_H_ +struct shminf_args { + int shmid; + shmid_pi *buf; +}; +#endif + int +sys_shminf(struct thread *td, struct shminf_args *uap) +{ + struct shmid_kernel *shmseg; + struct shmid_pi *buffer, *userbuf; + int error = 0; + + userbuf = uap->buf; + shmseg = shm_find_segment_by_shmid(uap->shmid); + if (shmseg == NULL) + return(EINVAL); + LIST_FOREACH(buffer, &shmseg->u.shm_pi, shm_pi_list) + { + copyout(&buffer->shm_pid, uap->buf, sizeof(struct shmid_pi)); + uap->buf++; + } + uap->buf = userbuf; + return(error); +} + +int kern_shmctl(td, shmid, cmd, buf, bufsz) struct thread *td; int shmid; @@ -525,7 +589,7 @@ shmseg->u.shm_ctime = time_second; break; } - case IPC_RMID: + case IPC_RMID: { error = ipcperm(td, &shmseg->u.shm_perm, IPC_M); if (error) goto done2; @@ -536,6 +600,7 @@ shm_last_free = IPCID_TO_IX(shmid); } break; + } #if 0 case SHM_LOCK: case SHM_UNLOCK: @@ -799,7 +864,7 @@ } static void -shmexit_myhook(struct vmspace *vm) +shmexit_myhook(struct vmspace *vm, int *pid) { struct shmmap_state *base, *shm; int i; @@ -809,7 +874,7 @@ mtx_lock(&Giant); for (i = 0, shm = base; i < shminfo.shmseg; i++, shm++) { if (shm->shmid != -1) - shm_delete_mapping(vm, shm); + shm_delete_mapping(vm, shm, pid); } mtx_unlock(&Giant); free(base, M_SHM); @@ -847,6 +912,7 @@ SYSCALL_INIT_HELPER(shmctl), SYSCALL_INIT_HELPER(shmdt), SYSCALL_INIT_HELPER(shmget), + SYSCALL_INIT_HELPER(shminf), #if defined(COMPAT_FREEBSD4) || defined(COMPAT_FREEBSD5) || \ defined(COMPAT_FREEBSD6) || defined(COMPAT_FREEBSD7) SYSCALL_INIT_HELPER_COMPAT(freebsd7_shmctl), @@ -1040,7 +1106,7 @@ static sy_call_t *shmcalls[] = { (sy_call_t *)sys_shmat, (sy_call_t *)oshmctl, (sy_call_t *)sys_shmdt, (sy_call_t *)sys_shmget, - (sy_call_t *)freebsd7_shmctl + (sy_call_t *)freebsd7_shmctl, (sy_call_t *)sys_shminf }; int Index: sys/kern/init_sysent.c =================================================================== --- sys/kern/init_sysent.c (revision 249257) +++ sys/kern/init_sysent.c (working copy) @@ -567,4 +567,5 @@ { AS(posix_fallocate_args), (sy_call_t *)sys_posix_fallocate, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 530 = posix_fallocate */ { AS(posix_fadvise_args), (sy_call_t *)sys_posix_fadvise, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 531 = posix_fadvise */ { AS(wait6_args), (sy_call_t *)sys_wait6, AUE_WAIT6, NULL, 0, 0, 0, SY_THR_STATIC }, /* 532 = wait6 */ + { AS(shminf_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 533 = shminf */ }; Index: sys/kern/kern_exec.c =================================================================== --- sys/kern/kern_exec.c (revision 249257) +++ sys/kern/kern_exec.c (working copy) @@ -1052,7 +1052,7 @@ sv_minuser = MAX(sv->sv_minuser, PAGE_SIZE); if (vmspace->vm_refcnt == 1 && vm_map_min(map) == sv_minuser && vm_map_max(map) == sv->sv_maxuser) { - shmexit(vmspace); + shmexit(vmspace, &p->p_pid); pmap_remove_pages(vmspace_pmap(vmspace)); vm_map_remove(map, vm_map_min(map), vm_map_max(map)); } else { Index: sys/kern/syscalls.c =================================================================== --- sys/kern/syscalls.c (revision 249257) +++ sys/kern/syscalls.c (working copy) @@ -540,4 +540,5 @@ "posix_fallocate", /* 530 = posix_fallocate */ "posix_fadvise", /* 531 = posix_fadvise */ "wait6", /* 532 = wait6 */ + "shminf", /* 533 = shminf */ }; Index: sys/kern/systrace_args.c =================================================================== --- sys/kern/systrace_args.c (revision 249257) +++ sys/kern/systrace_args.c (working copy) @@ -3256,6 +3256,14 @@ *n_args = 6; break; } + /* shminf */ + case 533: { + struct shminf_args *p = params; + iarg[0] = p->shmid; /* int */ + uarg[1] = (intptr_t) p->buf; /* struct shmid_pi * */ + *n_args = 2; + break; + } default: *n_args = 0; break; @@ -8669,6 +8677,19 @@ break; }; break; + /* shminf */ + case 533: + switch(ndx) { + case 0: + p = "int"; + break; + case 1: + p = "struct shmid_pi *"; + break; + default: + break; + }; + break; default: break; }; Index: sys/kern/kern_fork.c =================================================================== --- sys/kern/kern_fork.c (revision 249257) +++ sys/kern/kern_fork.c (working copy) @@ -938,7 +938,7 @@ racct_proc_exit(newproc); fail1: if (vm2 != NULL) - vmspace_free(vm2); + vmspace_free(vm2, NULL); uma_zfree(proc_zone, newproc); #ifdef PROCDESC if (((flags & RFPROCDESC) != 0) && (fp_procdesc != NULL)) Index: sys/kern/vfs_aio.c =================================================================== --- sys/kern/vfs_aio.c (revision 249257) +++ sys/kern/vfs_aio.c (working copy) @@ -1115,7 +1115,7 @@ * that it was acting on behalf of. */ if (tmpvm != myvm) { - vmspace_free(tmpvm); + vmspace_free(tmpvm, &mycp->p_pid); } curcp = userp; } @@ -1160,7 +1160,7 @@ } #endif /* Remove our vmspace reference. */ - vmspace_free(tmpvm); + vmspace_free(tmpvm, &mycp->p_pid); curcp = mycp; Index: sys/kern/sys_process.c =================================================================== --- sys/kern/sys_process.c (revision 249257) +++ sys/kern/sys_process.c (working copy) @@ -386,7 +386,7 @@ } while (0); vm_map_unlock_read(map); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); pve->pve_fsid = VNOVAL; pve->pve_fileid = VNOVAL; Index: sys/kern/sysv_ipc.c =================================================================== --- sys/kern/sysv_ipc.c (revision 249257) +++ sys/kern/sysv_ipc.c (working copy) @@ -49,7 +49,7 @@ #include void (*shmfork_hook)(struct proc *, struct proc *) = NULL; -void (*shmexit_hook)(struct vmspace *) = NULL; +void (*shmexit_hook)(struct vmspace *, int *) = NULL; /* called from kern_fork.c */ void @@ -64,11 +64,11 @@ /* called from kern_exit.c */ void -shmexit(struct vmspace *vm) +shmexit(struct vmspace *vm, int *pid) { if (shmexit_hook != NULL) - shmexit_hook(vm); + shmexit_hook(vm, pid); return; } Index: sys/kern/syscalls.master =================================================================== --- sys/kern/syscalls.master (revision 249257) +++ sys/kern/syscalls.master (working copy) @@ -952,5 +952,6 @@ int *status, int options, \ struct __wrusage *wrusage, \ siginfo_t *info); } +533 AUE_SHMINF NOSTD { int shminf(int shmid, struct shmid_pi *buf); } ; Please copy any additions and changes to the following compatability tables: ; sys/compat/freebsd32/syscalls.master Index: sys/dev/hwpmc/hwpmc_mod.c =================================================================== --- sys/dev/hwpmc/hwpmc_mod.c (revision 249257) +++ sys/dev/hwpmc/hwpmc_mod.c (working copy) @@ -1780,7 +1780,7 @@ } vm_map_unlock_read(map); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); return; } Index: sys/compat/freebsd32/freebsd32_systrace_args.c =================================================================== --- sys/compat/freebsd32/freebsd32_systrace_args.c (revision 249257) +++ sys/compat/freebsd32/freebsd32_systrace_args.c (working copy) @@ -3058,6 +3058,14 @@ *n_args = 6; break; } + /* shminf */ + case 533: { + struct shminf_args *p = params; + iarg[0] = p->shmid; /* int */ + uarg[1] = (intptr_t) p->buf; /* struct shmid_pi * */ + *n_args = 2; + break; + } default: *n_args = 0; break; @@ -8167,6 +8175,19 @@ break; }; break; + /* shminf */ + case 533: + switch(ndx) { + case 0: + p = "int"; + break; + case 1: + p = "struct shmid_pi *"; + break; + default: + break; + }; + break; default: break; }; Index: sys/compat/freebsd32/freebsd32_syscalls.c =================================================================== --- sys/compat/freebsd32/freebsd32_syscalls.c (revision 249257) +++ sys/compat/freebsd32/freebsd32_syscalls.c (working copy) @@ -556,4 +556,5 @@ "freebsd32_posix_fallocate", /* 530 = freebsd32_posix_fallocate */ "freebsd32_posix_fadvise", /* 531 = freebsd32_posix_fadvise */ "freebsd32_wait6", /* 532 = freebsd32_wait6 */ + "shminf", /* 533 = shminf */ }; Index: sys/compat/freebsd32/syscalls.master =================================================================== --- sys/compat/freebsd32/syscalls.master (revision 249257) +++ sys/compat/freebsd32/syscalls.master (working copy) @@ -1001,4 +1001,5 @@ int *status, int options, \ struct wrusage32 *wrusage, \ siginfo_t *info); } +533 AUE_SHMINF NOSTD|NOPROTO { int shminf(int shmid, struct shmid_pi *buf); } Index: sys/compat/freebsd32/freebsd32_syscall.h =================================================================== --- sys/compat/freebsd32/freebsd32_syscall.h (revision 249257) +++ sys/compat/freebsd32/freebsd32_syscall.h (working copy) @@ -426,4 +426,5 @@ #define FREEBSD32_SYS_freebsd32_posix_fallocate 530 #define FREEBSD32_SYS_freebsd32_posix_fadvise 531 #define FREEBSD32_SYS_freebsd32_wait6 532 -#define FREEBSD32_SYS_MAXSYSCALL 533 +#define FREEBSD32_SYS_shminf 533 +#define FREEBSD32_SYS_MAXSYSCALL 534 Index: sys/compat/freebsd32/freebsd32_sysent.c =================================================================== --- sys/compat/freebsd32/freebsd32_sysent.c (revision 249257) +++ sys/compat/freebsd32/freebsd32_sysent.c (working copy) @@ -593,4 +593,5 @@ { AS(freebsd32_posix_fallocate_args), (sy_call_t *)freebsd32_posix_fallocate, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 530 = freebsd32_posix_fallocate */ { AS(freebsd32_posix_fadvise_args), (sy_call_t *)freebsd32_posix_fadvise, AUE_NULL, NULL, 0, 0, 0, SY_THR_STATIC }, /* 531 = freebsd32_posix_fadvise */ { AS(freebsd32_wait6_args), (sy_call_t *)freebsd32_wait6, AUE_WAIT6, NULL, 0, 0, 0, SY_THR_STATIC }, /* 532 = freebsd32_wait6 */ + { AS(shminf_args), (sy_call_t *)lkmressys, AUE_NULL, NULL, 0, 0, 0, SY_THR_ABSENT }, /* 533 = shminf */ }; Index: sys/compat/linprocfs/linprocfs.c =================================================================== --- sys/compat/linprocfs/linprocfs.c (revision 249257) +++ sys/compat/linprocfs/linprocfs.c (working copy) @@ -1114,7 +1114,7 @@ } } vm_map_unlock_read(map); - vmspace_free(vm); + vmspace_free(vm, &p->p_pid); return (error); } --xHFwDpU9dbj6ez1V-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 20:43:15 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id F3E1ABC1; Mon, 15 Apr 2013 20:43:14 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 825BA1AA1; Mon, 15 Apr 2013 20:43:14 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id EF36E89FBE; Mon, 15 Apr 2013 20:43:06 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.6/8.14.6) with ESMTP id r3FKh6bm038497; Mon, 15 Apr 2013 20:43:06 GMT (envelope-from phk@phk.freebsd.dk) To: Alexander Motin Subject: Re: devstat overhead VS precision In-reply-to: <516C515A.9090602@FreeBSD.org> From: "Poul-Henning Kamp" References: <51692C95.3010901@FreeBSD.org> <20130415184203.GA1839@garage.freebsd.pl> <516C515A.9090602@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Date: Mon, 15 Apr 2013 20:43:06 +0000 Message-ID: <38496.1366058586@critter.freebsd.dk> X-Mailman-Approved-At: Mon, 15 Apr 2013 20:55:39 +0000 Cc: freebsd-hackers@FreeBSD.org, freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 20:43:15 -0000 In message <516C515A.9090602@FreeBSD.org>, Alexander Motin writes: >>> I propose to switch that >>> statistics from using binuptime() to getbinuptime() to solve the problem >>> globally. >> No objections here, but I wonder if you were able to compare the results >> somehow before and after the change so we have some hard numbers to show >> that we don't lose much by applying the change. > >I haven't tested it statistically, but I haven't noticed any visual >difference in gstat output with its 0.1ms displayed resolution. I have tested it statistically, back when I wrote GEOM: It leads to very significant statistical bias. Just about the only thing in devstat that has any predictive power with respect to filesystem performance, is the latency, which measures how long time it takes to satisfy each I/O request. If you run gstat(8), this is the "ms/*" numbers: milliseconds per this or that. The rest of what's in devstat, with the exception of the queue-length ("L(q)") has almost no predictive power, and is IMO, practically pointless. In particular the %busy is totally misleading and I deeply regret that I didn't fight to kill it back then. If you switch to getbinuptime(), the latency measurements will only be precise if the I/O operations take much longer than the timecounter update period, which is not guaranteed to be 1000 Hz btw. For measuring how much USB-sticks suck, that will work fine. For tuning anything on a non-ridiculous SSD device or modern harddisks, it will be useless because of the bias you introduce is *not* one which averages out over many operations. The fundamental problem is that on a busy system, getbinuptime() does not get called at random times, it will be heavily affected by the I/O traffic, because of the interrupts, the bus-traffic itself, the cache-effects of I/O transfers and the context-switches by the processes causing the I/O. So yes, you can switch to getbinuptime(), but the only statistical responsible way to do so, would be to supress latency measurements on all I/O operations which complete in less than 5-10 timecounter interrupts. Apart from some practical issues implementing it, the numbers that came out would be pretty useless. The right idea is probably to bucketize the latencies, so that rather than having to keep track of devstat in real time to find out, you could get a histogram at any time showing past performance something like: Latency distribution: <5msec: 92.12 % <10msec: 0.17 % <20msec: 1.34 % <50msec: 6.37 % >50msec: 0.00 % Doing that with getbinuptime() would be statistically defensible provided the top bucket is "<5msec" and it would very clearly tell people if they have I/O trouble or not, which IMO is what people want to know. The cost 20 64bit counters in struct devstat (N|R|W|E)*5*8 = 160 bytes, but since devstat is already 288 bytes, that isn't a major catastropy. The ability to measure latency precisly should be retained, but it could be made a sysctl enabled debugging facility. The %busy crap should be killed, all it does is confuse people. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 15 21:31:46 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7A15C728; Mon, 15 Apr 2013 21:31:46 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ea0-x231.google.com (mail-ea0-x231.google.com [IPv6:2a00:1450:4013:c01::231]) by mx1.freebsd.org (Postfix) with ESMTP id B26011C41; Mon, 15 Apr 2013 21:31:45 +0000 (UTC) Received: by mail-ea0-f177.google.com with SMTP id q14so2361957eaj.36 for ; Mon, 15 Apr 2013 14:31:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=BoWe5YrinTteFVc3UIVKwBJa1JITwjJq9jtj6xumihU=; b=KwA3cHtWWVKJ+weGpzCUC+TnXIlwAmN3qFJhQUzHubwX7r/xFwGxjdnvr02cqVkVxc DhI+FsQwQhJsF6Mzks4q2/UMRFkLmY4wHV6UkiqQv2mQsTN9If7fGzF0aMHfd2N7jo3w V42FUnFKj4o7qnB73GKc9ptA97wdPDlj9nZX45bzLAvZrCaXzOEqrucNQm64BuvYOXRs z7epN5WS3q4wUICOxkHvqhyqxdGdcfbEwhPbHbFCXtnvtRKa5gfR/c0GLcp8iXSIW0S/ rJ2ss6ZI9ZUmKBoFxNEckwKtG53QyO+Ev5O6Od+oP2AQsbl1UlL8RIUC22GQLyIDQTD4 hRaQ== X-Received: by 10.15.99.201 with SMTP id bl49mr66026891eeb.43.1366061504877; Mon, 15 Apr 2013 14:31:44 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPS id b47sm9445124eez.2.2013.04.15.14.31.42 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 15 Apr 2013 14:31:43 -0700 (PDT) Sender: Alexander Motin Message-ID: <516C71BC.4000902@FreeBSD.org> Date: Tue, 16 Apr 2013 00:31:40 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Poul-Henning Kamp Subject: Re: devstat overhead VS precision References: <51692C95.3010901@FreeBSD.org> <20130415184203.GA1839@garage.freebsd.pl> <516C515A.9090602@FreeBSD.org> <38496.1366058586@critter.freebsd.dk> In-Reply-To: <38496.1366058586@critter.freebsd.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org, freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Apr 2013 21:31:46 -0000 On 15.04.2013 23:43, Poul-Henning Kamp wrote: > In message <516C515A.9090602@FreeBSD.org>, Alexander Motin writes: > >>>> I propose to switch that >>>> statistics from using binuptime() to getbinuptime() to solve the problem >>>> globally. > >>> No objections here, but I wonder if you were able to compare the results >>> somehow before and after the change so we have some hard numbers to show >>> that we don't lose much by applying the change. >> >> I haven't tested it statistically, but I haven't noticed any visual >> difference in gstat output with its 0.1ms displayed resolution. > > I have tested it statistically, back when I wrote GEOM: It leads > to very significant statistical bias. > > Just about the only thing in devstat that has any predictive power > with respect to filesystem performance, is the latency, which measures > how long time it takes to satisfy each I/O request. > > If you run gstat(8), this is the "ms/*" numbers: milliseconds per > this or that. > > The rest of what's in devstat, with the exception of the queue-length > ("L(q)") has almost no predictive power, and is IMO, practically > pointless. In particular the %busy is totally misleading and I > deeply regret that I didn't fight to kill it back then. > > If you switch to getbinuptime(), the latency measurements will only > be precise if the I/O operations take much longer than the timecounter > update period, which is not guaranteed to be 1000 Hz btw. > > For measuring how much USB-sticks suck, that will work fine. > > For tuning anything on a non-ridiculous SSD device or modern > harddisks, it will be useless because of the bias you introduce is > *not* one which averages out over many operations. Could you please explain why? Unless disk I/O somehow aliased to hardclock(), each of them should get random error from 0 to max(1ms, 1s/HZ). With large number of I/Os that error should be hidden when calculating average time. I am not talking about microseconds, but I think fraction of millisecond should be realistic to get. > The fundamental problem is that on a busy system, getbinuptime() > does not get called at random times, it will be heavily affected > by the I/O traffic, because of the interrupts, the bus-traffic > itself, the cache-effects of I/O transfers and the context-switches > by the processes causing the I/O. I'm sorry, but I am not sure I understand above paragraphs. Do you want to say that in some realistic conditions (not counting entering debugger with disabled interrupts, etc) hardclock() can be delayed more then some significant percent of its period and that depends of I/O traffic itself? Or you want to say that disk I/Os somehow aliased with hardclock(), making impossible to hide error by averaging? > So yes, you can switch to getbinuptime(), but the only statistical > responsible way to do so, would be to supress latency measurements > on all I/O operations which complete in less than 5-10 timecounter > interrupts. Sure, getbinuptime() won't allow to answer how many requests completed within 0.5ms, but present API doesn't allow to calculate that any way, providing only total/average times. And why "_5-10_ timecounter interrupts"? > Apart from some practical issues implementing it, the numbers > that came out would be pretty useless. > > The right idea is probably to bucketize the latencies, so that > rather than having to keep track of devstat in real time to find > out, you could get a histogram at any time showing past > performance something like: > > Latency distribution: > > <5msec: 92.12 % > <10msec: 0.17 % > <20msec: 1.34 % > <50msec: 6.37 % > >50msec: 0.00 % > > Doing that with getbinuptime() would be statistically defensible > provided the top bucket is "<5msec" and it would very clearly tell > people if they have I/O trouble or not, which IMO is what people > want to know. > > The cost 20 64bit counters in struct devstat (N|R|W|E)*5*8 = 160 > bytes, but since devstat is already 288 bytes, that isn't a major > catastropy. I agree that such functionality could be interesting. The only worry is which buckets should be there. For modern HDDs above buckets could be fine. For high-end SSD it may go about microseconds then milliseconds. I have doubt that 5 buckets will be universal enough, unless separated by factor of 5-10. > The ability to measure latency precisly should be retained, but it > could be made a sysctl enabled debugging facility. > > The %busy crap should be killed, all it does is confuse people. I agree that it heavily lies, especially for cached writes, but at least it allows to make some very basic estimates. The value has valid explanation and the only problem is that users are misinterpreting it. -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 16 06:24:15 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5B9E13BE; Tue, 16 Apr 2013 06:24:15 +0000 (UTC) (envelope-from phk@freebsd.org) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id 244FCEF8; Tue, 16 Apr 2013 06:24:14 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id F205389FBE; Tue, 16 Apr 2013 06:24:13 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.6/8.14.6) with ESMTP id r3G6OD8N040153; Tue, 16 Apr 2013 06:24:13 GMT (envelope-from phk@freebsd.org) To: Alexander Motin Subject: Re: devstat overhead VS precision In-reply-to: <516C71BC.4000902@FreeBSD.org> From: "Poul-Henning Kamp" References: <51692C95.3010901@FreeBSD.org> <20130415184203.GA1839@garage.freebsd.pl> <516C515A.9090602@FreeBSD.org> <38496.1366058586@critter.freebsd.dk> <516C71BC.4000902@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1 Date: Tue, 16 Apr 2013 06:24:13 +0000 Message-ID: <40152.1366093453@critter.freebsd.dk> Cc: freebsd-hackers@FreeBSD.org, freebsd-geom@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2013 06:24:15 -0000 In message <516C71BC.4000902@FreeBSD.org>, Alexander Motin writes: >On 15.04.2013 23:43, Poul-Henning Kamp wrote: >> In message <516C515A.9090602@FreeBSD.org>, Alexander Motin writes: >> >> For tuning anything on a non-ridiculous SSD device or modern >> harddisks, it will be useless because of the bias you introduce is >> *not* one which averages out over many operations. > >Could you please explain why? > >> The fundamental problem is that on a busy system, getbinuptime() >> does not get called at random times, it will be heavily affected >> by the I/O traffic, because of the interrupts, the bus-traffic >> itself, the cache-effects of I/O transfers and the context-switches >> by the processes causing the I/O. > >I'm sorry, but I am not sure I understand above paragraphs. That was the exact explanation you asked for, and I'm not sure I can find a better way to explain it, but I'll try: Your assumption that the error will cancel out, implicitly assumes that the timestamp returned from getbinuptime() is updated at times which are totally independent from the I/O traffic you are trying to measure the latency of. That is not the case. The interrupt which updates getbinuptime()'s cached timestamp is affected a lot by the I/O traffic, for the various reasons I mention above. >Sure, getbinuptime() won't allow to answer how many requests completed >within 0.5ms, but present API doesn't allow to calculate that any way, >providing only total/average times. And why "_5-10_ timecounter interrupts"? A: Yes it actually does, a userland application running on a dedicated CPU core can poll the shared memory devstat structure at a very high rate and get very useful information about short latencies. Most people don't do that, becuase they don't care about the difference between 0.5 and 0.45 milliseconds. B: To get the systematic bias down to 10-20% of the measured interval. >> Latency distribution: >> >> <5msec: 92.12 % >> <10msec: 0.17 % >> <20msec: 1.34 % >> <50msec: 6.37 % >> >50msec: 0.00 % >> >I agree that such functionality could be interesting. The only worry is >which buckets should be there. For modern HDDs above buckets could be >fine. For high-end SSD it may go about microseconds then milliseconds. I >have doubt that 5 buckets will be universal enough, unless separated by >factor of 5-10. Remember what people use this for: Answering the question "Does my disk subsystem suck, and if so, how much" Buckets like the ones proposed will tell you that. >> The %busy crap should be killed, all it does is confuse people. > >I agree that it heavily lies, especially for cached writes, but at least >it allows to make some very basic estimates. For rotating disks: It always lies. For SSD: It almost always lies. Kill it. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 16 08:39:21 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B31E043E for ; Tue, 16 Apr 2013 08:39:21 +0000 (UTC) (envelope-from hps@bitfrost.no) Received: from mta.bitpro.no (mta.bitpro.no [92.42.64.202]) by mx1.freebsd.org (Postfix) with ESMTP id 77E9484D for ; Tue, 16 Apr 2013 08:39:20 +0000 (UTC) Received: from mail.bitfrost.no (mail.bitfrost.no [46.29.221.36]) by mta.bitpro.no (Postfix) with ESMTP id 730047A131; Tue, 16 Apr 2013 10:39:19 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at bitfrost.no Received: from laptop015.hselasky.homeunix.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: hanspetter) by mail.bitfrost.no (Postfix) with ESMTPSA id 66D6820CCD; Tue, 16 Apr 2013 10:39:17 +0200 (CEST) Message-ID: <516D0E8E.20807@bitfrost.no> Date: Tue, 16 Apr 2013 10:40:46 +0200 From: Hans Petter Selasky Organization: Bitfrost A/S MIME-Version: 1.0 To: Joshua Isom Subject: Re: USB Printer quirks References: <5169DFFB.4030101@gmail.com> <516BD94F.7030507@bitfrost.no> <516C5FC0.9060507@gmail.com> In-Reply-To: <516C5FC0.9060507@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2013 08:39:21 -0000 On 04/15/13 22:14, Joshua Isom wrote: > On 4/15/2013 5:41 AM, Hans Petter Selasky wrote: >> Hi, >> >> What does dmesg say about your printer. >> >> Is cups hooked up the correct /dev/uxxx device ? >> >> --HPS > > Here's what I got the last time I plugged it in. > > Apr 13 07:38:17 jri root: Unknown USB device: vendor 0x040a product > 0x4043 bus uhub7 > Apr 13 07:38:17 jri kernel: ugen8.2: at usbus8 > Apr 13 07:38:17 jri kernel: ulpt0: on usbus8 > Apr 13 07:38:17 jri kernel: ulpt0: using bi-directional mode > Apr 13 07:38:17 jri kernel: umass0: on usbus8 > Apr 13 07:38:17 jri kernel: umass0: SCSI over Bulk-Only; quirks = 0x0000 > Apr 13 07:38:17 jri kernel: umass0:14:0:-1: Attached to scbus14 > Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): got CAM status 0x50 > Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): fatal error, failed > to attach to device > Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): lost device - 0 > outstanding, 4 refs > Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): removing device entry > > The umass is probably for the card reader, I've never tried it with > FreeBSD. Cups would sometimes work, at most I'd be able to print one > document/test and then it'd hang. I would actually have to power cycle > the printer to try again. I think it was a problem reading from usb. > What I'm really curious about is why it would work properly in a linux > jail and not on FreeBSD. The closest hint I have is a kernel message > "linux: pid 8781 (usb): ioctl fd=5, cmd=0x604 ('^F',4) is not implemented". Hi, What software are you using to print? --HPS From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 16 21:13:21 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9844376D for ; Tue, 16 Apr 2013 21:13:21 +0000 (UTC) (envelope-from carl.shapiro@gmail.com) Received: from mail-qe0-f43.google.com (mail-qe0-f43.google.com [209.85.128.43]) by mx1.freebsd.org (Postfix) with ESMTP id 5FC0C179C for ; Tue, 16 Apr 2013 21:13:20 +0000 (UTC) Received: by mail-qe0-f43.google.com with SMTP id f6so554760qej.30 for ; Tue, 16 Apr 2013 14:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:from:date:message-id:subject:to :content-type; bh=u3AM3QBxTC6+TF2mVPpMuhkJHG1juHwEdvKXFt9pxRc=; b=mhrlg1jVtCyGu0JwZv+TNd3cki5xb93YrrKRqvgCxJFHnp+IvBsPXXE8bhIJVocNFR 38a+lPLlxRQOdgl00bpQnbSTvIAE3SBb+Yx2F9nAI1vplhrBtJ8OqW9ojOeD5XH30ugl 7hlWAC7w3werF7oYQVQXZkVYc1jGd8I5mqpdTc4v9Ag//A6K+eFAIKgmYv3YvVDMZPdK Lxsb44eRyZBvOpVhLZJje/88QNDq5JfhtTjKhW+NwVGAcWR7xU0eVJwNsBnpmbXLxeDA 3pwd8TLhtMU9pMaRnwF5kQLKhSZwqkUlZ1/v1o6jZdSL6rbzuAU4ttmpNesi8XmC/FoD dLEg== X-Received: by 10.224.41.200 with SMTP id p8mr4498082qae.99.1366146794692; Tue, 16 Apr 2013 14:13:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.49.130.225 with HTTP; Tue, 16 Apr 2013 14:12:54 -0700 (PDT) From: Carl Shapiro Date: Tue, 16 Apr 2013 14:12:54 -0700 Message-ID: Subject: MADV_FREE and wait4 EFAULT To: FreeBSD Hackers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2013 21:13:21 -0000 I am seeing wait4 system calls failing with an EFAULT and I am trying to understand what might be going wrong. An inspection of the wait4 implementation suggests the opportunity for EFAULT is within its invocations of copyout. In my situation, the status and rusage pointer arguments contain addresses to mmaped pages which have been madvised as MADV_FREE. Is it permissible to pass pages which have been madvised MADV_FREE to wait4 or any other system call for that matter? Might there be another opportunity for a wait4 to EFAULT? From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 16 21:51:30 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E4CE2696 for ; Tue, 16 Apr 2013 21:51:30 +0000 (UTC) (envelope-from jrisom@gmail.com) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) by mx1.freebsd.org (Postfix) with ESMTP id B8A6F1A5D for ; Tue, 16 Apr 2013 21:51:30 +0000 (UTC) Received: by mail-ie0-f170.google.com with SMTP id c11so1182835ieb.15 for ; Tue, 16 Apr 2013 14:51:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=xIoXpt9So+CJEQNvyUV4CwEOntWNRvOknIwyJFT8Ht4=; b=pVW/Ju0BPaVJbQ4EGucyX3VDfIA8wxqtRUyqP8hsVgcqNVmxXvCGjrYYrMiHL+UcZf uOXlxc2YpyUvNR7qFtMVNLyaULDBmbduu7CatgRvatERHwMxVOR8chkf8dFGz/bGzEmA lR2xOCgo6zujv96jGTRD5j52510qoOVU00/L3DDWXXK4TFJ7ZJnhN78gH3JxG2POWZKV mMT8HY/EOJauNdalVcoD/3+uSVJ02r8Y/v21M4QLyb/GFMgjUCZA8GQCAFbJESyeQeyB u7HhcM1xkwnrvb4ZlfqztuS5ZPiBS6p4tjVjJg3TEvj9BgqTpmrXsXy0tH6GPnlrOH4A Enww== X-Received: by 10.50.20.38 with SMTP id k6mr2497826ige.72.1366149090444; Tue, 16 Apr 2013 14:51:30 -0700 (PDT) Received: from [192.168.1.34] (c-98-212-197-211.hsd1.il.comcast.net. [98.212.197.211]) by mx.google.com with ESMTPS id w7sm2106383igl.4.2013.04.16.14.51.29 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 16 Apr 2013 14:51:29 -0700 (PDT) Message-ID: <516DC7DB.40203@gmail.com> Date: Tue, 16 Apr 2013 16:51:23 -0500 From: Joshua Isom User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Hans Petter Selasky Subject: Re: USB Printer quirks References: <5169DFFB.4030101@gmail.com> <516BD94F.7030507@bitfrost.no> <516C5FC0.9060507@gmail.com> <516D0E8E.20807@bitfrost.no> In-Reply-To: <516D0E8E.20807@bitfrost.no> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2013 21:51:31 -0000 On 4/16/2013 3:40 AM, Hans Petter Selasky wrote: > On 04/15/13 22:14, Joshua Isom wrote: >> On 4/15/2013 5:41 AM, Hans Petter Selasky wrote: >>> Hi, >>> >>> What does dmesg say about your printer. >>> >>> Is cups hooked up the correct /dev/uxxx device ? >>> >>> --HPS >> >> Here's what I got the last time I plugged it in. >> >> Apr 13 07:38:17 jri root: Unknown USB device: vendor 0x040a product >> 0x4043 bus uhub7 >> Apr 13 07:38:17 jri kernel: ugen8.2: at usbus8 >> Apr 13 07:38:17 jri kernel: ulpt0: on usbus8 >> Apr 13 07:38:17 jri kernel: ulpt0: using bi-directional mode >> Apr 13 07:38:17 jri kernel: umass0: on usbus8 >> Apr 13 07:38:17 jri kernel: umass0: SCSI over Bulk-Only; quirks = 0x0000 >> Apr 13 07:38:17 jri kernel: umass0:14:0:-1: Attached to scbus14 >> Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): got CAM status 0x50 >> Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): fatal error, failed >> to attach to device >> Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): lost device - 0 >> outstanding, 4 refs >> Apr 13 07:38:17 jri kernel: (da0:umass-sim0:0:0:0): removing device entry >> >> The umass is probably for the card reader, I've never tried it with >> FreeBSD. Cups would sometimes work, at most I'd be able to print one >> document/test and then it'd hang. I would actually have to power cycle >> the printer to try again. I think it was a problem reading from usb. >> What I'm really curious about is why it would work properly in a linux >> jail and not on FreeBSD. The closest hint I have is a kernel message >> "linux: pid 8781 (usb): ioctl fd=5, cmd=0x604 ('^F',4) is not >> implemented". > > Hi, > > What software are you using to print? > > --HPS Just using cups' own test page would cause the issue. I tried to keep testing simple to minimize problems. The driver's available on sourceforge, c2esp and is untested on FreeBSD by the developer. I had trouble with versions 1.6, and the current 2.6. From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 16 22:04:45 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 27E2D9B6 for ; Tue, 16 Apr 2013 22:04:45 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ea0-x236.google.com (mail-ea0-x236.google.com [IPv6:2a00:1450:4013:c01::236]) by mx1.freebsd.org (Postfix) with ESMTP id BD93B1AF7 for ; Tue, 16 Apr 2013 22:04:44 +0000 (UTC) Received: by mail-ea0-f182.google.com with SMTP id q15so460903ead.41 for ; Tue, 16 Apr 2013 15:04:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:message-id:date:from:user-agent:mime-version:to :subject:content-type:content-transfer-encoding; bh=JvQiIC98EoeWDPQME84XQFXY4bjvOwpFKKRJ6LZ16KM=; b=JCYuN7Vvb/j7i34RUppG4hl0teJdULimAdR0WNBeupKLt1gdGdhJ3z+8LKbOU6mdzq PSEy7pEAPSWw3Y9upKPeRfvj5AgfIVD6w1eZhz9LXqdmW2J0Ds1L3unFmg4ueg/qzNPT UqAOWZUFgiDz0zUyV4yH/ZjymkGMPb0F2WoS12i87u9tA4BZXBw6RqfyBVl7S0bptsLA 31esnAcJ4U9JM9sC0HqLwo3AP49eqBt72qcVg5NTRxAkivzjByefjAAOOk5XxHv+0Xoi YQpvSq5pcD/Ujj7w967iFbf8zeKJprvM32FvUZgU6/f0PgWfM2M9bwmTWLxzOkvzmBa7 RfDw== X-Received: by 10.14.182.72 with SMTP id n48mr11091388eem.3.1366149883888; Tue, 16 Apr 2013 15:04:43 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPS id s47sm5133559eeg.8.2013.04.16.15.04.41 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 16 Apr 2013 15:04:42 -0700 (PDT) Sender: Alexander Motin Message-ID: <516DCAF7.20400@FreeBSD.org> Date: Wed, 17 Apr 2013 01:04:39 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.org Subject: Synchronizing TSC Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2013 22:04:45 -0000 Hi. Recently I've got 6-core/12-thread system on Sandy Bridge-E Core i7-3930K CPU and was unpleasantly surprised to see that TSCs are not synchronized there. While all 11 APs were synchronized, BSP was far behind them. Since it is single-socket system, I don't know any good reason for such behavior except some BIOS bug. But I've recalled that somewhere was some discussions about possible TSC synchronization. I've implemented patch below that allows to adjust TSC values of BSPs to AP's one on boot using CPU MSRs, hoping that they should not diverge after that: http://people.freebsd.org/~mav/tsc_adj2.patch I don't know very much about all different TSC hardware to predict when it is safe to enable the functionality, but at least on my system being enabled via loader tunable it seems working well. Comments? -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 16 23:28:34 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4124928B for ; Tue, 16 Apr 2013 23:28:34 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) by mx1.freebsd.org (Postfix) with ESMTP id CFCD21E30 for ; Tue, 16 Apr 2013 23:28:33 +0000 (UTC) Received: by mail-wi0-f177.google.com with SMTP id hm14so1002540wib.4 for ; Tue, 16 Apr 2013 16:28:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=x-received:date:from:to:subject:message-id:in-reply-to:references :x-mailer:mime-version:content-type:content-transfer-encoding; bh=FsUABFXg3iEsZaM60fYuSZFlnPdc5ZNzPLspwq2Cb/I=; b=n9VHViOc65CU84l5tLaLtrTC5rSiUcY06WblCPX+bqgMT23ycZu9dglg44A7h9zFB5 XKtBmLqBoJW+7iq2k5M32zuZBfIiAEBx+rzDeppMzH0W1GsmdRnFOV6plxxO3qrRwOeb nXVxMuJYGSePUXtpLPziPTXNN+wAN0rfWFsZ5gjmqrPSsQkXsB1tcznDAYEhTjt8/d2V qSe3rZnDyjQX0U91KfmSQK/lO+3SD8VAv5AOpQmNi9z8Oua+OHIbnaulz9JZSCBF2lnS BhLb08rSaWq0OAoOE83ufmDMBk6z8HvzTRaN2P/5uVEakRMQn84feHHlNNZw6Q0OKIfo MrRw== X-Received: by 10.194.242.163 with SMTP id wr3mr7279333wjc.35.1366154912931; Tue, 16 Apr 2013 16:28:32 -0700 (PDT) Received: from gumby.homeunix.com (87-194-105-247.bethere.co.uk. [87.194.105.247]) by mx.google.com with ESMTPS id k5sm6111727wiy.5.2013.04.16.16.28.30 (version=SSLv3 cipher=RC4-SHA bits=128/128); Tue, 16 Apr 2013 16:28:32 -0700 (PDT) Date: Wed, 17 Apr 2013 00:28:28 +0100 From: RW To: freebsd-hackers@freebsd.org Subject: Re: MADV_FREE and wait4 EFAULT Message-ID: <20130417002828.2f5a0896@gumby.homeunix.com> In-Reply-To: References: X-Mailer: Claws Mail 3.9.0 (GTK+ 2.24.17; amd64-portbld-freebsd9.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2013 23:28:34 -0000 On Tue, 16 Apr 2013 14:12:54 -0700 Carl Shapiro wrote: > I am seeing wait4 system calls failing with an EFAULT and I am trying > to understand what might be going wrong. > > An inspection of the wait4 implementation suggests the opportunity > for EFAULT is within its invocations of copyout. In my situation, > the status and rusage pointer arguments contain addresses to mmaped > pages which have been madvised as MADV_FREE. > > Is it permissible to pass pages which have been madvised MADV_FREE to > wait4 or any other system call for that matter? Might there be > another opportunity for a wait4 to EFAULT? IIRC MADV_FREE pages are marked clean and placed on the end of the inactive queue as "low hanging fruit" for the page daemon. AFAIK they're no different to any other clean page. Malloc'ed memory pages are commonly in that state. From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 17 00:25:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 6D97EA6E; Wed, 17 Apr 2013 00:25:10 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-ee0-f52.google.com (mail-ee0-f52.google.com [74.125.83.52]) by mx1.freebsd.org (Postfix) with ESMTP id DC0811FC6; Wed, 17 Apr 2013 00:25:09 +0000 (UTC) Received: by mail-ee0-f52.google.com with SMTP id b57so326984eek.25 for ; Tue, 16 Apr 2013 17:25:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=tklgdmI2e1nLMRrFm1D9XUtlzGHV2QbplP2dodQrXF0=; b=E5G+pN9m/fz9AWrEuma0YMzF0wzA7BqfVA1aIqrf3fbsUdpp38Kok6M+SsAFsS0xbg j3iUXjNA5iYyVib2s65zVYaMwS4P8l4Mm8mjHvC1oJg6iT5emxgQRex+8WbDLWhm9lns fvHjmnHEoxemSgtkurIpshIHwgaH/L8qsDfQeJXzDEDC1U5ofy7kPJ2nP87KwvMLyZEW CxuVkGuWivNzjr6YsJmm1Np/VyPqykXcihJbx7i+XSblh52TGJEdRw5UtQZYDdZg8kc8 9CSQNmLoqM9w/kyJzP5NzZaMqA2fnxxVfwkCK7IlhkxGBSglLTNoZ+b6qcj4gR6tOpO7 P3Nw== MIME-Version: 1.0 X-Received: by 10.14.39.5 with SMTP id c5mr12008234eeb.27.1366158302767; Tue, 16 Apr 2013 17:25:02 -0700 (PDT) Received: by 10.14.111.2 with HTTP; Tue, 16 Apr 2013 17:25:02 -0700 (PDT) In-Reply-To: <516DCAF7.20400@FreeBSD.org> References: <516DCAF7.20400@FreeBSD.org> Date: Tue, 16 Apr 2013 17:25:02 -0700 Message-ID: Subject: Re: Synchronizing TSC From: Jim Harris To: Alexander Motin Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Apr 2013 00:25:10 -0000 On Tue, Apr 16, 2013 at 3:04 PM, Alexander Motin wrote: > Hi. > > Recently I've got 6-core/12-thread system on Sandy Bridge-E Core i7-3930K > CPU and was unpleasantly surprised to see that TSCs are not synchronized > there. While all 11 APs were synchronized, BSP was far behind them. Since > it is single-socket system, I don't know any good reason for such behavior > except some BIOS bug. But I've recalled that somewhere was some discussions > about possible TSC synchronization. I've implemented patch below that > allows to adjust TSC values of BSPs to AP's one on boot using CPU MSRs, > hoping that they should not diverge after that: > http://people.freebsd.org/~**mav/tsc_adj2.patch > > I don't know very much about all different TSC hardware to predict when it > is safe to enable the functionality, but at least on my system being > enabled via loader tunable it seems working well. > > Comments? > > You may be remembering this thread on r238755 last year: http://lists.freebsd.org/pipermail/svn-src-head/2012-July/038992.html This was a bug fix in the TSC synchronization test code though, not anything for trying to adjust out-of-sync TSCs. The Intel SDM (volume 3, section 17.13 of March 2013 revision) says earlier models can only write to lower 32 bits of IA32_TIME_STAMP_COUNTER, but these models also should not have invariant TSC so they would never even get to your new routine. So your patch seems OK for Intel CPUs, at least as a tunable that is disabled by default. My only concern would be why TSC on the BSP started out-of-sync on your system. Theoretically, BIOS could adjust TSCs in SMM to try to hide SMI code execution from the OS, which could then make them out-of-sync again. Not sure if that's what's happening here, but might be worth a test putting the TSC test code on a periodic timer to see if they ever get out of sync again. -Jim From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 17 06:53:08 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B7A9CFCB for ; Wed, 17 Apr 2013 06:53:08 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ea0-x230.google.com (mail-ea0-x230.google.com [IPv6:2a00:1450:4013:c01::230]) by mx1.freebsd.org (Postfix) with ESMTP id 54969EF3 for ; Wed, 17 Apr 2013 06:53:08 +0000 (UTC) Received: by mail-ea0-f176.google.com with SMTP id h10so548238eaj.7 for ; Tue, 16 Apr 2013 23:53:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=q5dKrG2KjVrvtji1z59W3IqMT5iUHs4/9/y2C0ViPC4=; b=tT4gKGxGhPiEm0GxOfsBiaotRyx+g5918UmqFoiV/rXtaaQBSDUx8HxVT+0wSRUvFa LAcVb9WCYpjErsV/2Wl2F+0KeN+M/WcZBMdry/FZtqkbyN93OtHGdWBym4sVRT0rh6GM zvUGsqn3dv8ctLRFnNXRMR8sZS0mwTHNDUWKCjvO6I+YL1n+g4X2q8S383KycOrUe7+2 zdXmpTu9WgrMA6bD7XRJOKA6BRDlT/GmwC9g4U+Mpf2NYLqGNnfVq/PDwRrUEZzDMKVl scTT5LlqLGnw7Xs2IJR4yzuRp4Xn55+Q314m6FlxjUmrtRGkzlFeqkId0im4lUNZmeDj g61g== X-Received: by 10.15.99.201 with SMTP id bl49mr14716983eeb.43.1366181178391; Tue, 16 Apr 2013 23:46:18 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPS id b5sm6703242eew.16.2013.04.16.23.46.16 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 16 Apr 2013 23:46:17 -0700 (PDT) Sender: Alexander Motin Message-ID: <516E4537.7050205@FreeBSD.org> Date: Wed, 17 Apr 2013 09:46:15 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Jim Harris Subject: Re: Synchronizing TSC References: <516DCAF7.20400@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Apr 2013 06:53:08 -0000 On 17.04.2013 03:25, Jim Harris wrote: > > On Tue, Apr 16, 2013 at 3:04 PM, Alexander Motin > wrote: > > Hi. > > Recently I've got 6-core/12-thread system on Sandy Bridge-E Core > i7-3930K CPU and was unpleasantly surprised to see that TSCs are not > synchronized there. While all 11 APs were synchronized, BSP was far > behind them. Since it is single-socket system, I don't know any good > reason for such behavior except some BIOS bug. But I've recalled > that somewhere was some discussions about possible TSC > synchronization. I've implemented patch below that allows to adjust > TSC values of BSPs to AP's one on boot using CPU MSRs, hoping that > they should not diverge after that: > http://people.freebsd.org/~__mav/tsc_adj2.patch > > > I don't know very much about all different TSC hardware to predict > when it is safe to enable the functionality, but at least on my > system being enabled via loader tunable it seems working well. > > Comments? > > > You may be remembering this thread on r238755 last year: > > http://lists.freebsd.org/pipermail/svn-src-head/2012-July/038992.html > > This was a bug fix in the TSC synchronization test code though, not > anything for trying to adjust out-of-sync TSCs. I remember that thread, but I think I've seen somebody told somewhere that it could be interesting to implement some MI mechanism. Never mind. > The Intel SDM (volume 3, section 17.13 of March 2013 revision) says > earlier models can only write to lower 32 bits of > IA32_TIME_STAMP_COUNTER, but these models also should not have invariant > TSC so they would never even get to your new routine. So your patch > seems OK for Intel CPUs, at least as a tunable that is disabled by default. Thanks. > My only concern would be why TSC on the BSP started out-of-sync on your > system. Theoretically, BIOS could adjust TSCs in SMM to try to hide SMI > code execution from the OS, which could then make them out-of-sync > again. Not sure if that's what's happening here, but might be worth a > test putting the TSC test code on a periodic timer to see if they ever > get out of sync again. I did one more interesting observation: on every reboot drift between BSP and APs is growing proportionally to the previous system power-on time. On first boot it is -3878361036 (just above one second), after reboot some minutes later it is -1123454492776 (about 6 minutes), after another reboot it is -1853033521804 (about 10 minutes). Unless my adjustment code would be active, I would guess that AP's TSC is running linearly while BSP's for some reason reset to zero on every reboot. But since I am synchronizing them on each boot, the only possibility for it I see is that there is some other timer(s) / counter(s) not affected by MSR writes that ticks linearly and reloading AP's TSC, but for some reason not reloading BSP's. -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 17 08:21:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 81B09E72 for ; Wed, 17 Apr 2013 08:21:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id F0355302 for ; Wed, 17 Apr 2013 08:21:47 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r3H8LhMt007492; Wed, 17 Apr 2013 11:21:43 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.2 kib.kiev.ua r3H8LhMt007492 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r3H8LhjT007491; Wed, 17 Apr 2013 11:21:43 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 17 Apr 2013 11:21:43 +0300 From: Konstantin Belousov To: Carl Shapiro Subject: Re: MADV_FREE and wait4 EFAULT Message-ID: <20130417082143.GW2930@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="JZV+hE4cb0ibQYPB" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Apr 2013 08:21:48 -0000 --JZV+hE4cb0ibQYPB Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 16, 2013 at 02:12:54PM -0700, Carl Shapiro wrote: > I am seeing wait4 system calls failing with an EFAULT and I am trying to > understand what might be going wrong. >=20 > An inspection of the wait4 implementation suggests the opportunity for > EFAULT is within its invocations of copyout. In my situation, the status > and rusage pointer arguments contain addresses to mmaped pages which have > been madvised as MADV_FREE. >=20 > Is it permissible to pass pages which have been madvised MADV_FREE to wai= t4 > or any other system call for that matter? Might there be another > opportunity for a wait4 to EFAULT? Did you ensured with e.g. ktrace and procstat -v that your assumptions hold, i.e. the addresses supplied as wait4(2) arguments are valid ? Please provide the minimal test case demonstrating the behaviour. MADV_FREE should only result in the possible lost of the previous content of the page, not in the faulting of the page access. From the inspection of the code, I do not see how MADV_FREE could result in the memory address becoming invalid. --JZV+hE4cb0ibQYPB Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRbluXAAoJEJDCuSvBvK1BiCgP/jCSkpdxMFcz6C4FIu5Itihe 6fAlhZ/TyDuxnWvRLAIUI+JiJ3JyxRXgImlixzvMnk3GO9scVaknHooztaLmFtmr UkWceD9zfBNJEtYqEtrKLwKYej2uIj4GUr0Ohh48/jTiqO8t7g3dUMdSxzByXzBp kbP5UOmMBfARb6dFCKkTYW5aUd5Ma8Y9WQhiUdj2iFJBIi2QSi06GMPsAmigtiy+ d4rtAE2B+Ci3qS/R9WEd+AHFKvIvNvZ29S2eXHUIAD1bTaX/ulz2uWrX+p4SQC0M PfGUC+zPML5Btrv8fDfM7T6oompQ7PBjUGi8Dt7pG8fb8R0xMtgwDkNY1VeyJ9uG 468Zot/dcmhVuc9rzWOHRgjjf8rFaDeGIHgDrOo75XYwJHlO42Nzqo2xEWo4ZM2z Cied4/Ra9OvmdrgyXk6jNuPemIvjD+xfNvhAvLasYUQQe0YycBOuiybSGJFaRb60 kF2RPVRQfkzWem63JN30BnvBI5WKeFmmI3WTdjMbHm0lQwEKfF00Pv6bOTjSxvjv uHdD/U/hIKP2TzuXGcb2W35NDjr2QDzbwwrx1lkoS5EoN2PRbvXtdVLqwPqnZqlY gGr4zuEv5k9HhUiHqRL65BUJOpgtY6fk9J+eBxgrK36IMnUq/QEcHm7zEbV1wANl wTnLy1WvvRnyIw+senit =MPJi -----END PGP SIGNATURE----- --JZV+hE4cb0ibQYPB-- From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 17 08:50:57 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 565DD92D; Wed, 17 Apr 2013 08:50:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id C0E6761A; Wed, 17 Apr 2013 08:50:56 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r3H8oqWe013232; Wed, 17 Apr 2013 11:50:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.2 kib.kiev.ua r3H8oqWe013232 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r3H8oqM5013231; Wed, 17 Apr 2013 11:50:52 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 17 Apr 2013 11:50:52 +0300 From: Konstantin Belousov To: Alexander Motin Subject: Re: Synchronizing TSC Message-ID: <20130417085052.GZ2930@kib.kiev.ua> References: <516DCAF7.20400@FreeBSD.org> <516E4537.7050205@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="sR4wgT97u2q5hWcW" Content-Disposition: inline In-Reply-To: <516E4537.7050205@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: "freebsd-hackers@freebsd.org" , Jim Harris X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Apr 2013 08:50:57 -0000 --sR4wgT97u2q5hWcW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Apr 17, 2013 at 09:46:15AM +0300, Alexander Motin wrote: > On 17.04.2013 03:25, Jim Harris wrote: > > > > On Tue, Apr 16, 2013 at 3:04 PM, Alexander Motin > > wrote: > > > > Hi. > > > > Recently I've got 6-core/12-thread system on Sandy Bridge-E Core > > i7-3930K CPU and was unpleasantly surprised to see that TSCs are not > > synchronized there. While all 11 APs were synchronized, BSP was far > > behind them. Since it is single-socket system, I don't know any good > > reason for such behavior except some BIOS bug. But I've recalled > > that somewhere was some discussions about possible TSC > > synchronization. I've implemented patch below that allows to adjust > > TSC values of BSPs to AP's one on boot using CPU MSRs, hoping that > > they should not diverge after that: > > http://people.freebsd.org/~__mav/tsc_adj2.patch > > > > > > I don't know very much about all different TSC hardware to predict > > when it is safe to enable the functionality, but at least on my > > system being enabled via loader tunable it seems working well. > > > > Comments? > > > > > > You may be remembering this thread on r238755 last year: > > > > http://lists.freebsd.org/pipermail/svn-src-head/2012-July/038992.html > > > > This was a bug fix in the TSC synchronization test code though, not > > anything for trying to adjust out-of-sync TSCs. >=20 > I remember that thread, but I think I've seen somebody told somewhere=20 > that it could be interesting to implement some MI mechanism. Never mind. >=20 > > The Intel SDM (volume 3, section 17.13 of March 2013 revision) says > > earlier models can only write to lower 32 bits of > > IA32_TIME_STAMP_COUNTER, but these models also should not have invariant > > TSC so they would never even get to your new routine. So your patch > > seems OK for Intel CPUs, at least as a tunable that is disabled by defa= ult. >=20 > Thanks. >=20 > > My only concern would be why TSC on the BSP started out-of-sync on your > > system. Theoretically, BIOS could adjust TSCs in SMM to try to hide SMI > > code execution from the OS, which could then make them out-of-sync > > again. Not sure if that's what's happening here, but might be worth a > > test putting the TSC test code on a periodic timer to see if they ever > > get out of sync again. >=20 > I did one more interesting observation: on every reboot drift between=20 > BSP and APs is growing proportionally to the previous system power-on=20 > time. On first boot it is -3878361036 (just above one second), after=20 > reboot some minutes later it is -1123454492776 (about 6 minutes), after= =20 > another reboot it is -1853033521804 (about 10 minutes). >=20 > Unless my adjustment code would be active, I would guess that AP's TSC=20 > is running linearly while BSP's for some reason reset to zero on every=20 > reboot. But since I am synchronizing them on each boot, the only=20 > possibility for it I see is that there is some other timer(s) /=20 > counter(s) not affected by MSR writes that ticks linearly and reloading= =20 > AP's TSC, but for some reason not reloading BSP's. For me it sounds as the BIOS bug, indeed. Could you verify the content of IA32_TSC_ADJUST on all cores (I believe it is present on E5) ? Also, using TSC_ADJUST to correct the skew seems to be preferrable, according to the Intel docs. Why do you use cpuid in the assembly sequence ? As I understand, you ensure that there is a serialization point, but why do you need it ? The common knowledge is that for CPUs with invariant TSC, the TSC counter is single-instance and located on uncore. For single-socket configurations, your patch would be fine. But, for multi-socket machines, each package has its own counter, and counters might drift. As result, the initial synchronization would still allow the eventual de-sync and this is problematic. --sR4wgT97u2q5hWcW Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRbmJrAAoJEJDCuSvBvK1BnpQP/Ao3ez1ajXqJ/QmY+9rwEe5z 261Yru3pwdJxR0Og3zdEeUOc7zUURTVBm1JB6rwoklalDTPosiQw9/PcWgaV/Dr/ M0Qy81K7cq6pz0Mpo2iztI8hqcOqt8nA9D3An6fqp+FgCOVKhk8IpwtMf1EblVaf ZrJKIev6sgQ4hiDHrHMbbJnfeiUQxdwxpxdWPfIyugtjRUvx1cFAwJDkGHVuWuPW +4GIusv38UHeoJ205+MJPoQ/lqalvWb+AsRlaG75gWz2DwqFPeS48PjRIa/2bnUW e3/4EFunGUHQUeyQwOAWtV23isPYJ65NAaYM5MotwY+WL0vqWnlETPFr+ix6wXtY t3A1gJVXMhjuGEeMvAG60VBE3CMDhzKUWdC0kf46X9PlFIu6+LE61HP/HcJjVrDj wNVaqsx7mMFUHGkjHE7Py8YoYAfZ2tscuI83uO499HoBWCBFwe3AfrTpPd4rE6OR 8QoMyx+ZuKHwXkIplwNxHaaVOoHBkmY5RIThIvBSCyQJNMlnmKHEJFF70aofFXzk 6I6ltIKyZCnaZXXjL5vCC0MszdAH0gInscqPMv2QoGXojNjFOUeWfq/lADdrvFAE JskmfmgJwC/jRGhDpqYjkCsuJeLcdMMvQTuqx9BNVz1HLYg32+J6/GwCIE/OURYV 5m+48QBNnTClt3U+teGd =JkPB -----END PGP SIGNATURE----- --sR4wgT97u2q5hWcW-- From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 11:37:30 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BC7D59B7 for ; Thu, 18 Apr 2013 11:37:30 +0000 (UTC) (envelope-from shonnur@chelsio.com) Received: from stargate.chelsio.com (stargate.chelsio.com [67.207.112.58]) by mx1.freebsd.org (Postfix) with ESMTP id A295D3E7 for ; Thu, 18 Apr 2013 11:37:30 +0000 (UTC) Received: from maui.asicdesigners.com (maui.asicdesigners.com [10.192.180.15]) by stargate.chelsio.com (8.13.1/8.13.1) with SMTP id r3IBbUAA026060 for ; Thu, 18 Apr 2013 04:37:30 -0700 Received: from corona.asicdesigners.com ([10.192.160.6]) by maui.asicdesigners.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 18 Apr 2013 04:37:29 -0700 Received: from NICE.asicdesigners.com (10.192.160.7) by corona.asicdesigners.com (10.192.160.6) with Microsoft SMTP Server (TLS) id 8.3.83.0; Thu, 18 Apr 2013 04:37:29 -0700 Received: from NICE.asicdesigners.com ([fe80::51b2:ba95:9d72:babc]) by nice.asicdesigners.com ([fe80::51b2:ba95:9d72:babc%15]) with mapi id 14.02.0247.003; Thu, 18 Apr 2013 04:37:29 -0700 From: Sreenivasa Honnur To: "freebsd-hackers@freebsd.org" Subject: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) Thread-Topic: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) Thread-Index: Ac48KRnQ60GQkgjzQL+SWp8/cIeQxg== Date: Thu, 18 Apr 2013 11:37:28 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.193.190.128] MIME-Version: 1.0 X-OriginalArrivalTime: 18 Apr 2013 11:37:29.0889 (UTC) FILETIME=[1BD98910:01CE3C29] X-Mailman-Approved-At: Thu, 18 Apr 2013 11:56:34 +0000 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 11:37:30 -0000 I have a ipv6 interface(ping6 to a remove ipv6 works) when I try to bind to= this address through a socket program "sobind" fails with "49" as return v= alue. If I give "saddr6.sin6_addr =3D in6addr_any;" sobind works. Any idea what could be going wrong here? roundhay# ifconfig cxgbe1 cxgbe1: flags=3D8843 metric 0 mtu 1= 500 options=3D6c07bb ether 00:07:43:11:89:88 inet6 2010::102 prefixlen 64 inet6 fe80::207:43ff:fe11:8988%cxgbe1 prefixlen 64 scopeid 0xd nd6 options=3D21 media: Ethernet 10Gbase-SR status: active roundhay# ping6 2010::101 PING6(56=3D40+8+8 bytes) 2010::102 --> 2010::101 16 bytes from 2010::101, icmp_seq=3D0 hlim=3D64 time=3D0.950 ms 16 bytes from 2010::101, icmp_seq=3D1 hlim=3D64 time=3D0.158 ms ^C --- 2010::101 ping6 statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/std-dev =3D 0.158/0.554/0.950/0.396 ms Code: =3D=3D=3D=3D=3D { rv =3D socreate(AF_INET6, &sock, SOCK_STREAM, IPPROTO_TCP, td->td_ucred, td); if (rv !=3D 0) { os_log_error("sock create ipv6 %s failed %d.\n", tbuf, rv); return NULL; } family =3D saddr6.sin6_family =3D AF_INET6; inet_pton(AF_INET6, "2010::102", &saddr6.sin6_addr); saddr6.sin6_port =3D htons(ep->port); saddr6.sin6_len =3D sizeof(struct sockaddr_in6); rv =3D sobind(sock, (struct sockaddr *)&saddr6, td); } Here is the code snippet from where EADDRNOTAVAIL is returned. File : netinet6/in6_pcb.c in6_pcbbind(register struct inpcb *inp, struct sockaddr *nam, struct ucred *cred) { ........ ........ } else if (!IN6_IS_ADDR_UNSPECIFIED(&sin6->sin6_addr)) { struct ifaddr *ifa; sin6->sin6_port =3D 0; /* yech... */ if ((ifa =3D ifa_ifwithaddr((struct sockaddr *)sin6= )) =3D=3D NULL && (inp->inp_flags & INP_BINDANY) =3D=3D 0) { return (EADDRNOTAVAIL); =3D=3D> it fails he= re since ifa_ifwithaddr() returns NULL } ........ ........ } File: net/if.c struct ifaddr * ifa_ifwithaddr(struct sockaddr *addr) { return (ifa_ifwithaddr_internal(addr, 1)); } ifa_ifwithaddr_internal(struct sockaddr *addr, int getref) { ........ ........ TAILQ_FOREACH(ifp, &V_ifnet, if_link) { IF_ADDR_RLOCK(ifp); printf("ifp->xname:%s ifp->dname:%s\n",ifp->if_xname, ifp->= if_dname); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family !=3D addr->sa_family) = =3D=3D> ifa->ifa_addr->sa_family=3D18 & addr->sa_family=3D28. Even though = the interface is in IPV6 mode its sa_family is not set properly, continue; } Ifa =3D NULL; ........ ........ } From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 12:03:10 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 7A602495 for ; Thu, 18 Apr 2013 12:03:10 +0000 (UTC) (envelope-from shonnur@chelsio.com) Received: from stargate.chelsio.com (stargate.chelsio.com [67.207.112.58]) by mx1.freebsd.org (Postfix) with ESMTP id 606B0880 for ; Thu, 18 Apr 2013 12:03:10 +0000 (UTC) Received: from maui.asicdesigners.com (maui.asicdesigners.com [10.192.180.15]) by stargate.chelsio.com (8.13.1/8.13.1) with SMTP id r3IC3AwH026229 for ; Thu, 18 Apr 2013 05:03:10 -0700 Received: from corona.asicdesigners.com ([10.192.160.6]) by maui.asicdesigners.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 18 Apr 2013 05:03:10 -0700 Received: from NICE.asicdesigners.com (10.192.160.7) by corona.asicdesigners.com (10.192.160.6) with Microsoft SMTP Server (TLS) id 8.3.83.0; Thu, 18 Apr 2013 05:03:09 -0700 Received: from NICE.asicdesigners.com ([fe80::51b2:ba95:9d72:babc]) by nice.asicdesigners.com ([fe80::51b2:ba95:9d72:babc%15]) with mapi id 14.02.0247.003; Thu, 18 Apr 2013 05:03:09 -0700 From: Sreenivasa Honnur To: "freebsd-hackers@freebsd.org" Subject: IPv6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) Thread-Topic: IPv6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) Thread-Index: Ac48LK8jT/jcUDbnSui9QRC6r7ExZA== Date: Thu, 18 Apr 2013 12:03:07 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.193.190.128] MIME-Version: 1.0 X-OriginalArrivalTime: 18 Apr 2013 12:03:10.0066 (UTC) FILETIME=[B1DDE520:01CE3C2C] Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 12:03:10 -0000 I have a ipv6 interface(ping6 to a remove ipv6 works) when I try to bind to= this address through a socket program "sobind" fails with "49" as return v= alue. If I give "saddr6.sin6_addr =3D in6addr_any;" sobind works. Any idea what could be going wrong here? roundhay# ifconfig cxgbe1 cxgbe1: flags=3D8843 metric 0 mtu 1= 500 options=3D6c07bb ether 00:07:43:11:89:88 inet6 2010::102 prefixlen 64 inet6 fe80::207:43ff:fe11:8988%cxgbe1 prefixlen 64 scopeid 0xd nd6 options=3D21 media: Ethernet 10Gbase-SR status: active roundhay# ping6 2010::101 PING6(56=3D40+8+8 bytes) 2010::102 --> 2010::101 16 bytes from 2010::101, icmp_seq=3D0 hlim=3D64 time=3D0.950 ms 16 bytes from 2010::101, icmp_seq=3D1 hlim=3D64 time=3D0.158 ms ^C --- 2010::101 ping6 statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/= avg/max/std-dev =3D 0.158/0.554/0.950/0.396 ms Code: =3D=3D=3D=3D=3D { rv =3D socreate(AF_INET6, &sock, SOCK_STREAM, IPPROTO_TCP, td->td_ucred, td); if (rv !=3D 0) { os_log_error("sock create ipv6 %s failed %d.\n", tbuf, rv); return NULL; } family =3D saddr6.sin6_family =3D AF_INET6; inet_pton(AF_INET6, "2010::102", &saddr6.sin6_addr); saddr6.sin6_port =3D htons(ep->port); saddr6.sin6_len =3D sizeof(struct sockaddr_in6); rv =3D sobind(sock, (struct sockaddr *)&saddr6, td); } Here is the code snippet from where EADDRNOTAVAIL is returned. File : netinet6/in6_pcb.c in6_pcbbind(register struct inpcb *inp, struct sockaddr *nam, struct ucred *cred) { ........ ........ } else if (!IN6_IS_ADDR_UNSPECIFIED(&sin6->sin6_addr)) { struct ifaddr *ifa; sin6->sin6_port =3D 0; /* yech... */ if ((ifa =3D ifa_ifwithaddr((struct sockaddr *)sin6= )) =3D=3D NULL && (inp->inp_flags & INP_BINDANY) =3D=3D 0) { return (EADDRNOTAVAIL); =E8 it fails here s= ince ifa_ifwithaddr() returns NULL } ........ ........ } File: net/if.c struct ifaddr * ifa_ifwithaddr(struct sockaddr *addr) { return (ifa_ifwithaddr_internal(addr, 1)); } ifa_ifwithaddr_internal(struct sockaddr *addr, int getref) { ........ ........ TAILQ_FOREACH(ifp, &V_ifnet, if_link) { IF_ADDR_RLOCK(ifp); printf("ifp->xname:%s ifp->dname:%s\n",ifp->if_xname, ifp->= if_dname); TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { if (ifa->ifa_addr->sa_family !=3D addr->sa_family) = =E8 ifa->ifa_addr->sa_family=3D18 & addr->sa_family=3D28. Even though the = interface is in IPV6 mode its sa_family is not set properly, continue; } Ifa =3D= NULL; ........ ........ } From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 13:41:19 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 8A91D313 for ; Thu, 18 Apr 2013 13:41:19 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward18.mail.yandex.net (forward18.mail.yandex.net [IPv6:2a02:6b8:0:1402::3]) by mx1.freebsd.org (Postfix) with ESMTP id 3ED06EBA for ; Thu, 18 Apr 2013 13:41:19 +0000 (UTC) Received: from smtp19.mail.yandex.net (smtp19.mail.yandex.net [95.108.252.19]) by forward18.mail.yandex.net (Yandex) with ESMTP id E042C1780842; Thu, 18 Apr 2013 17:41:16 +0400 (MSK) Received: from smtp19.mail.yandex.net (localhost [127.0.0.1]) by smtp19.mail.yandex.net (Yandex) with ESMTP id AF902BE06F0; Thu, 18 Apr 2013 17:41:16 +0400 (MSK) Received: from v10-166-111.yandex.net (v10-166-111.yandex.net [84.201.166.111]) by smtp19.mail.yandex.net (nwsmtp/Yandex) with ESMTP id oIvgLmNh5a-fGT8eIRD; Thu, 18 Apr 2013 17:41:16 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1366292476; bh=WWP+PWONMkLoo7CrBd9otgjann+n1bxGljEXUoVncOQ=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=PKRaFHc1MCkQs2NQpXgbDyzOy63RBvqRWdKit7K2fPhfVuZIzYiBATPX8fp2OYtn6 HsRP4ADjo5OItEB9PzYVRcvil5tbkJxOHTgwiEeiWU5Is1MfM4lFYDSrQaUdSJSk5W oMtQspzs2ZAXQEiGcG1ur+y6JBaYXVINRrsaApRA= Authentication-Results: smtp19.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <516FF783.1040609@yandex.ru> Date: Thu, 18 Apr 2013 17:39:15 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Sreenivasa Honnur Subject: Re: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) References: In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 13:41:19 -0000 On 18.04.2013 15:37, Sreenivasa Honnur wrote: > I have a ipv6 interface(ping6 to a remove ipv6 works) when I try to > bind to this address through a socket program "sobind" fails with "49" > as return value. If I give "saddr6.sin6_addr = in6addr_any;" sobind > works. > > Any idea what could be going wrong here? > ifa_ifwithaddr_internal(struct sockaddr *addr, int getref) > { > ........ > ........ > > TAILQ_FOREACH(ifp, &V_ifnet, if_link) { > IF_ADDR_RLOCK(ifp); > printf("ifp->xname:%s ifp->dname:%s\n",ifp->if_xname, ifp->if_dname); > TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) { > if (ifa->ifa_addr->sa_family != addr->sa_family) ==> ifa->ifa_addr->sa_family=18 & addr->sa_family=28. Even though the interface is in IPV6 mode its sa_family is not set properly, > continue; > } This tailq contains any addresses assigned to the interface, sa_fimily=18 is for AF_LINK address, so it's ok. -- WBR, Andrey V. Elsukov From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 13:49:16 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D3C93539 for ; Thu, 18 Apr 2013 13:49:16 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward17.mail.yandex.net (forward17.mail.yandex.net [IPv6:2a02:6b8:0:1402::2]) by mx1.freebsd.org (Postfix) with ESMTP id 82B30F2A for ; Thu, 18 Apr 2013 13:49:16 +0000 (UTC) Received: from smtp16.mail.yandex.net (smtp16.mail.yandex.net [95.108.252.16]) by forward17.mail.yandex.net (Yandex) with ESMTP id 3BA6D10619B7; Thu, 18 Apr 2013 17:49:15 +0400 (MSK) Received: from smtp16.mail.yandex.net (localhost [127.0.0.1]) by smtp16.mail.yandex.net (Yandex) with ESMTP id 084E26A0734; Thu, 18 Apr 2013 17:49:14 +0400 (MSK) Received: from v10-166-111.yandex.net (v10-166-111.yandex.net [84.201.166.111]) by smtp16.mail.yandex.net (nwsmtp/Yandex) with ESMTP id sU26mlUVfG-nECiRYWV; Thu, 18 Apr 2013 17:49:14 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1366292954; bh=DlSS7nRrwY4YdDjVFkEr6IKi5lsOXhquJC/fO+aF0vQ=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=RZ0WtxkTSN6FqPI88qc7ffqP9IDexHP2i6ftgWz0KlsOWuDzo/66IyL1mKnylW0yP iSsQf0Sz4ouMT4VdjqBljQkzo/HCo7Uqs9PrZX4pmyTbvic83H9/O5kqV4frGryCU0 0dXwo6Y7m/WtSO9dU89muR+Sh141EEa6RcQcSN/U= Authentication-Results: smtp16.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <516FF961.5070100@yandex.ru> Date: Thu, 18 Apr 2013 17:47:13 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Sreenivasa Honnur Subject: Re: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) References: In-Reply-To: X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 13:49:16 -0000 On 18.04.2013 15:37, Sreenivasa Honnur wrote: > I have a ipv6 interface(ping6 to a remove ipv6 works) when I try to > bind to this address through a socket program "sobind" fails with > "49" as return value. If I give "saddr6.sin6_addr = in6addr_any;" > sobind works. > > Any idea what could be going wrong here? What value has the sysctl variable security.jail.jailed? -- WBR, Andrey V. Elsukov From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 13:51:37 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 4562E707 for ; Thu, 18 Apr 2013 13:51:37 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward18.mail.yandex.net (forward18.mail.yandex.net [IPv6:2a02:6b8:0:1402::3]) by mx1.freebsd.org (Postfix) with ESMTP id ED70DF68 for ; Thu, 18 Apr 2013 13:51:36 +0000 (UTC) Received: from smtp17.mail.yandex.net (smtp17.mail.yandex.net [95.108.252.17]) by forward18.mail.yandex.net (Yandex) with ESMTP id 16B721780F36; Thu, 18 Apr 2013 17:51:23 +0400 (MSK) Received: from smtp17.mail.yandex.net (localhost [127.0.0.1]) by smtp17.mail.yandex.net (Yandex) with ESMTP id D8AB61900442; Thu, 18 Apr 2013 17:51:22 +0400 (MSK) Received: from v10-166-111.yandex.net (v10-166-111.yandex.net [84.201.166.111]) by smtp17.mail.yandex.net (nwsmtp/Yandex) with ESMTP id UAbXdZMz5P-pMeGPZ4Q; Thu, 18 Apr 2013 17:51:22 +0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1366293082; bh=yn6rBYZpeEz2G5P92enQCF/QZAxMsZ231qcXYEd2+nA=; h=Message-ID:Date:From:User-Agent:MIME-Version:To:CC:Subject: References:In-Reply-To:X-Enigmail-Version:Content-Type: Content-Transfer-Encoding; b=l2eee5BYfhGNO7B8Lk5EvXVVYQdwtQ3+ihdbTkrvnLPROJCbSeAA1gey44w1I0IEo uhuu+0v+R2ktcb/av4NMSuh/Yc1gC9C2TOaQO8193Y71Jd4fFQAGJHloKW0zzXuYKQ apCx/T5K03orH8dI33uakLsYCoYsSp/SupYomAao= Authentication-Results: smtp17.mail.yandex.net; dkim=pass header.i=@yandex.ru Message-ID: <516FF9E1.5030303@yandex.ru> Date: Thu, 18 Apr 2013 17:49:21 +0400 From: "Andrey V. Elsukov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: Sreenivasa Honnur Subject: Re: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) References: <516FF961.5070100@yandex.ru> In-Reply-To: <516FF961.5070100@yandex.ru> X-Enigmail-Version: 1.4.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 13:51:37 -0000 On 18.04.2013 17:47, Andrey V. Elsukov wrote: > On 18.04.2013 15:37, Sreenivasa Honnur wrote: >> I have a ipv6 interface(ping6 to a remove ipv6 works) when I try to >> bind to this address through a socket program "sobind" fails with >> "49" as return value. If I give "saddr6.sin6_addr = in6addr_any;" >> sobind works. >> >> Any idea what could be going wrong here? > > What value has the sysctl variable security.jail.jailed? Oh, of course, I mean, what do you have in td->td_ucred? -- WBR, Andrey V. Elsukov From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 14:16:27 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BEF66F2E for ; Thu, 18 Apr 2013 14:16:27 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-bk0-x232.google.com (mail-bk0-x232.google.com [IPv6:2a00:1450:4008:c01::232]) by mx1.freebsd.org (Postfix) with ESMTP id 553981A5 for ; Thu, 18 Apr 2013 14:16:27 +0000 (UTC) Received: by mail-bk0-f50.google.com with SMTP id jg1so1295976bkc.37 for ; Thu, 18 Apr 2013 07:16:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:sender:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-type :content-transfer-encoding; bh=8FtMKd3N1GKGimTPv+8BgVP68hc6RKn+yvYKHLLRHKQ=; b=QLuhkN28lgtvqgCruxyPxKqmM6bGGu6gem4aHaXFvPv0a/9UxzPcjz8/aGQOUsKVx8 4cxuuWqx+/IfC7s9+fq6e4r34LVpbXTszHaZGe4aD/5uDBaLCMNFa4wGnkjCDbLkBJ9s SG4MzxEfrn3A0CUaq40is6Fcb4PP0xep/2qX49FmgRrmKNbnqZIlJ8ZbbkHYMvIgtr3M nYOi84VXJi0AuafxYvnxPSlV39t22ZWBgqEGyA7iC2geXC6FU+db3wUJ4FcfUPYnEQgj NAAZQez7baTkzVzFeeDbnTYdZXuGGPpSpWLhyUrcfoOrIR27STHXnHXW0lae6dC5cYKL qPKQ== X-Received: by 10.204.170.202 with SMTP id e10mr4052928bkz.73.1366294586371; Thu, 18 Apr 2013 07:16:26 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPS id jm15sm3166060bkb.13.2013.04.18.07.16.23 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 18 Apr 2013 07:16:25 -0700 (PDT) Sender: Alexander Motin Message-ID: <51700036.3000306@FreeBSD.org> Date: Thu, 18 Apr 2013 17:16:22 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: Synchronizing TSC References: <516DCAF7.20400@FreeBSD.org> <516E4537.7050205@FreeBSD.org> <20130417085052.GZ2930@kib.kiev.ua> In-Reply-To: <20130417085052.GZ2930@kib.kiev.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Jim Harris X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 14:16:27 -0000 On 17.04.2013 11:50, Konstantin Belousov wrote: > On Wed, Apr 17, 2013 at 09:46:15AM +0300, Alexander Motin wrote: >> On 17.04.2013 03:25, Jim Harris wrote: >>> >>> On Tue, Apr 16, 2013 at 3:04 PM, Alexander Motin >> > wrote: >>> >>> Hi. >>> >>> Recently I've got 6-core/12-thread system on Sandy Bridge-E Core >>> i7-3930K CPU and was unpleasantly surprised to see that TSCs are not >>> synchronized there. While all 11 APs were synchronized, BSP was far >>> behind them. Since it is single-socket system, I don't know any good >>> reason for such behavior except some BIOS bug. But I've recalled >>> that somewhere was some discussions about possible TSC >>> synchronization. I've implemented patch below that allows to adjust >>> TSC values of BSPs to AP's one on boot using CPU MSRs, hoping that >>> they should not diverge after that: >>> http://people.freebsd.org/~__mav/tsc_adj2.patch >>> >>> >>> I don't know very much about all different TSC hardware to predict >>> when it is safe to enable the functionality, but at least on my >>> system being enabled via loader tunable it seems working well. >>> >>> Comments? >>> >>> >>> You may be remembering this thread on r238755 last year: >>> >>> http://lists.freebsd.org/pipermail/svn-src-head/2012-July/038992.html >>> >>> This was a bug fix in the TSC synchronization test code though, not >>> anything for trying to adjust out-of-sync TSCs. >> >> I remember that thread, but I think I've seen somebody told somewhere >> that it could be interesting to implement some MI mechanism. Never mind. >> >>> The Intel SDM (volume 3, section 17.13 of March 2013 revision) says >>> earlier models can only write to lower 32 bits of >>> IA32_TIME_STAMP_COUNTER, but these models also should not have invariant >>> TSC so they would never even get to your new routine. So your patch >>> seems OK for Intel CPUs, at least as a tunable that is disabled by default. >> >> Thanks. >> >>> My only concern would be why TSC on the BSP started out-of-sync on your >>> system. Theoretically, BIOS could adjust TSCs in SMM to try to hide SMI >>> code execution from the OS, which could then make them out-of-sync >>> again. Not sure if that's what's happening here, but might be worth a >>> test putting the TSC test code on a periodic timer to see if they ever >>> get out of sync again. >> >> I did one more interesting observation: on every reboot drift between >> BSP and APs is growing proportionally to the previous system power-on >> time. On first boot it is -3878361036 (just above one second), after >> reboot some minutes later it is -1123454492776 (about 6 minutes), after >> another reboot it is -1853033521804 (about 10 minutes). >> >> Unless my adjustment code would be active, I would guess that AP's TSC >> is running linearly while BSP's for some reason reset to zero on every >> reboot. But since I am synchronizing them on each boot, the only >> possibility for it I see is that there is some other timer(s) / >> counter(s) not affected by MSR writes that ticks linearly and reloading >> AP's TSC, but for some reason not reloading BSP's. > > For me it sounds as the BIOS bug, indeed. Could you verify the content > of IA32_TSC_ADJUST on all cores (I believe it is present on E5) ? > Also, using TSC_ADJUST to correct the skew seems to be preferrable, > according to the Intel docs. IA32_TSC_ADJUST register seems not present there. At least cpucontrol doesn't want to read it. In Intel docs I also see it mentioned only in context of future Haswell generation. And I don't see "Standard Extended Features" line in dmesg. > Why do you use cpuid in the assembly sequence ? As I understand, you > ensure that there is a serialization point, but why do you need it ? The idea was to minimize time distance between following MSR read and write. But may be it is not needed, I am not exactly sure about that magic. -- Alexander Motin From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 16:12:02 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 902C49C6; Thu, 18 Apr 2013 16:12:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 046FB9C4; Thu, 18 Apr 2013 16:12:01 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r3IGBv2f077829; Thu, 18 Apr 2013 19:11:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.2 kib.kiev.ua r3IGBv2f077829 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r3IGBvjZ077828; Thu, 18 Apr 2013 19:11:57 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 18 Apr 2013 19:11:57 +0300 From: Konstantin Belousov To: Alexander Motin Subject: Re: Synchronizing TSC Message-ID: <20130418161157.GI85148@kib.kiev.ua> References: <516DCAF7.20400@FreeBSD.org> <516E4537.7050205@FreeBSD.org> <20130417085052.GZ2930@kib.kiev.ua> <51700036.3000306@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="54ZiyWcDhi/7bWb8" Content-Disposition: inline In-Reply-To: <51700036.3000306@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: "freebsd-hackers@freebsd.org" , Jim Harris X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 16:12:02 -0000 --54ZiyWcDhi/7bWb8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 18, 2013 at 05:16:22PM +0300, Alexander Motin wrote: > On 17.04.2013 11:50, Konstantin Belousov wrote: > > On Wed, Apr 17, 2013 at 09:46:15AM +0300, Alexander Motin wrote: > >> On 17.04.2013 03:25, Jim Harris wrote: > >>> > >>> On Tue, Apr 16, 2013 at 3:04 PM, Alexander Motin >>> > wrote: > >>> > >>> Hi. > >>> > >>> Recently I've got 6-core/12-thread system on Sandy Bridge-E Core > >>> i7-3930K CPU and was unpleasantly surprised to see that TSCs are= not > >>> synchronized there. While all 11 APs were synchronized, BSP was = far > >>> behind them. Since it is single-socket system, I don't know any = good > >>> reason for such behavior except some BIOS bug. But I've recalled > >>> that somewhere was some discussions about possible TSC > >>> synchronization. I've implemented patch below that allows to adj= ust > >>> TSC values of BSPs to AP's one on boot using CPU MSRs, hoping th= at > >>> they should not diverge after that: > >>> http://people.freebsd.org/~__mav/tsc_adj2.patch > >>> > >>> > >>> I don't know very much about all different TSC hardware to predi= ct > >>> when it is safe to enable the functionality, but at least on my > >>> system being enabled via loader tunable it seems working well. > >>> > >>> Comments? > >>> > >>> > >>> You may be remembering this thread on r238755 last year: > >>> > >>> http://lists.freebsd.org/pipermail/svn-src-head/2012-July/038992.html > >>> > >>> This was a bug fix in the TSC synchronization test code though, not > >>> anything for trying to adjust out-of-sync TSCs. > >> > >> I remember that thread, but I think I've seen somebody told somewhere > >> that it could be interesting to implement some MI mechanism. Never min= d. > >> > >>> The Intel SDM (volume 3, section 17.13 of March 2013 revision) says > >>> earlier models can only write to lower 32 bits of > >>> IA32_TIME_STAMP_COUNTER, but these models also should not have invari= ant > >>> TSC so they would never even get to your new routine. So your patch > >>> seems OK for Intel CPUs, at least as a tunable that is disabled by de= fault. > >> > >> Thanks. > >> > >>> My only concern would be why TSC on the BSP started out-of-sync on yo= ur > >>> system. Theoretically, BIOS could adjust TSCs in SMM to try to hide = SMI > >>> code execution from the OS, which could then make them out-of-sync > >>> again. Not sure if that's what's happening here, but might be worth a > >>> test putting the TSC test code on a periodic timer to see if they ever > >>> get out of sync again. > >> > >> I did one more interesting observation: on every reboot drift between > >> BSP and APs is growing proportionally to the previous system power-on > >> time. On first boot it is -3878361036 (just above one second), after > >> reboot some minutes later it is -1123454492776 (about 6 minutes), after > >> another reboot it is -1853033521804 (about 10 minutes). > >> > >> Unless my adjustment code would be active, I would guess that AP's TSC > >> is running linearly while BSP's for some reason reset to zero on every > >> reboot. But since I am synchronizing them on each boot, the only > >> possibility for it I see is that there is some other timer(s) / > >> counter(s) not affected by MSR writes that ticks linearly and reloading > >> AP's TSC, but for some reason not reloading BSP's. > > > > For me it sounds as the BIOS bug, indeed. Could you verify the content > > of IA32_TSC_ADJUST on all cores (I believe it is present on E5) ? > > Also, using TSC_ADJUST to correct the skew seems to be preferrable, > > according to the Intel docs. >=20 > IA32_TSC_ADJUST register seems not present there. At least cpucontrol=20 > doesn't want to read it. In Intel docs I also see it mentioned only in=20 > context of future Haswell generation. And I don't see "Standard Extended= =20 > Features" line in dmesg. >=20 > > Why do you use cpuid in the assembly sequence ? As I understand, you > > ensure that there is a serialization point, but why do you need it ? >=20 > The idea was to minimize time distance between following MSR read and=20 > write. But may be it is not needed, I am not exactly sure about that magi= c. Well, I do not believe that such trick is useful. Patch with removed cpuid and with resync disabled by default (as it is now, AFAIR) would be IMO fine for the commit. --54ZiyWcDhi/7bWb8 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRcBtNAAoJEJDCuSvBvK1B1uwP/jTHbObpjYzDIS2tFjFpTZNA PYJLx9FEaLVAqc0kz5XZqXNM7k7XpobjKQuzZmVaIRzdjQaXcuCkfgWf5ABS4y9d E6Hqp/Qdp2YKtJBLwMpvtgj9IXo29fYeBj4495jFnWZSH2tYcaNZSEioBAY6Bx1w Ac/h+E9k47Bl5iVMBaE+WA239D1G5M9bDHyEMpcrIijJrUBFjMYouR+vyTYC+gXj UtsIviC0ZrRdHS0woqWpGAMgLEJJJyW53PIo6nqZb3B+Tr4VAKu5FiDrZrnT+fdf nPZDnZUHC1ggYnO/Y4ohYQXthu1u9cmq3ip3idsdVCm7eQLofDbONrHOGDJUDvQP zXjFui2D67GIUNBlLyxocPbjnkLuwjo1WVml3cKbgABmTqhV+r+wG8W1pj+VFeX9 CH0oqd5PzyecptN6SnyL1LzpKjFagZ0ZKE/L6SPhjZBh5HCtVABzOlPVIMLtMopd lklwT44jWnkdLSH39fhyb34iI29rdtgHdgo0nyi7ClWDJgCXAcVMpE37PuCWPkvd j7yUDHg2EcWdlQTD6lyZAsA8EFHTUk/NHCpVtXa2Urzx912kYx+j6+ZM3JcQ4xEp UGf833xUaTG8P516eS0+gOI9PRb7EhV9GmE7juxa/tzhht8Jym7qLX1EWXtbBY10 z/zn1VvmksC3/nsZeFH4 =Y3OR -----END PGP SIGNATURE----- --54ZiyWcDhi/7bWb8-- From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 17:52:41 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 08BA866D for ; Thu, 18 Apr 2013 17:52:41 +0000 (UTC) (envelope-from nbari@inbox.im) Received: from ca2.route.mx (ca2.route.mx [72.55.175.69]) by mx1.freebsd.org (Postfix) with ESMTP id 949FA10A1 for ; Thu, 18 Apr 2013 17:52:40 +0000 (UTC) Received: (route-mx 93846 invoked from network); 18 Apr 2013 17:52:32 -0000 Received: from 89-181-202-22.net.novis.pt (HELO [192.168.1.100]) (nbari@inbox.im@route.mx) (envelope-sender ) by ca2.route.mx (route-mx) with AES128-SHA encrypted SMTP for ; 18 Apr 2013 17:52:32 -0000 Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: [SOLVED] home lab [AMD A10 5800K , AsRock FM2A85X-ITX] panics too much From: Nicolas de Bari Embriz Garcia Rojas In-Reply-To: <122807B5-16CD-4F55-AB90-FB2ED433C425@inbox.im> Date: Thu, 18 Apr 2013 18:52:28 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <122807B5-16CD-4F55-AB90-FB2ED433C425@inbox.im> To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.1503) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 17:52:41 -0000 Hi, the solution was to properly set the voltages of the memory on the = BIOS. memory on BIOS was configured to only use 1.25. and the specs of the = memory say it required 1.5, setting the proper values on the BIOS did = the trick. regards. On Apr 8, 2013, at 4:22 AM, Nicolas de Bari Embriz Garcia Rojas = wrote: > Hi, all, I am having many kernel panic with an AMD A10 5800K cpu and a = AsRock FM2A85X-ITX motherboard.=20 >=20 > I successfully installed FreeBSD 9.1 amd64 with zfs raidz on root, = but when installing subversion from ports (make install clean), here my = first kernel painc occurred. After the machine was online again I = finished (make install clean) and well this time worked.=20 >=20 > I update the sources svn =85. and tried a make -j4 buildworld, it = started to build but later the machine just freeze.=20 >=20 > I later tried without the -j4 and was available to compile the world. >=20 > when compiling the kernel (make kernel) I got another kernel panic and = machine rebooted. >=20 > This time before booting, I tried to fix the bios setting the the OC = (overclook) features on the motherboard to match the hardware avoiding = fancy things, and well some times now I get panics others not. >=20 > I enable dumpdev=3D"AUTO" in rc.conf, but problem is that I have all = swap on the 'zfs' and I am not getting the dumps. >=20 > So wondering if some one could give me an advice of how could I fine = tune my system so that I can use this hardware. >=20 > the logs unfortunately don't show nothing I have enable *.* = /var/log/all.log on syslog.conf but don't show anything related to the = panics. >=20 > Any ideas ? >=20 > below I let the output of dmesg.boot and pciconf >=20 > 1 Copyright (c) 1992-2013 The FreeBSD Project. = = =20 > 2 Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, = 1994 = =20 > 3 The Regents of the University of California. All rights reserved. = = =20 > 4 FreeBSD is a registered trademark of The FreeBSD Foundation. = = =20 > 5 FreeBSD 9.1-STABLE #2 r249231: Sun Apr 7 13:36:17 UTC 2013 = = =20 > 6 root@amdtest:/usr/obj/usr/src/sys/amdtest amd64 = = =20 > 7 FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221 = = =20 > 8 CPU: AMD A10-5800K APU with Radeon(tm) HD Graphics (4192.46-MHz = K8-class CPU) = =20 > 9 Origin =3D "AuthenticAMD" Id =3D 0x610f01 Family =3D 0x15 = Model =3D 0x10 Stepping =3D 1 = =20 > 10 = Features=3D0x178bfbff = =20 > 11 = Features2=3D0x3e98320b = =20 > 12 AMD Features=3D0x2e500800 = = =20 > 13 AMD = Features2=3D0x1ebbfff,NodeId,TBM,Topology,,> = =20 > 14 Standard Extended Features=3D0x8 = = =20 > 15 TSC: P-state invariant, performance statistics = = =20 > 16 real memory =3D 17179869184 (16384 MB) = = =20 > 17 avail memory =3D 16455127040 (15692 MB) = = =20 > 18 Event timer "LAPIC" quality 400 = = =20 > 19 ACPI APIC Table: = = =20 > 20 FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs = = =20 > 21 FreeBSD/SMP: 1 package(s) x 4 core(s) = = =20 > 22 cpu0 (BSP): APIC ID: 16 = = =20 > 23 cpu1 (AP): APIC ID: 17 = = =20 > 24 cpu2 (AP): APIC ID: 18 = = =20 > 25 cpu3 (AP): APIC ID: 19 = = =20 > 26 WARNING: VIMAGE (virtualized network stack) is a highly = experimental feature. = =20 > 27 ACPI Warning: Optional field Pm2ControlBlock has zero address or = length: 0x0000000000000000/0x1 (20110527/tbfadt-583) = =20 > 28 ioapic0 irqs 0-23 on motherboard = = =20 > 29 kbd1 at kbdmux0 = = =20 > 30 aesni0: on motherboard = = =20 > 31 cryptosoft0: on motherboard = = =20 > 32 acpi0: on motherboard = = =20 > 33 acpi0: Power Button (fixed) = = =20 > 34 cpu0: on acpi0 = = =20 > 35 cpu1: on acpi0 = = =20 > 36 cpu2: on acpi0 = = =20 > 37 cpu3: on acpi0 = = =20 > 38 attimer0: port 0x40-0x43 irq 0 on acpi0 = = =20 > 39 Timecounter "i8254" frequency 1193182 Hz quality 0 = = =20 > 40 Event timer "i8254" frequency 1193182 Hz quality 100 = = =20 > 41 atrtc0: port 0x70-0x71 irq 8 on acpi0 = = =20 > 42 Event timer "RTC" frequency 32768 Hz quality 0 = = =20 > 43 hpet0: iomem 0xfed00000-0xfed003ff on = acpi0 = =20 > 44 Timecounter "HPET" frequency 14318180 Hz quality 950 = = =20 > 45 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 = = =20 > 46 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x808-0x80b on = acpi0 = =20 > 47 pcib0: port 0xcf8-0xcff on acpi0 = = =20 > 48 pci0: on pcib0 = = =20 > 49 pci0: at device 0.2 (no driver attached) = = =20 > 50 vgapci0: port 0xf000-0xf0ff mem = 0xfc000000-0xfdffffff,0xff700000-0xff73ffff irq 17 at device 1.0 on pci0 = =20 > 51 pci0: at device 1.1 (no driver attached) = = =20 > 52 pcib1: irq 18 at device 2.0 on pci0 = = =20 > 53 pci1: on pcib1 = = =20 > 54 em0: port = 0xe000-0xe01f mem = 0xff6c0000-0xff6dffff,0xff600000-0xff67ffff,0xff6e0000-0xff6e3fff irq 18 = at device 0.0 on pci1 =20 > 55 em0: Using MSIX interrupts with 3 vectors =20 > 56 em0: Ethernet address: 68:05:ca:11:d3:2c = = =20 > 57 xhci0: mem = 0xff74a000-0xff74bfff irq 18 at device 16.0 on pci0 = =20 > 58 xhci0: 32 byte context size. = = =20 > 59 usbus0 on xhci0 = = =20 > 60 xhci1: mem = 0xff748000-0xff749fff irq 17 at device 16.1 on pci0 = =20 > 61 xhci1: 32 byte context size. = = =20 > 62 usbus1 on xhci1 = = =20 > 63 ahci0: port = 0xf140-0xf147,0xf130-0xf133,0xf120-0xf127,0xf110-0xf113,0xf100-0xf10f = mem 0xff751000-0xff7517ff irq 19 at device 17.0 on pci0 =20 > 64 ahci0: AHCI v1.30 with 5 6Gbps ports, Port Multiplier supported = = =20 > 65 ahcich0: at channel 0 on ahci0 = = =20 > 66 ahcich1: at channel 1 on ahci0 = = =20 > 67 ahcich2: at channel 2 on ahci0 = = =20 > 68 ahcich3: at channel 3 on ahci0 = = =20 > 69 ahcich4: at channel 4 on ahci0 = = =20 > 70 ohci0: mem 0xff750000-0xff750fff = irq 18 at device 18.0 on pci0 = =20 > 71 usbus2 on ohci0 = = =20 > 72 ehci0: mem = 0xff74f000-0xff74f0ff irq 17 at device 18.2 on pci0 = =20 > 73 usbus3: EHCI version 1.0 = = =20 > 74 usbus3 on ehci0 = = =20 > 75 ohci1: mem 0xff74e000-0xff74efff = irq 18 at device 19.0 on pci0 = =20 > 76 usbus4 on ohci1 = = =20 > 77 ehci1: mem = 0xff74d000-0xff74d0ff irq 17 at device 19.2 on pci0 = =20 > 78 usbus5: EHCI version 1.0 = = =20 > 79 usbus5 on ehci1 = = =20 > 80 pci0: at device 20.0 (no driver attached) = = =20 > 81 pci0: at device 20.2 (no driver attached) = = =20 > 82 isab0: at device 20.3 on pci0 = = =20 > 83 isa0: on isab0 = = =20 > 84 pcib2: at device 20.4 on pci0 = = =20 > 85 pci2: on pcib2 = = =20 > 86 ohci2: mem 0xff74c000-0xff74cfff = irq 18 at device 20.5 on pci0 = =20 > 87 usbus6 on ohci2 = = =20 > 88 pcib3: at device 21.0 on pci0 = = =20 > 89 pci3: on pcib3 = = =20 > 90 re0: port = 0xd000-0xd0ff mem 0xfe004000-0xfe004fff,0xfe000000-0xfe003fff irq 16 at = device 0.0 on pci3 =20 > 91 re0: Using 1 MSI-X message = = =20 > 92 re0: Chip rev. 0x2c800000 = = =20 > 93 re0: MAC rev. 0x00000000 = = =20 > 94 miibus0: on re0 = = =20 > 95 rgephy0: PHY 1 on = miibus0 = =20 > 96 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, = 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT, 1000baseT-master, = 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-fl ow, = 1000baseT-FDX-flow-master, auto, auto-flow = = =20 > 97 re0: Ethernet address: bc:5f:f4:a3:25:65 = = =20 > 98 acpi_button0: on acpi0 = = =20 > 99 orm0: at iomem 0xcf800-0xd07ff on isa0 = = =20 > 100 sc0: at flags 0x100 on isa0 = = =20 > 101 sc0: VGA <16 virtual consoles, flags=3D0x300> = = =20 > 102 vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff = on isa0 = =20 > 103 atkbdc0: at port 0x60,0x64 on isa0 = = =20 > 104 atkbd0: irq 1 on atkbdc0 =20 > 105 kbd0 at atkbd0 = = =20 > 106 atkbd0: [GIANT-LOCKED] = = =20 > 107 ctl: CAM Target Layer loaded = = =20 > 108 acpi_throttle0: on cpu0 = = =20 > 109 hwpstate0: on cpu0 = = =20 > 110 acpi_throttle1: on cpu1 = = =20 > 111 acpi_throttle1: failed to attach P_CNT = = =20 > 112 device_attach: acpi_throttle1 attach returned 6 = = =20 > 113 acpi_throttle2: on cpu2 = = =20 > 114 acpi_throttle2: failed to attach P_CNT = = =20 > 115 device_attach: acpi_throttle2 attach returned 6 = = =20 > 116 acpi_throttle3: on cpu3 = = =20 > 117 acpi_throttle3: failed to attach P_CNT = = =20 > 118 device_attach: acpi_throttle3 attach returned 6 = = =20 > 119 ZFS filesystem version: 5 = = =20 > 120 ZFS storage pool version: features support (5000) = = =20 > 121 Timecounters tick every 1.000 msec = = =20 > 122 vboxdrv: fAsync=3D0 offMin=3D0x44e offMax=3D0x74f = = =20 > 123 IPsec: Initialized Security Association Processing. = = =20 > 124 ipfw2 (+ipv6) initialized, divert enabled, nat loadable, default = to accept, logging disabled = =20 > 125 usbus0: 5.0Gbps Super Speed USB v3.0 = = =20 > 126 usbus1: 5.0Gbps Super Speed USB v3.0 = = =20 > 127 usbus2: 12Mbps Full Speed USB v1.0 = = =20 > 128 usbus3: 480Mbps High Speed USB v2.0 = = =20 > 129 usbus4: 12Mbps Full Speed USB v1.0 = = =20 > 130 usbus5: 480Mbps High Speed USB v2.0 = = =20 > 131 usbus6: 12Mbps Full Speed USB v1.0 = = =20 > 132 ugen0.1: <0x1022> at usbus0 = = =20 > 133 uhub0: <0x1022 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on = usbus0 = =20 > 134 ugen1.1: <0x1022> at usbus1 = = =20 > 135 uhub1: <0x1022 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on = usbus1 = =20 > 136 ugen2.1: at usbus2 = = =20 > 137 uhub2: on = usbus2 = =20 > 138 ugen3.1: at usbus3 = = =20 > 139 uhub3: on = usbus3 = =20 > 140 ugen4.1: at usbus4 = = =20 > 141 uhub4: on = usbus4 = =20 > 142 ugen5.1: at usbus5 = = =20 > 143 uhub5: on = usbus5 = =20 > 144 ugen6.1: at usbus6 = = =20 > 145 uhub6: on = usbus6 = =20 > 146 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 = = =20 > 147 ada0: ATA-8 SATA 3.x device = = =20 > 148 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) = = =20 > 149 ada0: Command Queueing enabled = = =20 > 150 ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) = = =20 > 151 ada0: Previously was known as ad4 = = =20 > 152 ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 = = =20 > 153 ada1: ATA-8 SATA 3.x device = = =20 > 154 ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) = = =20 > 155 ada1: Command Queueing enabled = = =20 > 156 ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) = = =20 > 157 ada1: Previously was known as ad6 = = =20 > 158 ada2 at ahcich2 bus 0 scbus2 target 0 lun 0 = = =20 > 159 ada2: ATA-8 SATA 3.x device = = =20 > 160 ada2: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) = = =20 > 161 ada2: Command Queueing enabled = = =20 > 162 ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) = = =20 > 163 ada2: Previously was known as ad8 = = =20 > 164 ada3 at ahcich3 bus 0 scbus3 target 0 lun 0 = = =20 > 165 ada3: ATA-9 SATA 3.x device = = =20 > 166 ada3: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) = = =20 > 167 ada3: Command Queueing enabled = = =20 > 168 ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) = = =20 > 169 ada3: Previously was known as ad10 = = =20 > 170 SMP: AP CPU #2 Launched! = = =20 > 171 SMP: AP CPU #1 Launched! = = =20 > 172 SMP: AP CPU #3 Launched! = = =20 > 173 Timecounter "TSC-low" frequency 2096231117 Hz quality 1000 = = =20 > 174 uhub6: 2 ports with 2 removable, self powered = = =20 > 175 uhub4: 5 ports with 5 removable, self powered = = =20 > 176 uhub2: 5 ports with 5 removable, self powered = = =20 > 177 uhub1: 4 ports with 4 removable, self powered = = =20 > 178 uhub0: 4 ports with 4 removable, self powered = = =20 > 179 Root mount waiting for: usbus5 usbus3 = = =20 > 180 uhub3: 5 ports with 5 removable, self powered = = =20 > 181 uhub5: 5 ports with 5 removable, self powered = = =20 > 182 Trying to mount root from zfs:zroot []... = = =20 > 183 bridge0: Ethernet address: 02:fe:4a:c8:9c:00 = = =20 > 184 bridge1: Ethernet address: 02:fe:4a:c8:9c:01 = = =20 > 185 epair0a: Ethernet address: 02:00:a0:00:0f:0a = = =20 > 186 epair0b: Ethernet address: 02:00:a0:00:10:0b = = =20 > 187 epair1a: Ethernet address: 02:00:a0:00:11:0a = = =20 > 188 epair1b: Ethernet address: 02:00:a0:00:12:0b = = =20 > 189 tap0: Ethernet address: 00:bd:14:26:00:00 = = =20 > 190 re0: link state changed to UP =20 >=20 > the output of pciconf -l is: > hostb0@pci0:0:0:0: class=3D0x060000 card=3D0x14101849 = chip=3D0x14101022 rev=3D0x00 hdr=3D0x00 > none0@pci0:0:0:2: class=3D0x080600 card=3D0x14191022 = chip=3D0x14191022 rev=3D0x00 hdr=3D0x00 > vgapci0@pci0:0:1:0: class=3D0x030000 card=3D0x99011849 = chip=3D0x99011002 rev=3D0x00 hdr=3D0x00 > none1@pci0:0:1:1: class=3D0x040300 card=3D0x99021849 = chip=3D0x99021002 rev=3D0x00 hdr=3D0x00 > pcib1@pci0:0:2:0: class=3D0x060400 card=3D0x12341022 = chip=3D0x14121022 rev=3D0x00 hdr=3D0x01 > xhci0@pci0:0:16:0: class=3D0x0c0330 card=3D0x78121022 = chip=3D0x78121022 rev=3D0x03 hdr=3D0x00 > xhci1@pci0:0:16:1: class=3D0x0c0330 card=3D0x78121022 = chip=3D0x78121022 rev=3D0x03 hdr=3D0x00 > ahci0@pci0:0:17:0: class=3D0x010601 card=3D0x78011849 = chip=3D0x78011022 rev=3D0x40 hdr=3D0x00 > ohci0@pci0:0:18:0: class=3D0x0c0310 card=3D0x78071849 = chip=3D0x78071022 rev=3D0x11 hdr=3D0x00 > ehci0@pci0:0:18:2: class=3D0x0c0320 card=3D0x78081849 = chip=3D0x78081022 rev=3D0x11 hdr=3D0x00 > ohci1@pci0:0:19:0: class=3D0x0c0310 card=3D0x78071849 = chip=3D0x78071022 rev=3D0x11 hdr=3D0x00 > ehci1@pci0:0:19:2: class=3D0x0c0320 card=3D0x78081849 = chip=3D0x78081022 rev=3D0x11 hdr=3D0x00 > none2@pci0:0:20:0: class=3D0x0c0500 card=3D0x780b1849 = chip=3D0x780b1022 rev=3D0x14 hdr=3D0x00 > none3@pci0:0:20:2: class=3D0x040300 card=3D0x88921849 = chip=3D0x780d1022 rev=3D0x01 hdr=3D0x00 > isab0@pci0:0:20:3: class=3D0x060100 card=3D0x780e1849 = chip=3D0x780e1022 rev=3D0x11 hdr=3D0x00 > pcib2@pci0:0:20:4: class=3D0x060401 card=3D0x00000000 = chip=3D0x780f1022 rev=3D0x40 hdr=3D0x01 > ohci2@pci0:0:20:5: class=3D0x0c0310 card=3D0x78091849 = chip=3D0x78091022 rev=3D0x11 hdr=3D0x00 > pcib3@pci0:0:21:0: class=3D0x060400 card=3D0x00001022 = chip=3D0x43a01022 rev=3D0x00 hdr=3D0x01 > hostb1@pci0:0:24:0: class=3D0x060000 card=3D0x00000000 = chip=3D0x14001022 rev=3D0x00 hdr=3D0x00 > hostb2@pci0:0:24:1: class=3D0x060000 card=3D0x00000000 = chip=3D0x14011022 rev=3D0x00 hdr=3D0x00 > hostb3@pci0:0:24:2: class=3D0x060000 card=3D0x00000000 = chip=3D0x14021022 rev=3D0x00 hdr=3D0x00 > hostb4@pci0:0:24:3: class=3D0x060000 card=3D0x00000000 = chip=3D0x14031022 rev=3D0x00 hdr=3D0x00 > hostb5@pci0:0:24:4: class=3D0x060000 card=3D0x00000000 = chip=3D0x14041022 rev=3D0x00 hdr=3D0x00 > hostb6@pci0:0:24:5: class=3D0x060000 card=3D0x00000000 = chip=3D0x14051022 rev=3D0x00 hdr=3D0x00 > em0@pci0:1:0:0: class=3D0x020000 card=3D0xa01f8086 chip=3D0x10d38086 = rev=3D0x00 hdr=3D0x00 > re0@pci0:3:0:0: class=3D0x020000 card=3D0x81681849 chip=3D0x816810ec = rev=3D0x06 hdr=3D0x00 >=20 >=20 >=20 From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 18 21:52:04 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 458FFCE0 for ; Thu, 18 Apr 2013 21:52:04 +0000 (UTC) (envelope-from carl.shapiro@gmail.com) Received: from mail-qc0-x235.google.com (mail-qc0-x235.google.com [IPv6:2607:f8b0:400d:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id 0D0B53FA for ; Thu, 18 Apr 2013 21:52:03 +0000 (UTC) Received: by mail-qc0-f181.google.com with SMTP id a22so1663152qcs.12 for ; Thu, 18 Apr 2013 14:52:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=Y/URqVT4dYtrahLFRfGnVAS8kXWH8yKlHvr+C/PjVZE=; b=dJ2FiNe1nrFXd52SU0sVqQwxEWmab3s3mulmZPJLNLhWMJCUP5C5acAPwYdJY1/QD3 SHCo+RpqFm7Wg3geUVozZ1bl2B82Vgxuhgr/LuuAGTe9cFLk0WQk6CMKnMrb5l0pWmX1 tt7cB9kQATRlYcmfwph4DYp5J/ZgNmfzT09BFfIPEpxLk3gSPtBxRFYScU7QytXgsBWV hLKSTjdkJKYQ6tqNmD4rh4R+8ZMwctRC23EZHe/UOSbGdFCl/b7aT81gaQc0BbidYOng MwbNEunvTUw+I2czd12Mv+S6vJeKCDXPynOWbJuRvoQ6pxbC+O/pPg95uODMz5NXkt2c gLPw== X-Received: by 10.229.0.141 with SMTP id 13mr682425qcb.80.1366321923633; Thu, 18 Apr 2013 14:52:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.49.130.225 with HTTP; Thu, 18 Apr 2013 14:51:43 -0700 (PDT) In-Reply-To: <20130417082143.GW2930@kib.kiev.ua> References: <20130417082143.GW2930@kib.kiev.ua> From: Carl Shapiro Date: Thu, 18 Apr 2013 14:51:43 -0700 Message-ID: Subject: Re: MADV_FREE and wait4 EFAULT To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Apr 2013 21:52:04 -0000 On Wed, Apr 17, 2013 at 1:21 AM, Konstantin Belousov wrote: > Did you ensured with e.g. ktrace and procstat -v that your assumptions > hold, i.e. the addresses supplied as wait4(2) arguments are valid ? > Please provide the minimal test case demonstrating the behaviour. > Yes. I instrumented my code to check for a wait4 failure, print the addresses of the status and rusage arguments, and dump the contents of /proc/curproc/map. The addresses of the status and rusage arguments are always in the range of a mapping and marked as read write. I have yet to distill the failure to a minimal test case. The test case I do have is the test harness for the Go language. After running for about 45 minutes I can observe a failure. I have been working to produce something smaller and faster. > MADV_FREE should only result in the possible lost of the previous > content of the page, not in the faulting of the page access. From the > inspection of the code, I do not see how MADV_FREE could result in > the memory address becoming invalid. > I see. What has lead us to believe this might be an issue with page faults is that writing zeroes to the page with memset before passing it to wait4 makes the error go away. Do you have any advice about how one might go about instrumenting wait4 to generate more information about a failed copyout? Are tools such as dtrace useful in these situations or might it be too invasive? Because of the protracted test cycle and my lack of knowledge in this area, conducting experiments is quite painful at the moment. Thanks, Carl From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 19 12:49:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 2D169EC3 for ; Fri, 19 Apr 2013 12:49:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 7EAF91FA for ; Fri, 19 Apr 2013 12:49:47 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r3JCng46045277; Fri, 19 Apr 2013 15:49:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.2 kib.kiev.ua r3JCng46045277 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r3JCngEa045276; Fri, 19 Apr 2013 15:49:42 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 19 Apr 2013 15:49:42 +0300 From: Konstantin Belousov To: Carl Shapiro Subject: Re: MADV_FREE and wait4 EFAULT Message-ID: <20130419124942.GA67273@kib.kiev.ua> References: <20130417082143.GW2930@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="82I3+IH0IqGh5yIs" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 12:49:48 -0000 --82I3+IH0IqGh5yIs Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 18, 2013 at 02:51:43PM -0700, Carl Shapiro wrote: > On Wed, Apr 17, 2013 at 1:21 AM, Konstantin Belousov wrote: >=20 > > Did you ensured with e.g. ktrace and procstat -v that your assumptions > > hold, i.e. the addresses supplied as wait4(2) arguments are valid ? > > Please provide the minimal test case demonstrating the behaviour. > > >=20 > Yes. I instrumented my code to check for a wait4 failure, print the > addresses of the status and rusage arguments, and dump the contents of > /proc/curproc/map. The addresses of the status and rusage arguments are > always in the range of a mapping and marked as read write. It would be of some interest to see the evidence. Is your code multithreaded ? >=20 > I have yet to distill the failure to a minimal test case. The test case I > do have is the test harness for the Go language. After running for about > 45 minutes I can observe a failure. I have been working to produce > something smaller and faster. The test case is required to decide whether the bug is in the application or in the OS. >=20 >=20 > > MADV_FREE should only result in the possible lost of the previous > > content of the page, not in the faulting of the page access. From the > > inspection of the code, I do not see how MADV_FREE could result in > > the memory address becoming invalid. > > >=20 > I see. What has lead us to believe this might be an issue with page faul= ts > is that writing zeroes to the page with memset before passing it to wait4 > makes the error go away. There is no difference in the access performed by copyout vs. access caused by the usermode write. >=20 > Do you have any advice about how one might go about instrumenting wait4 to > generate more information about a failed copyout? Are tools such as dtra= ce > useful in these situations or might it be too invasive? Because of the > protracted test cycle and my lack of knowledge in this area, conducting > experiments is quite painful at the moment. No, I cannot give an advice, I think we should first decide which code to blame. BTW, you could try enabling sysctl machdep.uprintf_signal. Oh, you did not specified the architecture and version of the system. --82I3+IH0IqGh5yIs Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRcT1lAAoJEJDCuSvBvK1B8DYP/00fJjKjqGegn+hv8HlyjlHY zhvyKcEyeHLOKUBWV9cNvOZOmo8TsPmW95vd78dBR9xoCCjfXz0YLDA3gulkHhWz zaNTzfn+BT8AyEmDu3/lthPchZwonLUeGlb5X0tnuQ8/beRNivMBr671ckn4rJZs rqtW0bpsBvBmvKN5L6aHEH8Rf9yQTh8VGR6DGdrX0LK7RhzQVLgeLtnbvDXWAD9p Rfw39LJWwVwNC/UbbhTlOfnPCf0O9kCMy9zdt2p2w+6k/Kql2XbwCzbhKSgf3c6l YYLr0y9Xw6HujixW/aaS4LKegnAX9y2L1oNTtdOdFKbjpLuNAWJ3W10eSmCjSeWa BYP+l7L1x6HkrHmPMibJTwC7ruJuzCoCWlOMQ2Aiw2TdpoE0ZNw+5mTl3tz8xm2d hHRfApRqMENanidOV3qvCptT0wYCsEd+bnqCqdHHazNFV4NzeMYOUTRoshpmNE02 bhBzCUdDuMX55fLytkjdvl35u78gUOJ+0ZSJ5wy+qgzQkocxJahnj9rlnqkXKjHz B4xmwGvUdq+mX1YGSCjGJkbbKkdPrn5sRfHr9cQuqVtP5tLP4ZlirHM5ulHHNE1a BArZYirqaFXHEAd7eoWcoC4KPQnCNcIEWH09qsiaUR7Rswor70VzHNW5ey1wbLkg 3Ch70R2bB8oyKU75j0mI =62ko -----END PGP SIGNATURE----- --82I3+IH0IqGh5yIs-- From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 19 15:39:20 2013 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9C924E7E; Fri, 19 Apr 2013 15:39:20 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B947A8C6; Fri, 19 Apr 2013 15:39:19 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA27251; Fri, 19 Apr 2013 13:08:22 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <51711795.2030701@FreeBSD.org> Date: Fri, 19 Apr 2013 13:08:21 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130404 Thunderbird/17.0.5 MIME-Version: 1.0 To: FreeBSD Hackers Subject: sdt panic X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 15:39:20 -0000 #0 doadump (textdump=1) at pcpu.h:233 #1 0xffffffff80591d69 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:444 #2 0xffffffff8059210c in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:620 #3 0xffffffff80775fee in trap_fatal (frame=0xc, eva=18446744071571306953) at /usr/src/sys/amd64/amd64/trap.c:872 #4 0xffffffff80776112 in trap_pfault (frame=0xffffff8236165030, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:730 #5 0xffffffff807768ee in trap (frame=0xffffff8236165030) at /usr/src/sys/amd64/amd64/trap.c:463 #6 0xffffffff807601e3 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #7 0xffffffff81162c96 in dtrace_probe_lookup (prid=0, mod=0xffffff82361651e0 "", func=0xffffff82361651a0 "stat", name=0xffffff8236165160 "reg") at /usr/src/sys/modules/dtrace/dtrace/../../../cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c:7876 #8 0xffffffff810c2164 in sdt_probe_callback (probe=0xffffffff80b5e760, arg=) at /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c:133 #9 0xffffffff8058f9ec in sdt_probe_listall (prov=, callback_func=0xffffffff810c20e5 , arg=0x0) at /usr/src/sys/kern/kern_sdt.c:269 #10 0xffffffff810c224b in sdt_provider_entry (prov=, arg=) at /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c:145 #11 0xffffffff8058f26f in sdt_provider_listall_locked (callback_func=0xffffffff810c2236 , arg=0x0) at /usr/src/sys/kern/kern_sdt.c:246 #12 0xffffffff8058facb in sdt_provider_listall (callback_func=0xffffffff810c2236 , arg=0x0) at /usr/src/sys/kern/kern_sdt.c:230 #13 0xffffffff810c2234 in sdt_provide_probes (arg=0x0, desc=0xffffff82361651e0) at /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c:154 #14 0xffffffff8116249c in dtrace_probe_provide (desc=0x0, prv=0xfffffe0112520800) at /usr/src/sys/modules/dtrace/dtrace/../../../cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c:7984 #15 0xffffffff81162825 in dtrace_enabling_provide (prv=0xfffffe0112520800) at /usr/src/sys/modules/dtrace/dtrace/../../../cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c:11607 #16 0xffffffff81167d1d in dtrace_register (name=0xffffffff808a3bc7 "proc", pap=0xffffffff810c2398, priv=2, cr=0x0, pops=0xffffffff810c23c0, arg=0x0, idp=0xffffffff80b48c28) at /usr/src/sys/modules/dtrace/dtrace/../../../cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c:7486 #17 0xffffffff810c21f4 in sdt_provider_reg_callback (prov=, arg=) at /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c:182 #18 0xffffffff8058f26f in sdt_provider_listall_locked (callback_func=0xffffffff810c21be , arg=0x0) at /usr/src/sys/kern/kern_sdt.c:246 #19 0xffffffff8058f7da in sdt_register_callbacks (register_prov=0xffffffff810c21be , reg_prov_arg=0x0, deregister_prov=0xffffffff810c21af , dereg_prov_arg=0x0, register_probe=0xffffffff810c20e5 , reg_probe_arg=0x0) at /usr/src/sys/kern/kern_sdt.c:320 #20 0xffffffff810c20a8 in sdt_load (dummy=) at /usr/src/sys/modules/dtrace/sdt/../../../cddl/dev/sdt/sdt.c:195 #21 0xffffffff80573e23 in linker_load_module (kldname=, modname=0xfffffe000a156800 "sdt", parent=0x0, verinfo=0x0, lfpp=0xffffff8236165928) at /usr/src/sys/kern/kern_linker.c:241 #22 0xffffffff8057436f in kern_kldload (td=0xfffffe012812b940, file=0x0, fileid=0xffffff8236165974) at /usr/src/sys/kern/kern_linker.c:1041 #23 0xffffffff80574533 in sys_kldload (td=0xfffffe012812b940, uap=) at /usr/src/sys/kern/kern_linker.c:1075 #24 0xffffffff807754e4 in amd64_syscall (td=0xfffffe012812b940, traced=0) at subr_syscall.c:135 #25 0xffffffff807604c7 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 My understanding is that this happens if sdt.ko is loaded later than some other dtrace module(s)/providers and there are any (non-sdt) probes active at the time of loading. It seems to be caused by sdt_provider_entry() being called on a sdt-based provider that has not been assigned an id (yet). -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 19 18:17:52 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id EBFA1E2D for ; Fri, 19 Apr 2013 18:17:52 +0000 (UTC) (envelope-from shonnur@chelsio.com) Received: from stargate.chelsio.com (stargate.chelsio.com [67.207.112.58]) by mx1.freebsd.org (Postfix) with ESMTP id 7A67E105D for ; Fri, 19 Apr 2013 18:17:52 +0000 (UTC) Received: from maui.asicdesigners.com (maui.asicdesigners.com [10.192.180.15]) by stargate.chelsio.com (8.13.1/8.13.1) with SMTP id r3J9hOsS001128; Fri, 19 Apr 2013 02:43:24 -0700 Received: from corona.asicdesigners.com ([10.192.160.6]) by maui.asicdesigners.com with Microsoft SMTPSVC(6.0.3790.4675); Fri, 19 Apr 2013 02:43:23 -0700 Received: from NICE.asicdesigners.com (10.192.160.7) by corona.asicdesigners.com (10.192.160.6) with Microsoft SMTP Server (TLS) id 8.3.83.0; Fri, 19 Apr 2013 02:43:23 -0700 Received: from NICE.asicdesigners.com ([fe80::51b2:ba95:9d72:babc]) by nice.asicdesigners.com ([fe80::51b2:ba95:9d72:babc%15]) with mapi id 14.02.0247.003; Fri, 19 Apr 2013 02:43:23 -0700 From: Sreenivasa Honnur To: "Andrey V. Elsukov" Subject: RE: IPV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) Thread-Topic: IPV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) Thread-Index: AQHOPDvZutqyOvZawUK9KhABruxLHJjc8tlggABZPZA= Date: Fri, 19 Apr 2013 09:43:22 +0000 Message-ID: References: <516FF961.5070100@yandex.ru> <516FF9E1.5030303@yandex.ru> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.193.190.128] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginalArrivalTime: 19 Apr 2013 09:43:23.0708 (UTC) FILETIME=[559EF7C0:01CE3CE2] Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 18:17:53 -0000 roundhay# sysctl -a | grep jailed security.jail.jailed: 0 Oh, of course, I mean, what do you have in td->td_ucred? [Sreenivas] not sure, I referred to socreate() usage in the kernel source a= nd used it. One more observation. Interface "cxgbe1" has a IPv6 interface and an IPv4 i= nterface, print in ifa_ifwithaddr_internal () shows like below ifp->xname:cxgbe1 ifp->dname:cxgbe ifa->ifa_addr->sa_family:18 addr->sa_family:28 =3D=3D> this is for AF_LINK ifa->ifa_addr->sa_family:2 addr->sa_family:28 =3D=3D> for AF_INET Though cxgbe1 has a IPv6 interface there is no entry for AF_INET6 Interface "cxgbe3" is not configured with any IP, print in ifa_ifwithaddr_i= nternal () shows like below ifp->xname:cxgbe3 ifp->dname:cxgbe ifa->ifa_addr->sa_family:18 addr->sa_family:28 =3D=3D> for AF_LINK roundhay# ifconfig cxgbe1 cxgbe1: flags=3D8843 metric 0 mtu 1= 500 options=3D6c07bb ether 00:07:43:11:89:88 inet6 2010::102 prefixlen 64 inet6 fe80::207:43ff:fe11:8988%cxgbe1 prefixlen 64 scopeid 0xd inet 16.1.1.154 netmask 0xff000000 broadcast 16.255.255.255 nd6 options=3D21 media: Ethernet 10Gbase-SR status: active roundhay# iscsictl -S target=3DALL Failed to start Target iqn.2004-05.com.chelsio.target1 roundhay# ifconfig c= xgbe3 cxgbe3: flags=3D8802 metric 0 mtu 1500 options=3D6c07bb ether 00:07:43:11:89:98 nd6 options=3D29 media: Ethernet 10Gbase-SR status: no carrier -----Original Message----- From: Andrey V. Elsukov [mailto:bu7cher@yandex.ru] Sent: Thursday, April 18, 2013 7:19 PM To: Sreenivasa Honnur Cc: freebsd-hackers@freebsd.org Subject: Re: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assi= gn requested address */) On 18.04.2013 17:47, Andrey V. Elsukov wrote: > On 18.04.2013 15:37, Sreenivasa Honnur wrote: >> I have a ipv6 interface(ping6 to a remove ipv6 works) when I try to=20 >> bind to this address through a socket program "sobind" fails with=20 >> "49" as return value. If I give "saddr6.sin6_addr =3D in6addr_any;" >> sobind works. >> >> Any idea what could be going wrong here? >=20 > What value has the sysctl variable security.jail.jailed? Oh, of course, I mean, what do you have in td->td_ucred? -- WBR, Andrey V. Elsukov From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 19 18:17:53 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1756FE2E for ; Fri, 19 Apr 2013 18:17:53 +0000 (UTC) (envelope-from shonnur@chelsio.com) Received: from stargate.chelsio.com (stargate.chelsio.com [67.207.112.58]) by mx1.freebsd.org (Postfix) with ESMTP id EC4BA105F for ; Fri, 19 Apr 2013 18:17:52 +0000 (UTC) Received: from maui.asicdesigners.com (maui.asicdesigners.com [10.192.180.15]) by stargate.chelsio.com (8.13.1/8.13.1) with SMTP id r3J4aYvu032495; Thu, 18 Apr 2013 21:36:34 -0700 Received: from corona.asicdesigners.com ([10.192.160.6]) by maui.asicdesigners.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 18 Apr 2013 21:36:33 -0700 Received: from NICE.asicdesigners.com (10.192.160.7) by corona.asicdesigners.com (10.192.160.6) with Microsoft SMTP Server (TLS) id 8.3.83.0; Thu, 18 Apr 2013 21:36:33 -0700 Received: from NICE.asicdesigners.com ([fe80::51b2:ba95:9d72:babc]) by nice.asicdesigners.com ([fe80::51b2:ba95:9d72:babc%15]) with mapi id 14.02.0247.003; Thu, 18 Apr 2013 21:36:33 -0700 From: Sreenivasa Honnur To: "Andrey V. Elsukov" Subject: RE: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) Thread-Topic: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assign requested address */) Thread-Index: AQHOPDvZutqyOvZawUK9KhABruxLHJjc8tlg Date: Fri, 19 Apr 2013 04:36:32 +0000 Message-ID: References: <516FF961.5070100@yandex.ru> <516FF9E1.5030303@yandex.ru> In-Reply-To: <516FF9E1.5030303@yandex.ru> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.193.190.128] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginalArrivalTime: 19 Apr 2013 04:36:33.0506 (UTC) FILETIME=[78491420:01CE3CB7] Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 18:17:53 -0000 roundhay# sysctl -a | grep jailed security.jail.jailed: 0 Oh, of course, I mean, what do you have in td->td_ucred? [Sreenivas] not sure, I referred to socreate() usage in the kernel source a= nd used it. One more observation. Interface "cxgbe1" has a IPv6 interface and an IPv4 i= nterface, print in ifa_ifwithaddr_internal () shows like below ifp->xname:cxgbe1 ifp->dname:cxgbe ifa->ifa_addr->sa_family:18 addr->sa_family:28 =3D=3D> this is for AF_LINK ifa->ifa_addr->sa_family:2 addr->sa_family:28 =3D=3D> for AF_INET Though cxgbe1 has a IPv6 interface there is no entry for AF_INET6 Interface "cxgbe3" is not configured with any IP, print in ifa_ifwithaddr_i= nternal () shows like below ifp->xname:cxgbe3 ifp->dname:cxgbe ifa->ifa_addr->sa_family:18 addr->sa_family:28 =3D=3D> for AF_LINK roundhay# ifconfig cxgbe1 cxgbe1: flags=3D8843 metric 0 mtu 1= 500 options=3D6c07bb ether 00:07:43:11:89:88 inet6 2010::102 prefixlen 64 inet6 fe80::207:43ff:fe11:8988%cxgbe1 prefixlen 64 scopeid 0xd inet 16.1.1.154 netmask 0xff000000 broadcast 16.255.255.255 nd6 options=3D21 media: Ethernet 10Gbase-SR status: active roundhay# iscsictl -S target=3DALL Failed to start Target iqn.2004-05.com.chelsio.target1 roundhay# ifconfig cxgbe3 cxgbe3: flags=3D8802 metric 0 mtu 1500 options=3D6c07bb ether 00:07:43:11:89:98 nd6 options=3D29 media: Ethernet 10Gbase-SR status: no carrier -----Original Message----- From: Andrey V. Elsukov [mailto:bu7cher@yandex.ru]=20 Sent: Thursday, April 18, 2013 7:19 PM To: Sreenivasa Honnur Cc: freebsd-hackers@freebsd.org Subject: Re: PV6 bind fails with 49 (#define EADDRNOTAVAIL 49 /* Can't assi= gn requested address */) On 18.04.2013 17:47, Andrey V. Elsukov wrote: > On 18.04.2013 15:37, Sreenivasa Honnur wrote: >> I have a ipv6 interface(ping6 to a remove ipv6 works) when I try to=20 >> bind to this address through a socket program "sobind" fails with=20 >> "49" as return value. If I give "saddr6.sin6_addr =3D in6addr_any;" >> sobind works. >> >> Any idea what could be going wrong here? >=20 > What value has the sysctl variable security.jail.jailed? Oh, of course, I mean, what do you have in td->td_ucred? -- WBR, Andrey V. Elsukov From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 19 18:21:34 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 9C7CB1B6 for ; Fri, 19 Apr 2013 18:21:34 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from qmta12.emeryville.ca.mail.comcast.net (qmta12.emeryville.ca.mail.comcast.net [IPv6:2001:558:fe2d:44:76:96:27:227]) by mx1.freebsd.org (Postfix) with ESMTP id 83D111101 for ; Fri, 19 Apr 2013 18:21:34 +0000 (UTC) Received: from omta07.emeryville.ca.mail.comcast.net ([76.96.30.59]) by qmta12.emeryville.ca.mail.comcast.net with comcast id RoTl1l0021GXsucACsUbMy; Fri, 19 Apr 2013 16:28:35 +0000 Received: from koitsu.strangled.net ([67.180.84.87]) by omta07.emeryville.ca.mail.comcast.net with comcast id RsUa1l00e1t3BNj8UsUbYc; Fri, 19 Apr 2013 16:28:35 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id BF87373A33; Fri, 19 Apr 2013 09:28:34 -0700 (PDT) Date: Fri, 19 Apr 2013 09:28:34 -0700 From: Jeremy Chadwick To: freebsd-hackers@freebsd.org Subject: Rebooting from loader causes a "fault" in VMware Workstation Message-ID: <20130419162834.GA90217@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20121106; t=1366388915; bh=OpikcpHl421BfsS44/DvaAuQL/kh6SFv99cq+uS0Uko=; h=Received:Received:Received:Date:From:To:Subject:Message-ID: MIME-Version:Content-Type; b=fNwd1OJLUjXvKz3gXlMoLDaQEeQcWNtjZHe71mKHcTl0mau8HjrzMKa9Le70Oqkwz /NtVZxw1Cim4ebDRWxU8ud0QRvyeiNPu/w5lB/wW2xZfRvwT5g0ypxiw135oi9VEi5 iiwkOWr9YLqQqIYEb/8BKxAkHj68tH3Fg5T5+Q8lCCeoEflBSVy8VfeZVsrn8jthAU kvhLgRFpuEKgBfptOu/pKrF05K7MnwQ+6KzY0/vyzbOdbjSpUwN3Ff7AYFnX6+Vs9w EiaceaiW3PQ57sua+0DW77R8O7TpcWW2qZXAVRkGWDvwlhuokS366ZLVA2LA6sjXHU xVcRGgP2BzSWg== X-Mailman-Approved-At: Fri, 19 Apr 2013 18:40:06 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 18:21:34 -0000 (Please keep me CC'd as I'm not subscribed to -hackers) When running FreeBSD under VMware Workstation (I'm using 9.0.1, but this issue has existed for many years now, I remember it occurring on Workstation 6.x), the following is reproducible: 1. Power on + boot FreeBSD VM 2. At loader menu, press "3" to reboot 3. Loader prints "Rebooting..." 4. VMware proceeds to show the following message in a dialog box: "A fault has occurred causing a virtual CPU to enter the shutdown state. If this fault had occurred outside of a virtual machine, it would have caused the physical machine to restart. The shutdown state can be reached incorrectly configuring the virtual machine, a bug in the guest operating system, or a problem in VMware Workstation." It can also happen when dropping to the loader prompt and doing "reboot". It *does not* happen when booting fully into FreeBSD and issuing "shutdown -r now". Likewise, hw.acpi.disable_on_reboot and hw.acpi.handle_reboot have no bearing (e.g. changing either of those to 1 (default = 0) then doing "shutdown -r now" to try and induce the problem). So it seems the issue is specific to the bootstrap/loader env. FreeBSD 9.1-STABLE is being used, but I've seen this happen with FreeBSD 8.x as well as 7.x. It does not happen with other OSes like Linux and Solaris. I have not tried other VM systems (VirtualBox, etc.) but I imagine they might just silently deal with the situation rather than provide a useful message (although I know VirtualBox has an amazingly detailed debugger). I've looked at sys/boot/i386/loader/main.c (func command_reboot()) and the actual magic seems to happen inside of __exit. __exit comes from sys/boot/i386/btx/lib/btxsys.s, which zeros eax then issues INT 0x30 (syscall interrupt). That lead me to this: http://www.freebsd.org/doc/en/books/arch-handbook/book.html Eek. x86 architecture is a lot different than I remember it being in my 386 days, so this is all a bit over my head. -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 19 18:53:48 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DF834806 for ; Fri, 19 Apr 2013 18:53:48 +0000 (UTC) (envelope-from carl.shapiro@gmail.com) Received: from mail-qe0-f45.google.com (mail-qe0-f45.google.com [209.85.128.45]) by mx1.freebsd.org (Postfix) with ESMTP id A547C16AB for ; Fri, 19 Apr 2013 18:53:48 +0000 (UTC) Received: by mail-qe0-f45.google.com with SMTP id 1so2904004qee.4 for ; Fri, 19 Apr 2013 11:53:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type; bh=dLLa+ibREzH7kQujv/SkDCDSWb0RAWuezP8m2ZiCneY=; b=D+jtxA/LeNjUAx9b5IXzFxTjQLND0BzejwS95tOkACjM/dLwKd7P01ZHl5J+0/WOkB /7yMuz2+NQwQC6Pnzo7Cp3zFiHmLpytJIn4mN2V0JHC1P6sqQu11WS9tXJVZ8Ml+fZAZ 5awbnnySuWd9Eif0ZQMoYiIH5849FJoV3Fqz13YmBz630QVdfS580DhKtZ5/iWxC9HXW mhm0BxC+GsYhWhEIEwJZNwDjMI+GkFbyIsqZIdycIYictJZ8iFa3ikWxt5HsjkYwPwC9 RvMHkNEQyAu4TSoVf2Xt9fWOf5ySD3D14ni4tbIzo4lQq6ML4DA0RzVFFS8xoLuh3ZF9 ZUPA== X-Received: by 10.49.82.4 with SMTP id e4mr16938207qey.62.1366397622665; Fri, 19 Apr 2013 11:53:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.49.130.225 with HTTP; Fri, 19 Apr 2013 11:53:22 -0700 (PDT) In-Reply-To: <20130419124942.GA67273@kib.kiev.ua> References: <20130417082143.GW2930@kib.kiev.ua> <20130419124942.GA67273@kib.kiev.ua> From: Carl Shapiro Date: Fri, 19 Apr 2013 11:53:22 -0700 Message-ID: Subject: Re: MADV_FREE and wait4 EFAULT To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 18:53:48 -0000 On Fri, Apr 19, 2013 at 5:49 AM, Konstantin Belousov wrote: > It would be of some interest to see the evidence. > Certainly. Here is some of the debugging messages that I added to my application. The first line is a print statement that executes after the system call returns. (As an aside, we issue system calls directly and do not link against the C library.) The other 2 lines of interest are output from the dump of /proc/curproc/map that correspond to the status and rusage addresses. wait4 returned EFAULT, status is 0xc20021c0e8 rusage is 0xc2000eaf30 ... 0xc1ffff0000 0xc200100000 245 0 0xfffffe07cd9ebbc8 rw- 1 0 0x3000 COW NNC default - CH 1001 ... 0xc200200000 0xc200300000 250 0 0xfffffe0215e3ed98 rw- 1 0 0x3000 COW NNC default - CH 1001 I realize this might not be satisfying but I am happy to provide any other information you might be interested in. Is your code multithreaded ? > Yes. The test case is required to decide whether the bug is in the application > or in the OS. > To be clear, I do not have a strong reason to believe there bug is in FreeBSD. My original enquiry was solely into whether we were misusing MADV_FREE pages. However, the wait4 failure is very suspicious because the only two addresses written to are "out" parameters. > There is no difference in the access performed by copyout vs. access > caused > by the usermode write. > Thanks. > No, I cannot give an advice, I think we should first decide which code > to blame. > Well, I have a strong suspicion the problem is caused by my application for the reasons mentioned above. > BTW, you could try enabling sysctl machdep.uprintf_signal. This does not seem to be available in my kernel but I could patch this change... http://svnweb.freebsd.org/base/head/sys/amd64/amd64/trap.c?r1=239251&r2=239252& ...and it looks like I need to rebuild my kernel anyway to enable DTrace. > Oh, you did not > specified the architecture and version of the system. Sorry, here is what "uname -mv" says FreeBSD 9.1-RELEASE #0 r243825: Tue Dec 4 09:23:10 UTC 2012 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 19 22:48:56 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D36CD6A4 for ; Fri, 19 Apr 2013 22:48:56 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (unknown [IPv6:2001:610:1108:5012::107]) by mx1.freebsd.org (Postfix) with ESMTP id 4118A2D8 for ; Fri, 19 Apr 2013 22:48:56 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id A777012013B for ; Sat, 20 Apr 2013 00:48:39 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id 900482848C; Sat, 20 Apr 2013 00:48:39 +0200 (CEST) Date: Sat, 20 Apr 2013 00:48:39 +0200 From: Jilles Tjoelker To: freebsd-hackers@freebsd.org Subject: [patch] accept4 Message-ID: <20130419224839.GA69212@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 22:48:56 -0000 The accept4() function, compared to accept(), allows setting the new file descriptor atomically close-on-exec and explicitly controlling the non-blocking status on the new socket. (Note that the latter point means that accept() is not equivalent to any form of accept4().) The linuxulator's accept4 implementation leaves a race window where the new file descriptor is not close-on-exec because it calls sys_accept(). This implementation leaves no such race window (by using falloc() flags). The linuxulator could be fixed and simplified by using the new code. The comms/openobex port now compiles again without the patch-lib_cloexec.h patch. More information about API extensions to set new file descriptors atomically close-on-exec is at https://wiki.freebsd.org/AtomicCloseOnExec I'm also working on pipe2() (using linuxulator work) and dup3() (patch from Jukka A. Ukkonen). commit 2e410d9260d12328b689b0938e09d09649b0460a Author: Jilles Tjoelker Date: Sat Mar 30 21:44:14 2013 +0100 Add accept4() system call. diff --git a/lib/libc/sys/Makefile.inc b/lib/libc/sys/Makefile.inc index fef0f3c..105f469 100644 --- a/lib/libc/sys/Makefile.inc +++ b/lib/libc/sys/Makefile.inc @@ -270,6 +270,7 @@ MAN+= sctp_generic_recvmsg.2 \ wait.2 \ write.2 +MLINKS+=accept.2 accept4.2 MLINKS+=access.2 eaccess.2 \ access.2 faccessat.2 MLINKS+=brk.2 sbrk.2 diff --git a/lib/libc/sys/Symbol.map b/lib/libc/sys/Symbol.map index 6faa0af..149fa41 100644 --- a/lib/libc/sys/Symbol.map +++ b/lib/libc/sys/Symbol.map @@ -378,6 +378,7 @@ FBSD_1.2 { }; FBSD_1.3 { + accept4; bindat; cap_fcntls_get; cap_fcntls_limit; @@ -461,6 +462,8 @@ FBSDprivate_1.0 { __sys_abort2; _accept; __sys_accept; + _accept4; + __sys_accept4; _access; __sys_access; _acct; diff --git a/lib/libc/sys/accept.2 b/lib/libc/sys/accept.2 index 978b948..e8bccaf 100644 --- a/lib/libc/sys/accept.2 +++ b/lib/libc/sys/accept.2 @@ -41,6 +41,8 @@ .In sys/socket.h .Ft int .Fn accept "int s" "struct sockaddr * restrict addr" "socklen_t * restrict addrlen" +.Ft int +.Fn accept4 "int s" "struct sockaddr * restrict addr" "socklen_t * restrict addrlen" "int flags" .Sh DESCRIPTION The argument .Fa s @@ -66,6 +68,26 @@ and signals from the original socket .Fa s . .Pp +The +.Fn accept4 +system call is similar, +but the +.Dv O_NONBLOCK +property of the new socket is instead determined by the +.Dv SOCK_NONBLOCK +flag in the +.Fa flags +argument, +the +.Dv O_ASYNC +property is cleared, +the signal destination is cleared +and the close-on-exec flag on the new file descriptor can be set via the +.Dv SOCK_CLOEXEC +flag in the +.Fa flags +argument. +.Pp If no pending connections are present on the queue, and the original socket is not marked as non-blocking, @@ -141,13 +163,15 @@ properties and the signal destination being inherited, but should set them explicitly using .Xr fcntl 2 . .Sh RETURN VALUES -The call returns \-1 on error. -If it succeeds, it returns a non-negative +These calls return \-1 on error. +If they succeed, they return a non-negative integer that is a descriptor for the accepted socket. .Sh ERRORS The .Fn accept -system call will fail if: +and +.Fn accept4 +system calls will fail if: .Bl -tag -width Er .It Bq Er EBADF The descriptor is invalid. @@ -180,6 +204,16 @@ are present to be accepted. A connection arrived, but it was closed while waiting on the listen queue. .El +.Pp +The +.Fn accept4 +system call will also fail if: +.Bl -tag -width Er +.It Bq Er EINVAL +The +.Fa flags +argument is invalid. +.El .Sh SEE ALSO .Xr bind 2 , .Xr connect 2 , @@ -194,3 +228,8 @@ The .Fn accept system call appeared in .Bx 4.2 . +.Pp +The +.Fn accept4 +system call appeared in +.Fx 10.0 . diff --git a/lib/libthr/pthread.map b/lib/libthr/pthread.map index 355edea..bbbd930e 100644 --- a/lib/libthr/pthread.map +++ b/lib/libthr/pthread.map @@ -181,6 +181,7 @@ FBSDprivate_1.0 { ___wait; ___waitpid; __accept; + __accept4; __aio_suspend; __close; __connect; @@ -408,3 +409,7 @@ FBSD_1.2 { setcontext; swapcontext; }; + +FBSD_1.3 { + accept4; +}; diff --git a/lib/libthr/thread/thr_syscalls.c b/lib/libthr/thread/thr_syscalls.c index 2327d74..7a08302 100644 --- a/lib/libthr/thread/thr_syscalls.c +++ b/lib/libthr/thread/thr_syscalls.c @@ -101,6 +101,7 @@ extern pid_t __waitpid(pid_t, int *, int); extern int __sys_aio_suspend(const struct aiocb * const[], int, const struct timespec *); extern int __sys_accept(int, struct sockaddr *, socklen_t *); +extern int __sys_accept4(int, struct sockaddr *, socklen_t *, int); extern int __sys_connect(int, const struct sockaddr *, socklen_t); extern int __sys_fsync(int); extern int __sys_msync(void *, size_t, int); @@ -129,6 +130,7 @@ int ___usleep(useconds_t useconds); pid_t ___wait(int *); pid_t ___waitpid(pid_t, int *, int); int __accept(int, struct sockaddr *, socklen_t *); +int __accept4(int, struct sockaddr *, socklen_t *, int); int __aio_suspend(const struct aiocb * const iocbs[], int, const struct timespec *); int __close(int); @@ -176,6 +178,26 @@ __accept(int s, struct sockaddr *addr, socklen_t *addrlen) return (ret); } +__weak_reference(__accept4, accept4); + +/* + * Cancellation behavior: + * If thread is canceled, no socket is created. + */ +int +__accept4(int s, struct sockaddr *addr, socklen_t *addrlen, int flags) +{ + struct pthread *curthread; + int ret; + + curthread = _get_curthread(); + _thr_cancel_enter(curthread); + ret = __sys_accept4(s, addr, addrlen, flags); + _thr_cancel_leave(curthread, ret == -1); + + return (ret); +} + __weak_reference(__aio_suspend, aio_suspend); int diff --git a/sys/compat/freebsd32/syscalls.master b/sys/compat/freebsd32/syscalls.master index 0a40ab2..2cbdf31 100644 --- a/sys/compat/freebsd32/syscalls.master +++ b/sys/compat/freebsd32/syscalls.master @@ -1022,3 +1022,7 @@ int namelen); } 540 AUE_CHFLAGSAT NOPROTO { int chflagsat(int fd, const char *path, \ u_long flags, int atflag); } +541 AUE_ACCEPT NOPROTO { int accept4(int s, \ + struct sockaddr * __restrict name, \ + __socklen_t * __restrict anamelen, \ + int flags); } diff --git a/sys/kern/capabilities.conf b/sys/kern/capabilities.conf index f54c25d..0b64503 100644 --- a/sys/kern/capabilities.conf +++ b/sys/kern/capabilities.conf @@ -78,6 +78,7 @@ abort2 ## relies on existing bindings on a socket, subject to capability rights. ## accept +accept4 ## ## Allow AIO operations by file descriptor, subject to capability rights. diff --git a/sys/kern/syscalls.master b/sys/kern/syscalls.master index 4b3348f..922db30 100644 --- a/sys/kern/syscalls.master +++ b/sys/kern/syscalls.master @@ -972,5 +972,9 @@ int namelen); } 540 AUE_CHFLAGSAT STD { int chflagsat(int fd, const char *path, \ u_long flags, int atflag); } +541 AUE_ACCEPT STD { int accept4(int s, \ + struct sockaddr * __restrict name, \ + __socklen_t * __restrict anamelen, \ + int flags); } ; Please copy any additions and changes to the following compatability tables: ; sys/compat/freebsd32/syscalls.master diff --git a/sys/kern/uipc_syscalls.c b/sys/kern/uipc_syscalls.c index ac2fd42..4bbd595 100644 --- a/sys/kern/uipc_syscalls.c +++ b/sys/kern/uipc_syscalls.c @@ -97,10 +97,18 @@ __FBSDID("$FreeBSD$"); #endif /* SCTP */ #endif /* INET || INET6 */ +/* + * Flags for accept1() and kern_accept4(), in addition to SOCK_CLOEXEC + * and SOCK_NONBLOCK. + */ +#define ACCEPT4_INHERIT 0x1 +#define ACCEPT4_COMPAT 0x2 + static int sendit(struct thread *td, int s, struct msghdr *mp, int flags); static int recvit(struct thread *td, int s, struct msghdr *mp, void *namelenp); -static int accept1(struct thread *td, struct accept_args *uap, int compat); +static int accept1(struct thread *td, int s, struct sockaddr *uname, + socklen_t *anamelen, int flags); static int do_sendfile(struct thread *td, struct sendfile_args *uap, int compat); static int getsockname1(struct thread *td, struct getsockname_args *uap, int compat); @@ -317,49 +325,46 @@ sys_listen(td, uap) * accept1() */ static int -accept1(td, uap, compat) +accept1(td, s, uname, anamelen, flags) struct thread *td; - struct accept_args /* { - int s; - struct sockaddr * __restrict name; - socklen_t * __restrict anamelen; - } */ *uap; - int compat; + int s; + struct sockaddr *uname; + socklen_t *anamelen; + int flags; { struct sockaddr *name; socklen_t namelen; struct file *fp; int error; - if (uap->name == NULL) - return (kern_accept(td, uap->s, NULL, NULL, NULL)); + if (uname == NULL) + return (kern_accept4(td, s, NULL, NULL, flags, NULL)); - error = copyin(uap->anamelen, &namelen, sizeof (namelen)); + error = copyin(anamelen, &namelen, sizeof (namelen)); if (error) return (error); - error = kern_accept(td, uap->s, &name, &namelen, &fp); + error = kern_accept4(td, s, &name, &namelen, flags, &fp); /* * return a namelen of zero for older code which might * ignore the return value from accept. */ if (error) { - (void) copyout(&namelen, - uap->anamelen, sizeof(*uap->anamelen)); + (void) copyout(&namelen, anamelen, sizeof(*anamelen)); return (error); } - if (error == 0 && name != NULL) { + if (error == 0 && uname != NULL) { #ifdef COMPAT_OLDSOCK - if (compat) + if (flags & ACCEPT4_COMPAT) ((struct osockaddr *)name)->sa_family = name->sa_family; #endif - error = copyout(name, uap->name, namelen); + error = copyout(name, uname, namelen); } if (error == 0) - error = copyout(&namelen, uap->anamelen, + error = copyout(&namelen, anamelen, sizeof(namelen)); if (error) fdclose(td->td_proc->p_fd, fp, td->td_retval[0], td); @@ -372,6 +377,13 @@ int kern_accept(struct thread *td, int s, struct sockaddr **name, socklen_t *namelen, struct file **fp) { + return (kern_accept4(td, s, name, namelen, ACCEPT4_INHERIT, fp)); +} + +int +kern_accept4(struct thread *td, int s, struct sockaddr **name, + socklen_t *namelen, int flags, struct file **fp) +{ struct filedesc *fdp; struct file *headfp, *nfp = NULL; struct sockaddr *sa = NULL; @@ -400,7 +412,7 @@ kern_accept(struct thread *td, int s, struct sockaddr **name, if (error != 0) goto done; #endif - error = falloc(td, &nfp, &fd, 0); + error = falloc(td, &nfp, &fd, (flags & SOCK_CLOEXEC) ? O_CLOEXEC : 0); if (error) goto done; ACCEPT_LOCK(); @@ -441,7 +453,10 @@ kern_accept(struct thread *td, int s, struct sockaddr **name, TAILQ_REMOVE(&head->so_comp, so, so_list); head->so_qlen--; - so->so_state |= (head->so_state & SS_NBIO); + if (flags & ACCEPT4_INHERIT) + so->so_state |= (head->so_state & SS_NBIO); + else + so->so_state |= (flags & SOCK_NONBLOCK) ? SS_NBIO : 0; so->so_qstate &= ~SQ_COMP; so->so_head = NULL; @@ -454,9 +469,15 @@ kern_accept(struct thread *td, int s, struct sockaddr **name, /* connection has been removed from the listen queue */ KNOTE_UNLOCKED(&head->so_rcv.sb_sel.si_note, 0); - pgid = fgetown(&head->so_sigio); - if (pgid != 0) - fsetown(pgid, &so->so_sigio); + if (flags & ACCEPT4_INHERIT) { + pgid = fgetown(&head->so_sigio); + if (pgid != 0) + fsetown(pgid, &so->so_sigio); + } else { + fflag &= ~(FNONBLOCK | FASYNC); + if (flags & SOCK_NONBLOCK) + fflag |= FNONBLOCK; + } finit(nfp, fflag, DTYPE_SOCKET, so, &socketops); /* Sync socket nonblocking/async state with file flags */ @@ -527,7 +548,18 @@ sys_accept(td, uap) struct accept_args *uap; { - return (accept1(td, uap, 0)); + return (accept1(td, uap->s, uap->name, uap->anamelen, ACCEPT4_INHERIT)); +} + +int +sys_accept4(td, uap) + struct thread *td; + struct accept4_args *uap; +{ + if (uap->flags & ~(SOCK_CLOEXEC | SOCK_NONBLOCK)) + return (EINVAL); + + return (accept1(td, uap->s, uap->name, uap->anamelen, uap->flags)); } #ifdef COMPAT_OLDSOCK @@ -537,7 +569,8 @@ oaccept(td, uap) struct accept_args *uap; { - return (accept1(td, uap, 1)); + return (accept1(td, uap->s, uap->name, uap->anamelen, + ACCEPT4_INHERIT | ACCEPT4_COMPAT)); } #endif /* COMPAT_OLDSOCK */ diff --git a/sys/sys/socket.h b/sys/sys/socket.h index 41c85b6..90411d7 100644 --- a/sys/sys/socket.h +++ b/sys/sys/socket.h @@ -634,6 +634,7 @@ int accept(int, struct sockaddr * __restrict, socklen_t * __restrict); int bind(int, const struct sockaddr *, socklen_t); int connect(int, const struct sockaddr *, socklen_t); #if __BSD_VISIBLE +int accept4(int, struct sockaddr * __restrict, socklen_t * __restrict, int); int bindat(int, int, const struct sockaddr *, socklen_t); int connectat(int, int, const struct sockaddr *, socklen_t); #endif diff --git a/sys/sys/syscallsubr.h b/sys/sys/syscallsubr.h index fa0d351..49e8be1 100644 --- a/sys/sys/syscallsubr.h +++ b/sys/sys/syscallsubr.h @@ -60,6 +60,8 @@ int kern___getcwd(struct thread *td, u_char *buf, enum uio_seg bufseg, u_int buflen); int kern_accept(struct thread *td, int s, struct sockaddr **name, socklen_t *namelen, struct file **fp); +int kern_accept4(struct thread *td, int s, struct sockaddr **name, + socklen_t *namelen, int flags, struct file **fp); int kern_access(struct thread *td, char *path, enum uio_seg pathseg, int flags); int kern_accessat(struct thread *td, int fd, char *path, -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 19 22:49:40 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 336EF790 for ; Fri, 19 Apr 2013 22:49:40 +0000 (UTC) (envelope-from jrisom@gmail.com) Received: from mail-ie0-x22d.google.com (mail-ie0-x22d.google.com [IPv6:2607:f8b0:4001:c03::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 08FAD2E9 for ; Fri, 19 Apr 2013 22:49:39 +0000 (UTC) Received: by mail-ie0-f173.google.com with SMTP id k5so5251404iea.18 for ; Fri, 19 Apr 2013 15:49:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=m2yuMHAYtoJg/cM4X4zr2Xkuy8a5rZVp2SDhYbsiqiQ=; b=h6BWx2qZWduk4HNUEzWZ3GHdw9yg9PknU4X15Kh3s9WHSi6naV4y8aKXiyjd1b9x47 ZdhhLuXY7a3PYt0vHP37OqSS/0CzFPazJsj+sSFwmriiuZJBiwUhO7fh+ID3q6fiFfoQ ryYI+iIrM1RpwKgpaukGlXAvVV553Bapn/dhSoGmstyUTjZ1fyDm/Nv7DzsjSfWJ6qvR LLQS3sBEvGoqNnOepK/7+U/DW0vjWL2P3MOLNjA/TTPJoETfRX6FeKNqVSYIcgCSc6AM QzxTSKxC1XPOKHVRDQXxZoVw+Sn2nggFiSJIYEXOx1F9LR/Oqfk6v+pWXjBL02xgBQdN WW+g== X-Received: by 10.50.1.38 with SMTP id 6mr16760922igj.106.1366411779673; Fri, 19 Apr 2013 15:49:39 -0700 (PDT) Received: from [192.168.1.34] (c-98-212-197-211.hsd1.il.comcast.net. [98.212.197.211]) by mx.google.com with ESMTPS id dy5sm5417738igc.1.2013.04.19.15.49.38 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 19 Apr 2013 15:49:38 -0700 (PDT) Message-ID: <5171C9FE.8080409@gmail.com> Date: Fri, 19 Apr 2013 17:49:34 -0500 From: Joshua Isom User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Jeremy Chadwick Subject: Re: Rebooting from loader causes a "fault" in VMware Workstation References: <20130419162834.GA90217@icarus.home.lan> In-Reply-To: <20130419162834.GA90217@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Apr 2013 22:49:40 -0000 Basically, the loader finds a simple safe way to reboot that's worked since the 286, and VMWare doesn't like it. It's called a triple fault. FreeBSD and Linux even use it to reboot as a fail safe. Read sys/i386/i386/vm_machdep.c and cpu_reset_real to see how FreeBSD handles it. VMWare at least says that "it would have caused the physical machine to restart." Blame VMWare. On 4/19/2013 11:28 AM, Jeremy Chadwick wrote: > (Please keep me CC'd as I'm not subscribed to -hackers) > > When running FreeBSD under VMware Workstation (I'm using 9.0.1, but this > issue has existed for many years now, I remember it occurring on > Workstation 6.x), the following is reproducible: > > 1. Power on + boot FreeBSD VM > 2. At loader menu, press "3" to reboot > 3. Loader prints "Rebooting..." > 4. VMware proceeds to show the following message in a dialog box: > > "A fault has occurred causing a virtual CPU to enter the shutdown state. > If this fault had occurred outside of a virtual machine, it would have > caused the physical machine to restart. The shutdown state can be > reached incorrectly configuring the virtual machine, a bug in the guest > operating system, or a problem in VMware Workstation." > > It can also happen when dropping to the loader prompt and doing > "reboot". > > It *does not* happen when booting fully into FreeBSD and issuing > "shutdown -r now". Likewise, hw.acpi.disable_on_reboot and > hw.acpi.handle_reboot have no bearing (e.g. changing either of those to > 1 (default = 0) then doing "shutdown -r now" to try and induce the > problem). > > So it seems the issue is specific to the bootstrap/loader env. > > FreeBSD 9.1-STABLE is being used, but I've seen this happen with FreeBSD > 8.x as well as 7.x. It does not happen with other OSes like Linux and > Solaris. I have not tried other VM systems (VirtualBox, etc.) but I > imagine they might just silently deal with the situation rather than > provide a useful message (although I know VirtualBox has an amazingly > detailed debugger). > > I've looked at sys/boot/i386/loader/main.c (func command_reboot()) and > the actual magic seems to happen inside of __exit. > > __exit comes from sys/boot/i386/btx/lib/btxsys.s, which zeros eax then > issues INT 0x30 (syscall interrupt). That lead me to this: > > http://www.freebsd.org/doc/en/books/arch-handbook/book.html > > Eek. x86 architecture is a lot different than I remember it being in > my 386 days, so this is all a bit over my head. > From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 20 01:48:24 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3373CC82 for ; Sat, 20 Apr 2013 01:48:24 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from qmta10.emeryville.ca.mail.comcast.net (qmta10.emeryville.ca.mail.comcast.net [IPv6:2001:558:fe2d:43:76:96:30:17]) by mx1.freebsd.org (Postfix) with ESMTP id 0FC38931 for ; Sat, 20 Apr 2013 01:48:23 +0000 (UTC) Received: from omta18.emeryville.ca.mail.comcast.net ([76.96.30.74]) by qmta10.emeryville.ca.mail.comcast.net with comcast id S1iw1l0041bwxycAA1oNfW; Sat, 20 Apr 2013 01:48:22 +0000 Received: from koitsu.strangled.net ([67.180.84.87]) by omta18.emeryville.ca.mail.comcast.net with comcast id S1oM1l00W1t3BNj8e1oNLi; Sat, 20 Apr 2013 01:48:22 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id CA0EE73A33; Fri, 19 Apr 2013 18:48:21 -0700 (PDT) Date: Fri, 19 Apr 2013 18:48:21 -0700 From: Jeremy Chadwick To: Joshua Isom Subject: Re: Rebooting from loader causes a "fault" in VMware Workstation Message-ID: <20130420014821.GA98555@icarus.home.lan> References: <20130419162834.GA90217@icarus.home.lan> <5171C9FE.8080409@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5171C9FE.8080409@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20121106; t=1366422502; bh=qO4GxaMSFkmgABQvviWyh8jiAt2c7LFCks+F3n1mvfg=; h=Received:Received:Received:Date:From:To:Subject:Message-ID: MIME-Version:Content-Type; b=p5VB19MbMiRTrop41F5mvBIiRvhdU5FjGcdZI6H8dxv8yxFbUYXo46JuhlHRB9a+2 SQPV5wK5ojVpmR+q0ZEFLvUU098r7zzD8pzPWJ9Hm3OV60m7H/xng45CNtLICmhE+r SZnxNZ+6IFUiEaFcLMdf2Fcg3e+sJP+5xfBV8Pjqhkuc8M/TKk8cukVN/Jv+L4fQwk GDUXmygwmNbjWvtKsOApA7f8iftzs9NP7lyYySVR9Oa3Jq8uX27OaXPWXEWH4c2/DU 9t4/BxC3c5eLiRnjFuJ2K7AsQi+JmdYlfOVEG2VCqxUmvifWiM3CpJJ3ITUKDwi8NG Z0YTZugj9wtjA== X-Mailman-Approved-At: Sat, 20 Apr 2013 01:57:01 +0000 Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 01:48:24 -0000 On Fri, Apr 19, 2013 at 05:49:34PM -0500, Joshua Isom wrote: > Basically, the loader finds a simple safe way to reboot that's > worked since the 286, and VMWare doesn't like it. It's called a > triple fault. FreeBSD and Linux even use it to reboot as a fail > safe. Read sys/i386/i386/vm_machdep.c and cpu_reset_real to see how > FreeBSD handles it. VMWare at least says that "it would have caused > the physical machine to restart." Blame VMWare. > > On 4/19/2013 11:28 AM, Jeremy Chadwick wrote: > >(Please keep me CC'd as I'm not subscribed to -hackers) > > > >When running FreeBSD under VMware Workstation (I'm using 9.0.1, but this > >issue has existed for many years now, I remember it occurring on > >Workstation 6.x), the following is reproducible: > > > >1. Power on + boot FreeBSD VM > >2. At loader menu, press "3" to reboot > >3. Loader prints "Rebooting..." > >4. VMware proceeds to show the following message in a dialog box: > > > >"A fault has occurred causing a virtual CPU to enter the shutdown state. > >If this fault had occurred outside of a virtual machine, it would have > >caused the physical machine to restart. The shutdown state can be > >reached incorrectly configuring the virtual machine, a bug in the guest > >operating system, or a problem in VMware Workstation." > > > >It can also happen when dropping to the loader prompt and doing > >"reboot". > > > >It *does not* happen when booting fully into FreeBSD and issuing > >"shutdown -r now". Likewise, hw.acpi.disable_on_reboot and > >hw.acpi.handle_reboot have no bearing (e.g. changing either of those to > >1 (default = 0) then doing "shutdown -r now" to try and induce the > >problem). > > > >So it seems the issue is specific to the bootstrap/loader env. > > > >FreeBSD 9.1-STABLE is being used, but I've seen this happen with FreeBSD > >8.x as well as 7.x. It does not happen with other OSes like Linux and > >Solaris. I have not tried other VM systems (VirtualBox, etc.) but I > >imagine they might just silently deal with the situation rather than > >provide a useful message (although I know VirtualBox has an amazingly > >detailed debugger). > > > >I've looked at sys/boot/i386/loader/main.c (func command_reboot()) and > >the actual magic seems to happen inside of __exit. > > > >__exit comes from sys/boot/i386/btx/lib/btxsys.s, which zeros eax then > >issues INT 0x30 (syscall interrupt). That lead me to this: > > > >http://www.freebsd.org/doc/en/books/arch-handbook/book.html > > > >Eek. x86 architecture is a lot different than I remember it being in > >my 386 days, so this is all a bit over my head. I'm happy to open up a ticket with VMware about the issue as I'm a customer, but I find it a little odd that other operating systems do not exhibit this problem, including another BSD. Ones which reboot just fine from their bootloaders: - Linux -- so many that I don't know where to begin: ArchLinux 2012.10.06, CentOS 6.3, Debian 6.0.7, Finnix 1.0.5, Knoppix 7.0.4, Slackware 14.0, and Ubuntu 11.10 - OpenBSD 5.2 - OpenIndiana -- build 151a7 (server version) So when you say "Blame VMware", I'd be happy to, except there must be something FreeBSD's bootstraps are doing differently than everyone else that causes this oddity. Would you not agree? -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 20 09:55:28 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 94D4AE84; Sat, 20 Apr 2013 09:55:28 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) by mx1.freebsd.org (Postfix) with ESMTP id 0CFC41563; Sat, 20 Apr 2013 09:55:27 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id c10so1962176wiw.6 for ; Sat, 20 Apr 2013 02:55:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=c21KDMvgMopz2Lnv5Z8ImTr/Q3dk7PEu/hrlHfFObbg=; b=Snt5WevnCvbqPhiDM7JKk+CcOd74h/Rc0bnDraJM6lO1wfskRdz+p/hRx74LSkMEOH 3r5NKYSTqBtW2wfOhfyQz3fQA7gsgLRY/RRgFa3/yW/StgGMHeBaS2YiJzqDtq7K5sid yfezw98Z2zFFZhJHfln3w8W/ZNZNnLjW8lU3nyBK9NGGGUFSlMYEX7LSdKrLO/8pcAyD xs0mBC0kOPnbHy0xIzCl9D8ISM5+hR1V3T5zqWQHMupD59YfW8Ed08n1i3nMYfGfUSVe GxiUB3/ZQDadG///dRq13cJ8V18gsAehdi1f1NLFiLayBkHgwZNrlTuDC7tj9E7sXo22 79sQ== MIME-Version: 1.0 X-Received: by 10.194.103.72 with SMTP id fu8mr33296880wjb.42.1366451727082; Sat, 20 Apr 2013 02:55:27 -0700 (PDT) Received: by 10.216.139.72 with HTTP; Sat, 20 Apr 2013 02:55:26 -0700 (PDT) Date: Sat, 20 Apr 2013 12:55:26 +0300 Message-ID: Subject: WORLDTMP on a ram disk From: Kimmo Paasiala To: freebsd-hackers@freebsd.org, FreeBSD current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 09:55:28 -0000 Poking around the /usr/src/Makefile.inc1 I see that there's a variable called "WORLDTMP" that seems to set the location for temporary files during the world build. My question is now: Is it safe to put WORLDTMP on a ram disk, for example tmpfs(5)? Looking at the build process it seems to me that the temporary files are no longer needed after make buildworld has finished? -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 20 10:07:47 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id ED7C91A8 for ; Sat, 20 Apr 2013 10:07:47 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id A918915C9 for ; Sat, 20 Apr 2013 10:07:47 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-237-17.lns20.per1.internode.on.net [121.45.237.17]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id r3JCtZPa035647 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 19 Apr 2013 05:55:40 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <51713EC2.4030401@freebsd.org> Date: Fri, 19 Apr 2013 20:55:30 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Carl Shapiro Subject: Re: MADV_FREE and wait4 EFAULT References: <20130417082143.GW2930@kib.kiev.ua> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 10:07:48 -0000 On 4/19/13 5:51 AM, Carl Shapiro wrote: > On Wed, Apr 17, 2013 at 1:21 AM, Konstantin Belousov wrote: > >> Did you ensured with e.g. ktrace and procstat -v that your assumptions >> hold, i.e. the addresses supplied as wait4(2) arguments are valid ? >> Please provide the minimal test case demonstrating the behaviour. >> > Yes. I instrumented my code to check for a wait4 failure, print the > addresses of the status and rusage arguments, and dump the contents of > /proc/curproc/map. The addresses of the status and rusage arguments are > always in the range of a mapping and marked as read write. > > I have yet to distill the failure to a minimal test case. The test case I > do have is the test harness for the Go language. After running for about > 45 minutes I can observe a failure. I have been working to produce > something smaller and faster. > > >> MADV_FREE should only result in the possible lost of the previous >> content of the page, not in the faulting of the page access. From the >> inspection of the code, I do not see how MADV_FREE could result in >> the memory address becoming invalid. >> > I see. What has lead us to believe this might be an issue with page faults > is that writing zeroes to the page with memset before passing it to wait4 > makes the error go away. > > Do you have any advice about how one might go about instrumenting wait4 to > generate more information about a failed copyout? Are tools such as dtrace > useful in these situations or might it be too invasive? Because of the > protracted test cycle and my lack of knowledge in this area, conducting > experiments is quite painful at the moment. looks like a great example of something that dtrace should be able to help with. basically you can do a speculative dump of the stuff going on before the copyout and just store it in the case where the copyout fails. I'm just learning getting my dtrace training wheels but I think it may be able to give you what you need. > > Thanks, > > Carl > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 20 11:51:12 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6D9A5E0A for ; Sat, 20 Apr 2013 11:51:12 +0000 (UTC) (envelope-from jrisom@gmail.com) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) by mx1.freebsd.org (Postfix) with ESMTP id 40459189F for ; Sat, 20 Apr 2013 11:51:12 +0000 (UTC) Received: by mail-ie0-f177.google.com with SMTP id 9so5669630iec.36 for ; Sat, 20 Apr 2013 04:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:cc :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=/Ih+Us/exaIuVnHmavUy/l11QbXCIaNzhOt3OdsjlH8=; b=vYx+Xi9freD1yIsryeKXHR6LLys7ysm2CwzY7FFAajL6eR33hQVF+093dMTkvBjPZp I8XB3ui8Hs2kiZLHuc8cZzJ9dslq0Rd0Q4cjcF8Xv9PMzMHdHQCSex6pilQK1AQiorxC PyWCiBAEhguOVy80vmy3mlFtMjzZfw6IOcguokfC5WYRgsJtdx11ibVv94w1zY6c4UVe 7hEj41cbH84MwPYAQ09UER28neVJMI5GsxacjhUcTprXFe7b646F3atcGqiXaXYBEnNV bMqp4jOzmdydlS/962QiMQg247lVSRQdRVG8yV7VQIyeg2dQfEETBUhWQWTCyDnrRVYb EivA== X-Received: by 10.42.64.69 with SMTP id f5mr9704651ici.29.1366458671947; Sat, 20 Apr 2013 04:51:11 -0700 (PDT) Received: from [192.168.1.34] (c-98-212-197-211.hsd1.il.comcast.net. [98.212.197.211]) by mx.google.com with ESMTPS id b3sm205808igd.5.2013.04.20.04.51.10 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 20 Apr 2013 04:51:11 -0700 (PDT) Message-ID: <5172812A.10309@gmail.com> Date: Sat, 20 Apr 2013 06:51:06 -0500 From: Joshua Isom User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 MIME-Version: 1.0 To: Jeremy Chadwick Subject: Re: Rebooting from loader causes a "fault" in VMware Workstation References: <20130419162834.GA90217@icarus.home.lan> <5171C9FE.8080409@gmail.com> <20130420014821.GA98555@icarus.home.lan> In-Reply-To: <20130420014821.GA98555@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 11:51:12 -0000 On 4/19/2013 8:48 PM, Jeremy Chadwick wrote: > I'm happy to open up a ticket with VMware about the issue as I'm a > customer, but I find it a little odd that other operating systems do not > exhibit this problem, including another BSD. Ones which reboot just > fine from their bootloaders: > > - Linux -- so many that I don't know where to begin: ArchLinux > 2012.10.06, CentOS 6.3, Debian 6.0.7, Finnix 1.0.5, Knoppix 7.0.4, > Slackware 14.0, and Ubuntu 11.10 > - OpenBSD 5.2 > - OpenIndiana -- build 151a7 (server version) > > So when you say "Blame VMware", I'd be happy to, except there must be > something FreeBSD's bootstraps are doing differently than everyone else > that causes this oddity. Would you not agree? A triple fault is standard practice as a fail safe guarantee of reboot. It's used either as a reboot, switch to real mode(IBM OS/2), or catastrophic unrecoverable failure. By the looks of grub(Linux and Solaris), it either jumps to it's own instruction, hoping the bios catches it("tell the BIOS a boot failure, which may result in no effect"), or jumps to a location that I can't yet determine what code exists there. I can't seem to find OpenBSD's reboot method from OpenBSD's cvsweb, only an exit but not where that exit leads to. The native operating system is irrelevant, only the boot loader so all the Linux distributions and Solaris forks all count as "grub." Many other bootloaders don't even have the reboot option, just "fail." Here's barebox, a Das U-Boot fork: /** How to reset the machine? */ while(1) In any case, it's a bag of tricks, finding something that works and is "nice." We're talking 30 years of legacy. A triple fault, assuming the mbr and loader ignores or zeroes previous memory, is guaranteed and doesn't hang. From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 20 12:16:39 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A8AFE4A3; Sat, 20 Apr 2013 12:16:39 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) by mx1.freebsd.org (Postfix) with ESMTP id 7120D1ADE; Sat, 20 Apr 2013 12:16:39 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::15fb:7073:6a12:b988] (unknown [IPv6:2001:7b8:3a7:0:15fb:7073:6a12:b988]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 438B75C44; Sat, 20 Apr 2013 14:16:36 +0200 (CEST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: WORLDTMP on a ram disk From: Dimitry Andric In-Reply-To: Date: Sat, 20 Apr 2013 14:16:45 +0200 Content-Transfer-Encoding: 7bit Message-Id: <60A8F6C1-2A06-4646-9A28-83608FF23443@FreeBSD.org> References: To: Kimmo Paasiala X-Mailer: Apple Mail (2.1503) Cc: freebsd-hackers@freebsd.org, FreeBSD current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 12:16:39 -0000 On Apr 20, 2013, at 11:55, Kimmo Paasiala wrote: > Poking around the /usr/src/Makefile.inc1 I see that there's a variable > called "WORLDTMP" that seems to set the location for temporary files > during the world build. My question is now: > > Is it safe to put WORLDTMP on a ram disk, for example tmpfs(5)? > > Looking at the build process it seems to me that the temporary files > are no longer needed after make buildworld has finished? The problem is that you need those temporary files for installworld, and you must reboot with your new kernel before running that... From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 20 12:30:27 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AE5E37FC; Sat, 20 Apr 2013 12:30:27 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-wi0-x22d.google.com (mail-wi0-x22d.google.com [IPv6:2a00:1450:400c:c05::22d]) by mx1.freebsd.org (Postfix) with ESMTP id F216B1B7A; Sat, 20 Apr 2013 12:30:26 +0000 (UTC) Received: by mail-wi0-f173.google.com with SMTP id c10so2093165wiw.0 for ; Sat, 20 Apr 2013 05:30:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=WetEaqd0L2smFm4B0R6yqhJWyrGjd/RwWz/3ruCDZMo=; b=o3VczXVIFbpuK88HVI9X0J1AFo+F+KhgHQFAxegzqMhbO6Ur6dvhwY33Hys6SI7qHS TmyaSx5chOogxiYVlyPhk2GRWahWUbPOLoSfoaIK0QchSkm+X7p7aHGzi3AS4y8hPE/q zZLEfVM7/Sxp/SEw4aC83gAdJNTGM3/jKtcsPNUwWjOWayOFYjrMTLHFxPP/BErbQPSL b+zWqPNhq9PhaWN9Cu17+FzkVdq4sGx9HIjDJ/314ROiH7KiJ6WXlnOHXicAj08WcJEo n0t5MXAEOklIiakC9kqZAnT4AcJkEaTpKoCFCHlTz5xWl51jOzrtYuqYykg4dPgFP/RB fa2w== MIME-Version: 1.0 X-Received: by 10.194.157.138 with SMTP id wm10mr34326957wjb.28.1366461026164; Sat, 20 Apr 2013 05:30:26 -0700 (PDT) Received: by 10.216.139.72 with HTTP; Sat, 20 Apr 2013 05:30:26 -0700 (PDT) In-Reply-To: <60A8F6C1-2A06-4646-9A28-83608FF23443@FreeBSD.org> References: <60A8F6C1-2A06-4646-9A28-83608FF23443@FreeBSD.org> Date: Sat, 20 Apr 2013 15:30:26 +0300 Message-ID: Subject: Re: WORLDTMP on a ram disk From: Kimmo Paasiala To: Dimitry Andric Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers@freebsd.org, FreeBSD current X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 12:30:27 -0000 On Sat, Apr 20, 2013 at 3:16 PM, Dimitry Andric wrote: > On Apr 20, 2013, at 11:55, Kimmo Paasiala wrote: >> Poking around the /usr/src/Makefile.inc1 I see that there's a variable >> called "WORLDTMP" that seems to set the location for temporary files >> during the world build. My question is now: >> >> Is it safe to put WORLDTMP on a ram disk, for example tmpfs(5)? >> >> Looking at the build process it seems to me that the temporary files >> are no longer needed after make buildworld has finished? > > > The problem is that you need those temporary files for installworld, and > you must reboot with your new kernel before running that... > Well that was what I was after, I was somehow under the impression that the files in WORLDTMP would be moved to more permanent location under /usr/obj for installworld to find them. -Kimmo From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 20 15:32:18 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 178F2D08 for ; Sat, 20 Apr 2013 15:32:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 671FC16B for ; Sat, 20 Apr 2013 15:32:17 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r3KFW871085758; Sat, 20 Apr 2013 18:32:08 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.2 kib.kiev.ua r3KFW871085758 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r3KFW8m6085757; Sat, 20 Apr 2013 18:32:08 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 20 Apr 2013 18:32:08 +0300 From: Konstantin Belousov To: Carl Shapiro Subject: Re: MADV_FREE and wait4 EFAULT Message-ID: <20130420153208.GD67273@kib.kiev.ua> References: <20130417082143.GW2930@kib.kiev.ua> <20130419124942.GA67273@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="pZs/OQEoSSbxGlYw" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 15:32:18 -0000 --pZs/OQEoSSbxGlYw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Apr 19, 2013 at 11:53:22AM -0700, Carl Shapiro wrote: > On Fri, Apr 19, 2013 at 5:49 AM, Konstantin Belousov wrote: >=20 > > It would be of some interest to see the evidence. > > >=20 > Certainly. Here is some of the debugging messages that I added to my > application. The first line is a print statement that executes after the > system call returns. (As an aside, we issue system calls directly and do > not link against the C library.) The other 2 lines of interest are output > from the dump of /proc/curproc/map that correspond to the status and rusa= ge > addresses. >=20 > wait4 returned EFAULT, status is 0xc20021c0e8 rusage is 0xc2000eaf30 > ... > 0xc1ffff0000 0xc200100000 245 0 0xfffffe07cd9ebbc8 rw- 1 0 0x3000 COW NNC > default - CH 1001 > ... > 0xc200200000 0xc200300000 250 0 0xfffffe0215e3ed98 rw- 1 0 0x3000 COW NNC > default - CH 1001 >=20 > I realize this might not be satisfying but I am happy to provide any other > information you might be interested in. >=20 > Is your code multithreaded ? > > >=20 > Yes. Just observations/speculations: the addresses you shown are not the (usual) addresses for the malloc heap on amd64. It is possible to allocate and map enough shared libraries to make malloc start using these addresses, but more likely your app is using custom mmap() calls, possibly with the explicite address hints or MAP_FIXED. Am I right ? If yes and your app is multithreaded, it is possible that other thread performs some manipulations on the address space while current thread tries to access the range. >=20 > The test case is required to decide whether the bug is in the application > > or in the OS. > > >=20 > To be clear, I do not have a strong reason to believe there bug is in > FreeBSD. My original enquiry was solely into whether we were misusing > MADV_FREE pages. However, the wait4 failure is very suspicious because t= he > only two addresses written to are "out" parameters. In fact, the question is, why are you trying to access the memory after the MADV_FREE. Was it correctly marked as 'still used' to prevent mmaps there ? Note that all this is pure speculation. --pZs/OQEoSSbxGlYw Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRcrT3AAoJEJDCuSvBvK1B5noQAJXqtOP+Q7pavJEYcovI8dxp T9hwMEeu26AbMg1tC498o5lWFc4oO0evwbgTNje+f3Rn7DkRBClr1cDFmTi/Y594 Nzip21ywYCG9IYZJ/NI7txzQ7grFdSaaLh0NnC7a/7ctpXY2wKRO46DOAsEHxnkN RurnfgScIxwPXW7jWxAUapVZKmwxbThe3+v03it7GmVoMDqtSCtW+3d+RsyP9a3o QuY2XymUh2sqmpXsCNUP3qvu/RWfSQfhaTbyaMzyUOl9rGpPe4a20oSvxHbVkkgQ CtonV+A0j21nal1mx5gPiXPVwMQNBh2sb1QEeI4HJoQ/bdnfb9iq/WnoLDyfXYUg 6eD/9b3pp0vthoaIPgcb4fbDenRWNkjbRyrjYHrLl6fFM3X7TT0A2sLEPWogr6sO bXJHBNCP+Jd96dyHw0pJS396KPZIZ0zeo0lAIEzB2la8734q0kMXWf4Fb0/be80u OOmY7urPIo19FVHULFZhx8zPL5t5WcOiyHJCAYxk6vPs+QgOv3XcWdFjJzDm8X1T tSxDASPLLUohPWNzwItXg0+ri2kHs/hlfg2ejfOtNKsAb5xwyukf5IeyB51h5ryc u2Q0WJlJ/IlT+e0ZCjLdPnjXGACDU+HydhWnDXJvQV4uBApKErRw0qGewJaf/nFc u8aX6u6apcjQlHivYqDB =enUq -----END PGP SIGNATURE----- --pZs/OQEoSSbxGlYw-- From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 20 22:43:29 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D626BC1F for ; Sat, 20 Apr 2013 22:43:29 +0000 (UTC) (envelope-from waksmundzki@gmail.com) Received: from mail-ia0-x232.google.com (mail-ia0-x232.google.com [IPv6:2607:f8b0:4001:c02::232]) by mx1.freebsd.org (Postfix) with ESMTP id ACB221147 for ; Sat, 20 Apr 2013 22:43:29 +0000 (UTC) Received: by mail-ia0-f178.google.com with SMTP id j38so1047227iad.23 for ; Sat, 20 Apr 2013 15:43:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:from:content-type:content-transfer-encoding:subject :message-id:date:to:mime-version:x-mailer; bh=eSggNpUvnti+ixyfyVNaFofj2jqpWBBqwhEy4rO3fv0=; b=f2FfUUIlfbE41ZIRfShmHLRnq7GTKd65x8Tc/gKO2WmkFuBS/4w9TF+hDid26bKVWF hOo2QluAvnbOy15j2/7kZm8E/aa4dmXZSMF19pKUeeZsknYvsu2/bHujU3oaP7q6LcSs 3JbuxGvANjQzxxetyKe7ZMQNDdW6gRJfyzGQlJ6GmBIfmT5dcDEUoeAXmEABJv7lZ9Dk vEv2P9KFMYr7iFxWiR4W12wjgf3IZcKKiBLRf0/wcSV/Doq6KpKlk8Hv8Eazpfyr2yQC PBdqEV/f0326osO5Gw3ktO6E4bdLBU0U9GvAft7sd+FjuGbFJSNdNuBLkd/dJzMmEMCX q3nA== X-Received: by 10.50.118.7 with SMTP id ki7mr10013947igb.35.1366497808956; Sat, 20 Apr 2013 15:43:28 -0700 (PDT) Received: from [192.168.1.107] (c-98-253-129-150.hsd1.il.comcast.net. [98.253.129.150]) by mx.google.com with ESMTPSA id xc3sm9082580igb.10.2013.04.20.15.43.26 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 20 Apr 2013 15:43:27 -0700 (PDT) From: Robert Waksmundzki Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: NUMA, cpuset and malloc Message-Id: Date: Sat, 20 Apr 2013 17:43:26 -0500 To: freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) X-Mailer: Apple Mail (2.1503) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Apr 2013 22:43:29 -0000 On NUMA systems allocated memory is striped across local and non-local = banks in order to have consistent performance in case the task is = rescheduled to a different CPU socket. When a process is pinned to a single CPU socket with cpuset having the = memory allocator prefer local banks would probably improve performance. = Default system behavior would stay the same and the optimization would = only be triggered on big multi socket systems when administrator used = cpuset (command mostly used for performance optimization anyway). Is this something currently implemented in FreeBSD? Is this even a good = idea?=