From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 5 06:21:58 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7B775F56 for ; Sun, 5 Apr 2015 06:21:58 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 02B7AF95 for ; Sun, 5 Apr 2015 06:21:57 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t356LleV089188 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sun, 5 Apr 2015 08:21:48 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t356Lh3T000686 for ; Sun, 5 Apr 2015 08:21:43 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t356LbKT000683 for ; Sun, 5 Apr 2015 08:21:38 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Sun, 5 Apr 2015 08:21:37 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: freebsd-hackers@freebsd.org Subject: multiple X servers Message-ID: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Sun, 05 Apr 2015 08:21:48 +0200 (CEST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 06:21:58 -0000 can two (or more) X servers be run on one machine with machine with such config: - 2 GFX cards, say one intel embedded in CPU, other addon. - 2 keyboards - 2 mice Not with Xinerama, but 2 separate ones, first using first keyboard and first mouse, other using second keyboard and second mouse. i did this long time ago under NetBSD but X changed so much (as well as GFX hardware) that this patches are now irrevelant. From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 5 07:13:30 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3FB669BE for ; Sun, 5 Apr 2015 07:13:30 +0000 (UTC) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0663E63D for ; Sun, 5 Apr 2015 07:13:30 +0000 (UTC) Received: by pacyx8 with SMTP id yx8so10560566pac.1 for ; Sun, 05 Apr 2015 00:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=nTtXEyRp9lnamX5cws05J0UcSmBcCy/sWj7gstWveZ0=; b=puGZPUJNUPv9BGoRdhZHbJL0sHxmIajuFuefGpXyo1+YDHOxZbnTjnMngT6+jxCaiP lejC3NSFuTGcA1Qi0uho1TNux9BqxTASOBwNj/8WfuAXDuSI5H239Nst/ExBynCf/+xb Ipo4iow1xA8qHU3ngKjQgneSH05g5iH2MIGSIDnj4arwHt8pp6viqi9VrM+WwfEbzAFv o7NsaJiGZc/hDSnDpXMK7v2PwkYTjPnnQRB7SY54gORSQaNVhq6IYxmJwrH44I2BXzSp n4qgtjO1/ijmHQGqKRe4nOm4aeuyOZD80ygIWfuKDjVnZEaR1Y8OncS20IY/dlRXsFOa KTaA== X-Received: by 10.68.195.2 with SMTP id ia2mr362750pbc.33.1428218009332; Sun, 05 Apr 2015 00:13:29 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:981:fa61:f8b7:c531? ([2601:8:ab80:7d6:981:fa61:f8b7:c531]) by mx.google.com with ESMTPSA id pa6sm800269pac.45.2015.04.05.00.13.28 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sun, 05 Apr 2015 00:13:28 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_D27A2D6C-ED61-4E71-B8FC-D55F6AC05B9E"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: multiple X servers From: Garrett Cooper In-Reply-To: Date: Sun, 5 Apr 2015 00:13:28 -0700 Message-Id: References: To: Wojciech Puchar X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 07:13:30 -0000 --Apple-Mail=_D27A2D6C-ED61-4E71-B8FC-D55F6AC05B9E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 4, 2015, at 23:21, Wojciech Puchar wrote: > can two (or more) X servers be run on one machine with machine with = such config: >=20 > - 2 GFX cards, say one intel embedded in CPU, other addon. > - 2 keyboards > - 2 mice >=20 > Not with Xinerama, but 2 separate ones, first using first keyboard and = first mouse, other using second keyboard and second mouse. >=20 > i did this long time ago under NetBSD but X changed so much (as well = as GFX hardware) that this patches are now irrelevant. As long as you have 2 separate xorg.confs and use two separate displays, = I don=92t see why not... --Apple-Mail=_D27A2D6C-ED61-4E71-B8FC-D55F6AC05B9E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVIOCYAAoJEMZr5QU6S73e/bsH/2syAcvY78IKQFoGVwBWsSMm KwWFBqscl8JhH2wcPcL9LA49FrDWYzgX9wEEKmx3L8qfYvo8tSnWRl+S03MiSaqK CbaxL17HBzoR8ApmKNDTTpy7EtWvMmNKv2vv2wqHPcBqxdIAR5nsgE7BGKINJfPw laFFkKfYYFdM+0FmsYU40nmFiaZr/Pgue/fVOxwEARF45njl/eUvp1Y0y38Hb4TS sagDYKI++/dOIVvJDTUIeVQOb28B8K746NGMOOBpJlUZdkEP16K00IOhhzgOU21/ SXApQ9MCcP4eu7i/vdVrz/Pm69ySrjoMpGGYGnPUlg85Qsjnv3zsCoTQiYvNLeg= =MfDW -----END PGP SIGNATURE----- --Apple-Mail=_D27A2D6C-ED61-4E71-B8FC-D55F6AC05B9E-- From owner-freebsd-hackers@FreeBSD.ORG Sun Apr 5 12:24:59 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8DF6F96B for ; Sun, 5 Apr 2015 12:24:59 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 453DF6D1 for ; Sun, 5 Apr 2015 12:24:59 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-255-201.lns20.per4.internode.on.net [121.45.255.201]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t35COe86043502 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 5 Apr 2015 05:24:45 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <55212983.20201@freebsd.org> Date: Sun, 05 Apr 2015 20:24:35 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Yue Chen , Mateusz Guzik , Oliver Pinter , Benjamin Kaduk , "freebsd-hackers@freebsd.org" , HardenedBSD Core , PaX Team Subject: Re: How to traverse kernel threads? References: <20150321232622.GF14650@dft-labs.eu> <20150327194920.GB18158@dft-labs.eu> <20150330173358.GB9095@dft-labs.eu> <20150402214720.GE549@dft-labs.eu> In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 12:24:59 -0000 On 4/5/15 6:09 AM, Yue Chen wrote: > Thank you so much for the suggestions and kind help. Please do not take negative comments too seriously. Remember that an experiment with a negative result is as important as an experiment with a positive result. And if you do in fact succeed, in the face of much negative comment, it would be an important step. Even if you do not succeed, a paper carefully outlining all the issued you faced and your designed answers will make very interesting reading. > > On Thu, Apr 2, 2015 at 5:47 PM, Mateusz Guzik wrote: > >> On Mon, Mar 30, 2015 at 10:59:16PM -0400, Yue Chen wrote: >>>> This would be a performance hit. >>> The cost is to traverse the .text area and some other data in memory. >> It's >>> linear and only happens after a given time interval, like several >> minutes, >>> and does not put overhead on the normal execution. So the performance >>> overhead won't be high. >>> >> This was a remark about a runtime cost of executing kernel code with >> with an extra jump in every function. >> >>>> The only somewhat workable solution I see would require leaving >> function >>> entry points at constant addresses and only insert jumps to new >> locations. >>> Yes, we do this. For the PIC direct jump/call/"instructions including >>> %RIP", we just need to update them. >>> >> I have trouble parsing what you wrote here. >> >> What I wrote here suggests taking existing foo_func, placing a copy at a >> new location and putting a jump to that address as the very first >> instruction of foo_func. (and zeroing/whatever the rest of old >> foo_func). Quite terrible. >> >> Also quite irrelevant to how you arrive at given func. >> >> Also see below. >> >>>> Also runtime relocation of everything definitely has a lot of unknown >> unknowns, >>> so all in all I would say this is a non-starter. >>> >>> FreeBSD kernel itself only has 1500+ indirect jumps. There are two types: >>> 1. jump table. Like switch-case. 2. Tail call. We have not found other >>> types like manual assembly code, or context restoration like that in >>> longjmp in libc. Maybe there are some other but we have not found. >>> >> Unknown unknowns have the problem of not being known. You have to be >> quite skilled in the area to come up with a working solution and I >> highly doubt either of us is such a person. >> >>>> Leave a lot of data at constant addresses, defeating the point to some >>> extent. >>>> One could consider a different approach where kernel data is randomly >> shuffled >>> around with some granularity and >>>> relevant symbol relocated prior to booting. >>> Why do you think the data address exposure could be a serious problem? It >>> is non-executable and the code is randomized. Although for data itself, >> it >>> is really hard to protect from tampering and may cause other problems. >>> >> I see I wrote 'data', where I should have written '.text' or 'code', >> although from the context it should have been clear this refers to >> function entry points left behind. >> >> if you can get away with one function call, you don't need any infoleaks >> - you know the address of the function. That's one of the problems with >> leaving func entry points at known addresses. >> >>> Why the randomized data leads to kernel panic? I think if the data goes >>> somewhere else, the instruction addressing would follow, or page fault >> may >>> happen. If randomizing code, the exploit attempt may hit an illegal >>> instruction and the kernel panics. >>> >> What? >> >> Anyway I still don't understand how are you trying to achieve >> randomisation in the kernel. >> >> Previous mail suggests you just want to move funcs around and update all >> callsites and structs which store addresses to them. >> >> As outlined with 'struct meh' example (which can be found below) I do >> not believe this can work on real-world systems without a major surgery >> simply because there is no way to tell what looking like a pointer >> should be changed and what should not. I'm happy to be proved wrong. >> >> Kernel already leaks tons of addresses to userspace in documented ways, >> and I'm sure there are tons of situations where it does so as a result >> of a bug. >> >> I believe an alternative implementation I proposed which randomizes >> *all* kernel pages prior to booting it would be doable, but given >> aforementioned leaks not worth it. >> >> (well, striclty speaking you would want to move functions around with >> respect to each other, but keep the kernel physically contiguous so >> that you can still use superpages) >> >> Your previous mail in this thread suggests you are just starting any >> kind of work on kernel-level, which makes this very hard task even >> harder. >> >> Exploitation mitigation is a complicated subject on its own, I recommend >> reading what people were doing it have to say. Here is a piece about >> KASLR: https://forums.grsecurity.net/viewtopic.php?f=7&t=3367 >> >> In conclusion I strongly recommend dropping this project for the time >> being and focusing on something easier. >> >>>> return-oriented >>>>>> programming (ROP). It is basically a stronger form of ASLR. >>>>>> After each randomization procedure, the function return addresses >>>> saved in >>>>>> the stack are the ``old'' addresses before randomization, so we >> need to >>>>>> update them to the new addresses. >>>>>> That's why we need to get all the stack ranges to find those >> addresses. >>>>>> Also, in kernel, we believe that not only the return addresses in >>>> stacks >>>>>> need to be updated, there may exist other ``old'' saved instruction >>>> (PC) >>>>>> addresses in memory. Like in exception handling (maybe, do not >> know), >>>>>> debugging-purpose code and restartable atomic sequences (RAS) >>>>>> implementation. That's why I asked how to traverse all the kernel >>>> pages and >>>>>> get their virtual addresses here: >>>>>> >> https://lists.freebsd.org/pipermail/freebsd-hackers/2015-March/047336.html >>>>>> Now we found that it seems needed to traverse the ``pv_entry'' >>>> structure for >>>>>> x86_64 MMU. >>>>>> >>>>>> Another problem is that we do not know if FreeBSD has any form of >>>> special >>>>>> encodings or offset form for 64-bit instruction addresses (e.g., >> saved >>>> %RIP) >>>>>> on X86_64, instead of hard-coded addresses. For example, using a >> 32-bit >>>>>> offset instead of the 64-bit full address; and doing what glibc >> does >>>> for the >>>>>> setjmp/longjmp jmp_buf (special encodings (PTR_MANGLE) for the >> saved >>>>>> register values). >>>>>> >>>>>> Any suggestion or correction are highly appreciated. >>>>>> >>>>>> Best, >>>>>> Yue >>>>> (Added HardenedBSD core and PaXTeam to CC.) >>>>> >>>>> Until you can not fixed all of the infoleaks from kernel (try sysctl >>>>> -a | grep 0x or similar command) the KASLR and other kernel address >>>>> space randomization techniques are easily bypass-able... >>>>> >>>> I do not believe this can be implemented reliably without some serious >>>> tinkering around the kernel. Surely I'm not a live patching expert (not >>>> any other kind of expert), but hear my out. >>>> >>>> It seems proposed approach is to move the kernel around and then >> updated >>>> all relevant pointers in all pages. >>>> >>>> I do not believe this can be done reliably without corrupting data, >>>> unless a major surgery is performed on the kernel. >>>> >>>> Consider: >>>> struct meh { >>>> func_t *m_func; >>>> size_t m_len; >>>> data *m_buf; >>>> }; >>>> >>>> Here we have 'm_func' which needs to be updated, but we don't know if >>>> that's the only pointer so we have to scan the entire struct. But m_buf >>>> can have data which looks like kernel pointers (i.e. matches some >> kernel >>>> funcs) and how will you know not to modify it? What if this is some >> sort >>>> of a hack and in fact you *should* modify it? >>>> >>>> CTF or some other solutions like that don't help if you just traverse >>>> all allocated buffers since you have no idea what the object you found >>>> really is. >>>> >>>> So, assuming this has to be done in runtime, the only somewhat workable >>>> solution I see would require leaving function entry points at contant >>>> addresses and only insert jumps to new locations. >>>> >>>> This would be a performance hit and would leave a lot of data at >>>> constant addresses, defeating the point to some extent. >>>> >>>> Also runtime relocation of everything definitely has a lot of unknown >>>> unknowns, so all in all I would say this is a non-starter. >>>> >>>> One could consider a different approach where kernel data is randomly >>>> shuffled around with some granularity and relevant symbol relocated >>>> prior to booting. >>>> >>>> This should provide unique enough layout, which paired with big >>>> likelyhood of a kernel panic on first bad exploit attempt may be an >>>> acceptable solution. >>>> >>>> But then the kernel may still have enough info leaks for this to not >>>> matter, so I would not be so eager to implement it. >>>> >>>> That said, what prompted this entire effort? Is there an operating >>>> system which got this to work or what? >>>> >>>>>> >>>>>> >>>>>> On Fri, Mar 27, 2015 at 3:49 PM, Mateusz Guzik >>>> wrote: >>>>>>> On Fri, Mar 27, 2015 at 02:35:55PM -0400, Yue Chen wrote: >>>>>>>> When using the following code on kernel module loading: >>>>>>>> >>>>>>>> >> ------------------------------------------------------------------------------------------ >>>>>>>> struct thread *td = kdb_thr_first(); >>>>>>>> td = kdb_thr_next(td); >>>>>>>> >>>>>>>> >> ------------------------------------------------------------------------------------------ >>>>>>>> The kernel panics. >>>>>>>> >>>>>>> Panics how? >>>>>>> >>>>>>> Also you can easily see these functions don't lock anything, so it >>>> would >>>>>>> be assumed you took appropriate locks. >>>>>>> >>>>>>> Except it seems there routines are supposed to be only used when >>>>>>> execution is 'frozen' (e.g. when escaped to the debugger). >>>>>>> >>>>>>>> And when printing all threads in proc0 (all kernel threads?): >>>>>>>> >>>>>>>> >> ------------------------------------------------------------------------------------------ >>>>>>>> struct proc *p = pfind(0); >>>>>>>> FOREACH_THREAD_IN_PROC(p, td) { >>>>>>>> uprintf("td: %x\n", td); >>>>>>>> } >>>>>>>> >>>>>>> proc0 is an exported symbol, no need to pfind. >>>>>>> >>>>>>>> td = curthread; >>>>>>>> uprintf("cur td: %x\n", td); >>>>>>>> >>>>>>>> >> ------------------------------------------------------------------------------------------ >>>>>>>> The ``curthread'' (from this kernel module running the above >> code) >>>> is >>>>>>>> not >>>>>>>> in the 0 process group. >>>>>>>> >>>>>>> There is no 'curthread from kernel module'. >>>>>>> >>>>>>> My guess is you do this work from module initializator, and in >> that >>>> case >>>>>>> curthread is the thread which loads the module, and such a thread >> is >>>>>>> definitely not linked into proc0. >>>>>>> >>>>>>> Still nobody knows what you are trying to do. >>>>>>> >>>>>>> -- >>>>>>> Mateusz Guzik >>>>>> >>>> -- >>>> Mateusz Guzik >>>> >> -- >> Mateusz Guzik >> > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 6 18:03:23 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A11CDA9F for ; Mon, 6 Apr 2015 18:03:23 +0000 (UTC) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8CE6BCB5 for ; Mon, 6 Apr 2015 18:03:23 +0000 (UTC) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPSA id 4C69710CADC; Mon, 6 Apr 2015 11:03:22 -0700 (PDT) Date: Mon, 6 Apr 2015 11:03:22 -0700 From: hiren panchasara To: freebsd-hackers@freebsd.org Subject: Puzzled about VFS sysctl OIDs -- signed vs. unsigned (again) Message-ID: <20150406180322.GD96049@strugglingcoder.info> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="1giRMj6yz/+FOIRq" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Cc: mdf356@gmail.com X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 18:03:23 -0000 --1giRMj6yz/+FOIRq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Ran into this issue on a stable/10 box at work and found https://lists.freebsd.org/pipermail/freebsd-hackers/2011-March/034600.html I'd be interested in knowing how other are fixing this in-house (at $work). One suggestion I had was to raise it from int to long but that apparently may have performance impact as these routines get called a lot and needs to be profiled. Making them (at least) unsigned is a good first step. Cheers, Hiren --1giRMj6yz/+FOIRq Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQF8BAEBCgBmBQJVIsppXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lgTgH/2nMFSvuoXJdxgpxptVtwz8S 9E5/vSobDKRKO61bSSTfF5VbCwou+6isxPNVfF3Qv+jA6ox/Gnx6Vs+x8gub2Hdk HsCbM7rGIzZgkTISPCZlhndNNzqE94C2R0zP+Uczu4RN6OOxsnOv3NL0uGCpfH1G PqYOdwBwQ1sC9K9W2Gwhm5BAO9TFWw4rne1GFFsbH6YLXt3ILZ3HJN2kVWRN9U92 CiaDCOu6V+YSIgBoWOU1JEjt/Ufmg/Tn59q4bCT42XioRV9XD8zsZMOXFYTLT4cP m3KdNV3HRlslHBx1Web0mNjX3RZOzF6AiPYg1KfApV7psSlV1g5e2NkIU5JR1tg= =HGtc -----END PGP SIGNATURE----- --1giRMj6yz/+FOIRq-- From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 6 21:18:14 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 75BADAD2 for ; Mon, 6 Apr 2015 21:18:14 +0000 (UTC) Received: from mail-pd0-x22c.google.com (mail-pd0-x22c.google.com [IPv6:2607:f8b0:400e:c02::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 416C9800 for ; Mon, 6 Apr 2015 21:18:14 +0000 (UTC) Received: by pdea3 with SMTP id a3so56259327pde.3 for ; Mon, 06 Apr 2015 14:18:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version; bh=YCaIU/MfG92dliwz/gsyhjRmZqef/voRxukXQkW2HNw=; b=nUkHM1Jzyh/afyQuN8n/U3nyrN4mobob0SIg6P6tPhCInmMGMGto/uOIKJWSSsdrUo lTym3jaqYltMYTDpUE3uQ2w0M4BIvja3UqasfGWzO/pm9o2I977p8r3wyhF5a4Dibh/x f/JgIpBwADDvAiIkkiWFuFHh1vUd+BOySHl6IPyXWsAqhYnhKFJ0Agf2c3/9tOik4pVV JCz4XTzRrSVJQVU1NBYxYkxDLDMvVkS+NfChyIq2tVfPXVesG9N3/JxduFujLPjFQGJ5 kZ5/+lNtJsukGGBv4neh/NAjU0WhzhTLOEQvpsRBLkMtlaYWC3GmzWdBKYfS9scS8UUW zcLQ== X-Received: by 10.70.54.198 with SMTP id l6mr30799876pdp.74.1428355093731; Mon, 06 Apr 2015 14:18:13 -0700 (PDT) Received: from [192.168.1.188] (173-26-46-90.client.mchsi.com. [173.26.46.90]) by mx.google.com with ESMTPSA id ia3sm2328954pbc.31.2015.04.06.14.18.11 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 06 Apr 2015 14:18:12 -0700 (PDT) From: Guy Helmer Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: lockf() vs. flock() -- lockf() not locking? Message-Id: <3950D855-0F4E-49E0-A388-4C7ED102B68B@gmail.com> Date: Mon, 6 Apr 2015 16:18:09 -0500 To: FreeBSD Hackers Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) X-Mailer: Apple Mail (2.2070.6) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 21:18:14 -0000 Recently an application I use switched from using flock() for advisory = file locking to lockf() in the code that protects against concurrent = writes to a file that is being shared and updated by multiple processes = (not threads in a single process). The code seems reliable =E2=80=94 a = lock manager class opens the file & obtains the lock, then the = read/update method opens the file using a separate file descriptor & = reads/writes the file, flushes & closes the second file descriptor, and = then destroys the lock manager object which unlocks the file & closes = the first file descriptor. Surprisingly this simple change seems to have made the code unreliable = by allowing concurrent writers to the file and corrupting its contents: - if (flock(fd, LOCK_EX) !=3D 0) + if (lockf(fd, F_LOCK, 0) !=3D 0) throw std::runtime_error("Failed to get a lock of " + = filename); . . . if (fd !=3D -1) { - flock(fd, LOCK_EX); + lockf(fd, F_ULOCK, 0); close(fd); fd =3D -1; } =46rom my reading of the lockf(3) man page and reviewing the = implementation in lib/libc/gen/lockf.c, and corresponding code in = sys/kern/kern_descrip.c, it appears the lockf() call should be = successfully obtaining an advisory lock over the whole file like a = successful flock() did. However, I have a stress test that quickly = corrupts the target file using the lockf() implementation, and the test = fails to cause corruption using the flock() implementation. I=E2=80=99ve = instrumented the code, and it's clear that multiple processes are = simultaneously in the block of code after the =E2=80=9Clockf(fd, F_LOCK, = 0)=E2=80=9D line. Am I missing something obvious? Any ideas? Thanks, Guy From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 6 21:51:55 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEA7B114 for ; Mon, 6 Apr 2015 21:51:55 +0000 (UTC) Received: from mail-vn0-x233.google.com (mail-vn0-x233.google.com [IPv6:2607:f8b0:400c:c0f::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 84638BF3 for ; Mon, 6 Apr 2015 21:51:55 +0000 (UTC) Received: by vnbf129 with SMTP id f129so4319192vnb.9 for ; Mon, 06 Apr 2015 14:51:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=vjboXBSBqU0CspE5PSQhEhivfy8VjHEBPrKkj6lB7J4=; b=VOaJTV4gHbJrqZU4Xn4jeT4feXvG9sT3h0ZFglB+uyoVPksN4kXStq5cJw7eIQ7iqk KFnkiaqXpraPZXgSXjXA/wdT4dlsukwaa0wqnB8+npAvPXYu4ypCRd22tmOxbkbSVxOs CSZ7RVMUt1IDCPOzNGwIsczUQbsKZXV2NinG9fqRk/THZhVMqZ8m4+pPPriulTwEqSMM v6rcKttNYzL8ZqNtUUZeg0b0iHpdCPUJZ0Fen0U8mlwYq5tOkTB0LTApuaMCHKznn+ZE CwjXmImmeVzac/JK9C0hbgUBT9+xet/drcDEIfoY9TKFGM4JW1JSujzRhtFmwK7WV7ER ZDMg== MIME-Version: 1.0 X-Received: by 10.53.7.72 with SMTP id da8mr11119391vdd.58.1428357114627; Mon, 06 Apr 2015 14:51:54 -0700 (PDT) Received: by 10.52.164.104 with HTTP; Mon, 6 Apr 2015 14:51:54 -0700 (PDT) In-Reply-To: <3950D855-0F4E-49E0-A388-4C7ED102B68B@gmail.com> References: <3950D855-0F4E-49E0-A388-4C7ED102B68B@gmail.com> Date: Mon, 6 Apr 2015 17:51:54 -0400 Message-ID: Subject: Re: lockf() vs. flock() -- lockf() not locking? From: Conrad Meyer To: Guy Helmer Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 21:51:55 -0000 On Mon, Apr 6, 2015 at 5:18 PM, Guy Helmer wrote: > From my reading of the lockf(3) man page and reviewing the implementation= in lib/libc/gen/lockf.c, and corresponding code in sys/kern/kern_descrip.c= , it appears the lockf() call should be successfully obtaining an advisory = lock over the whole file like a successful flock() did. However, I have a s= tress test that quickly corrupts the target file using the lockf() implemen= tation, and the test fails to cause corruption using the flock() implementa= tion. I=E2=80=99ve instrumented the code, and it's clear that multiple proc= esses are simultaneously in the block of code after the =E2=80=9Clockf(fd, = F_LOCK, 0)=E2=80=9D line. > > Am I missing something obvious? Any ideas? We have switched from a whole file lock to a range lock. I think it should still make access exclusive, so this sounds like a bug. But just note that it is a slightly different mutual exclusion mechanism. Are the locked files on NFS? Best, Conrad From owner-freebsd-hackers@FreeBSD.ORG Mon Apr 6 23:19:36 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E6E5FA17 for ; Mon, 6 Apr 2015 23:19:36 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B015467B for ; Mon, 6 Apr 2015 23:19:36 +0000 (UTC) Received: by pdbqa5 with SMTP id qa5so1828330pdb.1 for ; Mon, 06 Apr 2015 16:19:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=v5cgoaHNW+VNLTDCXJ4qTcYoVZaFhR1OP5H0eBSgx/U=; b=dJx9WCmTjGWPQ7mrAjm+qQD7vyaxb8lnsjvKjK4AfhSMK5/r15m8keI9V4CM6Z+2Ao 49rNCJit7tmTXlfMtnOVWghcuNGcTc/SstIDGfus7wUhjs69HByojHJwd1LRP8uGax/A 8W46cGau41qFb6cnzuM0PLC7ESlF12bs2AusJUT9dbloxpcQVi8A/poOR9k0f3taq5V1 ieO62Qphj7rkncxTmodWbkHqLyO079A7InthVAVKDwdR1ZmaBoLWvP2LAaWp0cPB7bmc fF0PmaJa8+/GKe6y19yHupWiC512+o9JnCXMrZzL123op1UnucIZy62qgpygUa6CoF2E bhVw== X-Received: by 10.66.147.103 with SMTP id tj7mr31403401pab.85.1428362376247; Mon, 06 Apr 2015 16:19:36 -0700 (PDT) Received: from [192.168.1.188] (173-26-46-90.client.mchsi.com. [173.26.46.90]) by mx.google.com with ESMTPSA id mt11sm5800680pbc.20.2015.04.06.16.19.34 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 06 Apr 2015 16:19:34 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: lockf() vs. flock() -- lockf() not locking? From: Guy Helmer In-Reply-To: Date: Mon, 6 Apr 2015 18:19:32 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <38FF4CA2-5116-4168-AC6D-DD6A60AEAB3D@gmail.com> References: <3950D855-0F4E-49E0-A388-4C7ED102B68B@gmail.com> To: Conrad Meyer X-Mailer: Apple Mail (2.2070.6) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 23:19:37 -0000 > On Apr 6, 2015, at 4:51 PM, Conrad Meyer wrote: >=20 > On Mon, Apr 6, 2015 at 5:18 PM, Guy Helmer = wrote: >> =46rom my reading of the lockf(3) man page and reviewing the = implementation in lib/libc/gen/lockf.c, and corresponding code in = sys/kern/kern_descrip.c, it appears the lockf() call should be = successfully obtaining an advisory lock over the whole file like a = successful flock() did. However, I have a stress test that quickly = corrupts the target file using the lockf() implementation, and the test = fails to cause corruption using the flock() implementation. I=E2=80=99ve = instrumented the code, and it's clear that multiple processes are = simultaneously in the block of code after the =E2=80=9Clockf(fd, F_LOCK, = 0)=E2=80=9D line. >>=20 >> Am I missing something obvious? Any ideas? >=20 >=20 > We have switched from a whole file lock to a range lock. I think it > should still make access exclusive, so this sounds like a bug. But > just note that it is a slightly different mutual exclusion mechanism. >=20 > Are the locked files on NFS? No, this involves files on local filesystems. Regards, Guy From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 7 01:39:46 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1721751; Tue, 7 Apr 2015 01:39:46 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1on0092.outbound.protection.outlook.com [157.56.110.92]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3EC39617; Tue, 7 Apr 2015 01:39:45 +0000 (UTC) Received: from DM2PR0801MB0944.namprd08.prod.outlook.com (25.160.131.27) by DM2PR0801MB0941.namprd08.prod.outlook.com (25.160.131.24) with Microsoft SMTP Server (TLS) id 15.1.125.19; Tue, 7 Apr 2015 01:39:36 +0000 Received: from DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) by DM2PR0801MB0944.namprd08.prod.outlook.com ([25.160.131.27]) with mapi id 15.01.0125.002; Tue, 7 Apr 2015 01:39:36 +0000 From: "Pokala, Ravi" To: "freebsd-hackers@freebsd.org" , "freebsd-hardware@freebsd.org" Subject: Re: freebsd-hackers Digest, Vol 624, Issue 6 Thread-Topic: freebsd-hackers Digest, Vol 624, Issue 6 Thread-Index: AQHQbs7nb+BJ+JZOnU2SEAlzbl2XMJ1AVHyA Date: Tue, 7 Apr 2015 01:39:35 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.8.150116 x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [64.80.217.3] authentication-results: freebsd.org; dkim=none (message not signed) header.d=none; x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DM2PR0801MB0941; x-forefront-antispam-report: BMV:1; SFV:NSPM; SFS:(10009020)(6009001)(164054003)(10533003)(18543002)(122556002)(46102003)(66066001)(40100003)(450100001)(19580405001)(36756003)(92566002)(77156002)(87936001)(2950100001)(86362001)(2656002)(54356999)(76176999)(106116001)(83506001)(2900100001)(99286002)(2501003)(50986999)(512954002)(102836002)(19580395003)(62966003)(107886001); DIR:OUT; SFP:1101; SCL:1; SRVR:DM2PR0801MB0941; H:DM2PR0801MB0944.namprd08.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(5002010)(5005006); SRVR:DM2PR0801MB0941; BCL:0; PCL:0; RULEID:; SRVR:DM2PR0801MB0941; x-forefront-prvs: 0539EEBD11 Content-Type: text/plain; charset="us-ascii" Content-ID: <27496F200A9CF9458AA43B2C6FD73EE3@namprd08.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: panasas.com X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Apr 2015 01:39:35.4426 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: acf01c9d-c699-42af-bdbb-44bf582e60b0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM2PR0801MB0941 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 01:39:47 -0000 >Date: Sat, 4 Apr 2015 02:28:46 +0000 >From: "Pokala, Ravi" >To: "freebsd-hardware@freebsd.org" , > "freebsd-hackers@freebsd.org" >Subject: Booting off NVMe using traditional bootstrap? >Message-ID: >Content-Type: text/plain; charset=3D"us-ascii" > >Hi folks, > >Does anyone know off-hand if it's possible to boot (amd64) off of an NVMe >device using the traditional bootstrap code (i.e. *not* UEFI)? > >Thanks, > >Ravi Naturally, someone pointed out the obvious idea - put /boot on a USB stick and boot off that. As silly as it sounds, we've had trouble doing that in the past - USB hiccups caused '/' to disappear, and hilarity ensued. Nothing like an expensive server going offline because a $10 component failed. :-P Though I suppose in the past we were putting all of '/' on the USB, not just '/boot'... So, how little can we get away with putting on the USB? For it to have '/boot', doesn't that mean it also has have '/'? Or else, how would it recognize that the 'boot' directory on the USB was actually '/boot'? Doing this (minimal bootstrap on one device, everything else on other devices) seems like it should be documented, but a quick trip to Google and to freebsd.org don't reveal anything obvious. Any pointers? Thanks, Ravi From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 7 12:39:42 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0DABBB7B; Tue, 7 Apr 2015 12:39:42 +0000 (UTC) Received: from mail-wi0-x232.google.com (mail-wi0-x232.google.com [IPv6:2a00:1450:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8ACD2A5C; Tue, 7 Apr 2015 12:39:41 +0000 (UTC) Received: by wiun10 with SMTP id n10so16901990wiu.1; Tue, 07 Apr 2015 05:39:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=/pqHC5InaA4z972wjSMbvHB/vSBtG23bAJI54qCO3U8=; b=JdkspgvvlMxxDv1vUBH36izWMX2UQE52x6DRpxJEtcs+Acsr3P7kzdqce59AlzrDV3 csTYgLGAgCrZi72Ao6IlUd9PQAGlVbbgZlBgbX3DTjKuVdzVxOAwGj2cmMuo6rnXs2yj QxOqntT3/h3nS4JVS+rk2j+AthKTPJYGYXtrSkhTgyZmmb6UXIo8CurhSaAqSD3J6X0/ Fd78Iwq64a5cK4+dl3dKxBZlPhdOnOdDc63NGeZwXGZprLEij+mfNZOKipjRRuALkZ8E +iyCe6GMjPBmEFnooldDYq9CdiiLNHdCeO4gCvdWJcgjituJolwD4cfz4B/5JKw8Gmcw HgcQ== X-Received: by 10.194.158.234 with SMTP id wx10mr40301245wjb.23.1428410380143; Tue, 07 Apr 2015 05:39:40 -0700 (PDT) Received: from [192.168.1.130] (ppp-88-217-5-177.dynamic.mnet-online.de. [88.217.5.177]) by mx.google.com with ESMTPSA id it5sm10877949wid.3.2015.04.07.05.39.38 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Apr 2015 05:39:38 -0700 (PDT) Message-ID: <5523D009.7070302@gmail.com> Date: Tue, 07 Apr 2015 14:39:37 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: NVMe performance 4x slower than expected References: <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <551C6B62.7080205@gmail.com> <551D4E5F.9090400@gmail.com> <20150402221029.GE2379@kib.kiev.ua> <551FC7B3.80203@gmail.com> <20150404234839.GM2379@kib.kiev.ua> In-Reply-To: <20150404234839.GM2379@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 12:39:42 -0000 >>>> Something is severely wrong. It seems, there might be multiple issues >>>> (not only NVMe). And this is after already running with 3 patches to >>>> make it even boot. >>> The speed of dd already looks wrong. >> >> Yes. > Try some simple checks for the performance anomalies. E.g., verify the > raw memory bandwidth, check that it is in the expected range. Quick search > popped up the following tool, e.g.: > https://zsmith.co/bandwidth.html Here are results for those 2 machines (single socket 4 core E3, quad socket 48 core E7): https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_memperf.md https://raw.githubusercontent.com/oberstet/scratchbox/master/freebsd/cruncher/results/bw_4core.bmp https://raw.githubusercontent.com/oberstet/scratchbox/master/freebsd/cruncher/results/bw_48core.bmp From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 7 13:19:04 2015 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D782F8F4 for ; Tue, 7 Apr 2015 13:19:04 +0000 (UTC) Received: from mail-wi0-x236.google.com (mail-wi0-x236.google.com [IPv6:2a00:1450:400c:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 736D8FB1 for ; Tue, 7 Apr 2015 13:19:04 +0000 (UTC) Received: by wiaa2 with SMTP id a2so18371863wia.0 for ; Tue, 07 Apr 2015 06:19:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:mime-version:content-type :content-transfer-encoding; bh=LChEKjBQ/Y3jK2NLbAiBDikoCusPmMpMDaqEi+5Vj40=; b=ixA+T6AT4a3t2gWGekuO4cDyIId2IR0QrAabOJmYsvh0Ck0Oq3ROXv8sHSsKEXeNpv U4Kmi4bwhXwgwQHvq1wG84xP8Hpa7z1YvYJ/cxltuYssE89WzT1yWmbtfl/hpWUooyYf Ah/AL0HgZo6UX4seHEagAlPIhz0C5WD9yg+uYs9TObIg/jv57Fsjkl406fRga+FhZGKI VpQiIkK1V0MACe94YDNOTdU60BMEAw3OqiRBBJvOrzbCF1CFScJgd/aDopQSAoRlIL1e DVlFTO6yqyxb8IpQpjOcVFJIT0JFsTbC46NgIQyFWd7p2N6/jA+84d/qGGwk1sj8k3xX HO8g== X-Received: by 10.194.239.65 with SMTP id vq1mr38569019wjc.98.1428412742934; Tue, 07 Apr 2015 06:19:02 -0700 (PDT) Received: from localhost ([217.14.212.217]) by mx.google.com with ESMTPSA id i13sm10559477wic.13.2015.04.07.06.19.02 for (version=SSLv3 cipher=RC4-SHA bits=128/128); Tue, 07 Apr 2015 06:19:02 -0700 (PDT) Date: Tue, 7 Apr 2015 15:18:58 +0200 From: To: hackers@freebsd.org Subject: (GSoC 2015) Parallel ports build and installation => paraports Message-ID: <20150407151858.00004a21@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 13:19:04 -0000 Hello, You already know me so no need for introduction. ;) It is application for configuring, fetching, building and installing software (ports) on FreeBSD It provides a safe way for building AND installation of several ports simultaneously by taking care of order of dependencies. Each build process is independent of others with it's own dir where output (separated in 2 files, one for STDOUT and second for STDERR) is stored in case if build error occurs. Number of simultaneous builds depends on chosen amount of jobs, which defaults to 2 jobs per core. 1 core CPU performed fastest with 2 jobs = 2 simultaneous builds Allows multicore machines to use all its potential It also supports building AND installation in DESTDIR, which is demonstrated in video. With -e flag rnable all '-o' installed origins in rc.conf Best part => whole implementation is solely based on FreeBSD's base userland tools /bin/sh (+awk/sed/etc ...) To cut it short, here is unfinished working code: (because I could type and type lengthy description and am short on time) http://www.starforce.biz/paraports.mp4 I'll respond to questions (I bet it'll start with "why so much in configs"). ;) Mentor: ------- I would prefer to get mentor during this week. I understand theirs time is tight for mentorship, so it is a plus that I already have a working code demonstrated above and also understand whole process, so mentor's time consumption will be minimal. Domagoj From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 7 22:15:29 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 61B396BF for ; Tue, 7 Apr 2015 22:15:29 +0000 (UTC) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 24B22331 for ; Tue, 7 Apr 2015 22:15:29 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id B37F9358C57; Wed, 8 Apr 2015 00:15:25 +0200 (CEST) Received: by snail.stack.nl (Postfix, from userid 1677) id 79CD128494; Wed, 8 Apr 2015 00:15:25 +0200 (CEST) Date: Wed, 8 Apr 2015 00:15:25 +0200 From: Jilles Tjoelker To: Guy Helmer Subject: Re: lockf() vs. flock() -- lockf() not locking? Message-ID: <20150407221525.GA99106@stack.nl> References: <3950D855-0F4E-49E0-A388-4C7ED102B68B@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3950D855-0F4E-49E0-A388-4C7ED102B68B@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 22:15:29 -0000 On Mon, Apr 06, 2015 at 04:18:09PM -0500, Guy Helmer wrote: > Recently an application I use switched from using flock() for advisory > file locking to lockf() in the code that protects against concurrent > writes to a file that is being shared and updated by multiple > processes (not threads in a single process). The code seems reliable — > a lock manager class opens the file & obtains the lock, then the > read/update method opens the file using a separate file descriptor & > reads/writes the file, flushes & closes the second file descriptor, > and then destroys the lock manager object which unlocks the file & > closes the first file descriptor. > Surprisingly this simple change seems to have made the code unreliable > by allowing concurrent writers to the file and corrupting its > contents: > - if (flock(fd, LOCK_EX) != 0) > + if (lockf(fd, F_LOCK, 0) != 0) > throw std::runtime_error("Failed to get a lock of " + filename); > . . . > if (fd != -1) { > - flock(fd, LOCK_EX); > + lockf(fd, F_ULOCK, 0); > close(fd); > fd = -1; > } > From my reading of the lockf(3) man page and reviewing the > implementation in lib/libc/gen/lockf.c, and corresponding code in > sys/kern/kern_descrip.c, it appears the lockf() call should be > successfully obtaining an advisory lock over the whole file like a > successful flock() did. However, I have a stress test that quickly > corrupts the target file using the lockf() implementation, and the > test fails to cause corruption using the flock() implementation. I’ve > instrumented the code, and it's clear that multiple processes are > simultaneously in the block of code after the “lockf(fd, F_LOCK, 0)” > line. > Am I missing something obvious? Any ideas? With lockf/fcntl locks, the close of the second file descriptor actually already unlocks the file. If there is another close and open in there, it would explain your problem. Both the lockf(3) and the fcntl(2) man pages mention these strange semantics, but only fcntl(2) clearly warns about them. With flock locks, opening the file another time will not cause problems. The second thing that will not work with lockf/fcntl locks is having a child process inherit them. Changing flock() to lockf() seems like a bad idea, particularly in a reusable "lock manager" class, since it is then harder to see what operations must be avoided to avoid losing the lock. There is a proposal in the Austin Group for the next version of POSIX to add a form of file lock that allows both range locking and proper (flock-style) semantics. -- Jilles Tjoelker From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 8 02:15:13 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4745D4DF for ; Wed, 8 Apr 2015 02:15:13 +0000 (UTC) Received: from mail-ie0-x22d.google.com (mail-ie0-x22d.google.com [IPv6:2607:f8b0:4001:c03::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E7E02FE for ; Wed, 8 Apr 2015 02:15:13 +0000 (UTC) Received: by iebrs15 with SMTP id rs15so63122624ieb.3 for ; Tue, 07 Apr 2015 19:15:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=4oyg3RR/+qB1xVukCS3S5gqO1jYBDtC2vPjo/+Aypl4=; b=UT1fYNnBPtz1P7YrQ9QN0lJ60veFDvr9p7SfvZl74Gd76lrbewL42SCjFo73Tc8WFS s/bYg5/Al1nUvWn3DlGKXklY/40A/+u6S/2AnmmxYEjv8uzIBub6aeho/W3bTt5jHvay 1r50RLkaES5T8og6RHeCFpTIQ5vva3lUvifr4Hixu6RsgYCBGxCzzjsyC3dNEntmsbge OPCOJkOcOxZ/N5JL6IbQe2r9jEhh94pmu8r8dcp4JE4kKCuEFoLOG7v2PHqGEugvZqtx Ff51Bc748tdOo+PQXO0u6FYdAGBgnmjUKJdFAX9RMQjWDn2r6SEW30BiXsQKhe26SSC2 Q4pw== MIME-Version: 1.0 X-Received: by 10.50.43.197 with SMTP id y5mr8475285igl.14.1428459312395; Tue, 07 Apr 2015 19:15:12 -0700 (PDT) Received: by 10.64.78.105 with HTTP; Tue, 7 Apr 2015 19:15:12 -0700 (PDT) Date: Wed, 8 Apr 2015 07:45:12 +0530 Message-ID: Subject: Free BSD ARM Qualcomm kernel From: Rakesh Sharma To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 02:15:13 -0000 hey i applied for gsoc15, to port BSD to any android phone, i thought irrespective of me selected or not, i should do some research. I came across the page of BSD https://www.freebsd.org/platforms/arm.html Where is state hardware support for qualcomm snapdragon processor. But i am unable to get the source for qualcomm. I checked on git in the given link which is all for ARM. https://github.com/freebsd/freebsd/tree/master/sys/arm I will be glad to have help on the qualcomm source Thanks From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 8 02:22:11 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34E28A0B for ; Wed, 8 Apr 2015 02:22:11 +0000 (UTC) Received: from mail-ob0-x234.google.com (mail-ob0-x234.google.com [IPv6:2607:f8b0:4003:c01::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E824E3F7 for ; Wed, 8 Apr 2015 02:22:10 +0000 (UTC) Received: by obbeb7 with SMTP id eb7so44978105obb.3 for ; Tue, 07 Apr 2015 19:22:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=AgyoKCphT5yNVSaFuU2c3rEmRWcxfHABN+ZcX8f4kUE=; b=gku9jZgwbW/6NWibG7naKvoWK+V/fQR8ESwh1OODJxSb/dZZvV64zft1d2qJ4dbfR7 Gw37yjoM1aDAlm+Xa5DW1sBhJz2NUhwRXuLdzh6ktSeqos0n44fyntc9Fxev1AlKb4fU 9zJNBvV9USWb7E7yrbuJtp20vD5jrsYHktns9/uEuczvOTqP9JX06juy9cKMlQdPLmdb HafIEZjzQYvVDMcJl0MoANaPNUymWHK+TzfmdLTkkywCDxU1o5Qv377WKv3Oscjiju4R gLcoO7l5YdPUF1dVIJiD6I1JoiObEf6btN+IhUaFbHmptEp1mSq7FgPyUQ3PjYxvG1L+ o4yA== MIME-Version: 1.0 X-Received: by 10.60.124.83 with SMTP id mg19mr29016174oeb.42.1428459730331; Tue, 07 Apr 2015 19:22:10 -0700 (PDT) Received: by 10.182.204.9 with HTTP; Tue, 7 Apr 2015 19:22:10 -0700 (PDT) In-Reply-To: References: Date: Wed, 8 Apr 2015 10:22:10 +0800 Message-ID: Subject: Re: Free BSD ARM Qualcomm kernel From: Ganbold Tsagaankhuu To: Rakesh Sharma Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 02:22:11 -0000 On Wed, Apr 8, 2015 at 10:15 AM, Rakesh Sharma wrote: > hey i applied for gsoc15, to port BSD to any android phone, i thought > irrespective of me selected or not, i should do some research. > > I came across the page of BSD > https://www.freebsd.org/platforms/arm.html > > Where is state hardware support for qualcomm snapdragon processor. But i am > unable to get the source for qualcomm. I checked on git in the given link > which is all for ARM. > > https://github.com/freebsd/freebsd/tree/master/sys/arm > > I will be glad to have help on the qualcomm source > Snapdragon SoC support is not polished and ready yet to be included in FreeBSD source. Michal Meloun is working on it, you can take a look at his github: https://github.com/strejda/qualcomm/tree/master/sys/arm/qualcomm Ganbold > > Thanks > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 8 10:57:58 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9068BD27 for ; Wed, 8 Apr 2015 10:57:58 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 158271ED for ; Wed, 8 Apr 2015 10:57:57 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t38AvlwA078677 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 8 Apr 2015 12:57:48 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t38AvjsH001348; Wed, 8 Apr 2015 12:57:45 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t38AvdS4001345; Wed, 8 Apr 2015 12:57:39 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Wed, 8 Apr 2015 12:57:39 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Garrett Cooper Subject: Re: multiple X servers In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Wed, 08 Apr 2015 12:57:48 +0200 (CEST) Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 10:57:58 -0000 >> >> Not with Xinerama, but 2 separate ones, first using first keyboard and first mouse, other using second keyboard and second mouse. >> >> i did this long time ago under NetBSD but X changed so much (as well as GFX hardware) that this patches are now irrelevant. > > As long as you have 2 separate xorg.confs and use two separate displays, I don?t see why not... > because when Xorg starts it messed up with all display adapters. but maybe it changed. From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 8 12:16:06 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 72AB334E for ; Wed, 8 Apr 2015 12:16:06 +0000 (UTC) Received: from mail-qk0-f180.google.com (mail-qk0-f180.google.com [209.85.220.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C77CDA4 for ; Wed, 8 Apr 2015 12:16:05 +0000 (UTC) Received: by qku63 with SMTP id 63so82474051qku.3 for ; Wed, 08 Apr 2015 05:15:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=5F8UxfGVDs8XIGKfR5mhjc5UXhrDDICoGLdEfGCMirs=; b=TEEXbRNAHRlbAmU9X8TvMY0qQGcEkdHUp7hPzYBwqHuHq5suDnboxCR7cOkoR+o5ly vtlgZGCyBkF8aldycBpDpyhAPHmQnczFVb4j+Lrg5ynolihAq1p2nO2tAQyQqpZVB7Ie qbbsPAVYdQMsAIMwgac+nU+gk2lagST178OGWpK6jjuTV+W9KgdKbZ6r/orMKkfwefIn 5bmyPeC+izH8V6qhSAuosxUDpD/VNKZDwQmr+gPAQ+6z5881N1JV7kM2Ek2M5BZ0+RlD yrKGPWbUpPyg14rAigy4ztdHSTeGSIDveS/wkV5F2r3BuaWDZ2yw4ogdElTeYEBb/VkJ R2Ug== X-Gm-Message-State: ALoCoQn5NEbpHuaIW2bU1BnPC99+VAmJ0mDEVNkGd18+o0Il5pif+WPbBvn7rYqnQyh+gZQRB7AE X-Received: by 10.229.29.136 with SMTP id q8mr30520543qcc.2.1428494968663; Wed, 08 Apr 2015 05:09:28 -0700 (PDT) Received: from ?IPv6:2600:1001:b119:9f54:e146:2ec2:2c13:3db? ([2600:1001:b119:9f54:e146:2ec2:2c13:3db]) by mx.google.com with ESMTPSA id o90sm7355382qkh.25.2015.04.08.05.09.28 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 08 Apr 2015 05:09:28 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: multiple X servers From: Mark Saad X-Mailer: iPhone Mail (12D508) In-Reply-To: Date: Wed, 8 Apr 2015 08:09:26 -0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Wojciech Puchar Cc: "freebsd-hackers@freebsd.org" , Garrett Cooper X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 12:16:06 -0000 On Apr 8, 2015, at 6:57 AM, Wojciech Puchar wrote: >>>=20 >>> Not with Xinerama, but 2 separate ones, first using first keyboard and f= irst mouse, other using second keyboard and second mouse. >>>=20 >>> i did this long time ago under NetBSD but X changed so much (as well as G= FX hardware) that this patches are now irrelevant. >>=20 >> As long as you have 2 separate xorg.confs and use two separate displays, I= don?t see why not... > because when Xorg starts it messed up with all display adapters. > but maybe it changed. While linux oriented the arch wiki has some good starting material for this;= Also known as multiseat x . I tried this years ago on Debian , but never tr= ied it on FreeBSD .=20 https://wiki.archlinux.org/index.php/Xorg_multiseat --- Mark Saad | nonesuch@longcount.org > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"= From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 8 16:37:08 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5716569 for ; Wed, 8 Apr 2015 16:37:08 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 370A4348 for ; Wed, 8 Apr 2015 16:37:07 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t38Gb2Nw093981 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 8 Apr 2015 18:37:02 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t38Gb0aK000894; Wed, 8 Apr 2015 18:37:00 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t38Gasfp000891; Wed, 8 Apr 2015 18:36:54 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Wed, 8 Apr 2015 18:36:54 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: Mark Saad Subject: Re: multiple X servers In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Wed, 08 Apr 2015 18:37:03 +0200 (CEST) Cc: "freebsd-hackers@freebsd.org" , Garrett Cooper X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 16:37:08 -0000 seems like it should work. i will try it when actually get second card. On Wed, 8 Apr 2015, Mark Saad wrote: > > On Apr 8, 2015, at 6:57 AM, Wojciech Puchar wrote: > >>>> >>>> Not with Xinerama, but 2 separate ones, first using first keyboard and first mouse, other using second keyboard and second mouse. >>>> >>>> i did this long time ago under NetBSD but X changed so much (as well as GFX hardware) that this patches are now irrelevant. >>> >>> As long as you have 2 separate xorg.confs and use two separate displays, I don?t see why not... >> because when Xorg starts it messed up with all display adapters. >> but maybe it changed. > > While linux oriented the arch wiki has some good starting material for this; Also known as multiseat x . I tried this years ago on Debian , but never tried it on FreeBSD . > > https://wiki.archlinux.org/index.php/Xorg_multiseat > > > --- > Mark Saad | nonesuch@longcount.org > >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > From owner-freebsd-hackers@FreeBSD.ORG Wed Apr 8 16:50:27 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 801ACA08 for ; Wed, 8 Apr 2015 16:50:27 +0000 (UTC) Received: from mail-ie0-x22d.google.com (mail-ie0-x22d.google.com [IPv6:2607:f8b0:4001:c03::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3EA4E6BB for ; Wed, 8 Apr 2015 16:50:27 +0000 (UTC) Received: by iedfl3 with SMTP id fl3so89545272ied.1 for ; Wed, 08 Apr 2015 09:50:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0C5D8AJnoKj/16yuNJdRdNxmr7kL9B+E3lRmVG5/Was=; b=saMn9PTl1aXLhNFS0IsqvbtHOze8DUQVssfqRV7pZz8UkgkSF0zd4Fb+U/Se8WRY5t WGwIAJ5KOiDkQ+kc7EG80SxWHp/Xp/QK4jCPLFS30LKTeDcHevWoiZfQWhsBg4KuoJaT Z/MCUFy2vlr6I56YHcl6+LIO8emddRKDRUtoDIi0gnoNfFGrIKgj/nDCyfRbiPuSU1o1 nWtY4Ymk557SzhVoPiYFgAcqavLbp6PA2tKYCiG+xehnZCnEGGqFtjLe58FitKXlilsH +nvlbKyEzZGPIFWiU3fp1GgAgv/H3MmHd5uGc6+LHOb76EB/YEBlXTUosn9ZPQ8U+oFm 0kwQ== MIME-Version: 1.0 X-Received: by 10.42.216.145 with SMTP id hi17mr33595555icb.63.1428511826711; Wed, 08 Apr 2015 09:50:26 -0700 (PDT) Received: by 10.64.24.141 with HTTP; Wed, 8 Apr 2015 09:50:26 -0700 (PDT) In-Reply-To: References: Date: Wed, 8 Apr 2015 09:50:26 -0700 Message-ID: Subject: Re: multiple X servers From: Mehmet Erol Sanliturk To: Wojciech Puchar Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-hackers@freebsd.org" , Mark Saad , Garrett Cooper X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 16:50:27 -0000 On Wed, Apr 8, 2015 at 9:36 AM, Wojciech Puchar wrote: > seems like it should work. i will try it when actually get second card. > > On Wed, 8 Apr 2015, Mark Saad wrote: > > >> On Apr 8, 2015, at 6:57 AM, Wojciech Puchar wrote: >> >> >>>>> Not with Xinerama, but 2 separate ones, first using first keyboard and >>>>> first mouse, other using second keyboard and second mouse. >>>>> >>>>> i did this long time ago under NetBSD but X changed so much (as well >>>>> as GFX hardware) that this patches are now irrelevant. >>>>> >>>> >>>> As long as you have 2 separate xorg.confs and use two separate >>>> displays, I don?t see why not... >>>> >>> because when Xorg starts it messed up with all display adapters. >>> but maybe it changed. >>> >> >> While linux oriented the arch wiki has some good starting material for >> this; Also known as multiseat x . I tried this years ago on Debian , but >> never tried it on FreeBSD . >> >> https://wiki.archlinux.org/index.php/Xorg_multiseat >> >> >> --- >> Mark Saad | nonesuch@longcount.org >> >> _______________________________________________ >>> freebsd-hackers@freebsd.org mailing list >>> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >>> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@ >>> freebsd.org" >>> >> >> >> _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > If it is possible to execute a second instance of X , the following may be done : Include a command line parameter to X about configuration file to use instead of a default one : $ xorg --cf=A.conf $ xorg --cf=B.conf Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 9 13:04:34 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 12CEE451 for ; Thu, 9 Apr 2015 13:04:34 +0000 (UTC) Received: from mail-ie0-x22a.google.com (mail-ie0-x22a.google.com [IPv6:2607:f8b0:4001:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D207A611 for ; Thu, 9 Apr 2015 13:04:33 +0000 (UTC) Received: by iebmp1 with SMTP id mp1so99802957ieb.0 for ; Thu, 09 Apr 2015 06:04:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=u6ICOL/TTCuHRxtVL9dCJWEqiRsItXUQaBzoY/RKodk=; b=CVG/dKfjBK4s7aAMvgwy2eE2eAxuaWGvm6hnGYbPgkNtpXF2t6JIYl17gx7MoZ8gIY GH/kG5O9vQ0CdN2TnNKE265SOjEX8A0qU9YlOMu8lPqQzEtKWpV3+gf1elA8mrWm+Cwk us37cmurGo06MzYSN+d1sryadZe8+4Gt6Oe1f7jx9wjykPrxlW0Hq1dMjhBfDPBIHM4L lYKpGmzx6yW1s5jhYgc/s8/uMc8EQHyZkXCwLbPIEZyhLAinWwZCJbyuRMp4weG9MW1z weiXktbqDCX1YAZIh3guoz/7QWdoR7ZeDBm0f6tgnK9L/ZlFRgn4yOOafkwAq0F3LjGN ygJg== X-Received: by 10.50.78.9 with SMTP id x9mr19664542igw.44.1428584673177; Thu, 09 Apr 2015 06:04:33 -0700 (PDT) Received: from [172.22.132.117] ([192.119.231.58]) by mx.google.com with ESMTPSA id ka2sm8879974igb.2.2015.04.09.06.04.31 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 09 Apr 2015 06:04:31 -0700 (PDT) Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: lockf() vs. flock() -- lockf() not locking? From: Guy Helmer In-Reply-To: <20150407221525.GA99106@stack.nl> Date: Thu, 9 Apr 2015 08:04:31 -0500 Message-Id: <8F0FB344-3858-462F-A67D-97805379514C@gmail.com> References: <3950D855-0F4E-49E0-A388-4C7ED102B68B@gmail.com> <20150407221525.GA99106@stack.nl> To: Jilles Tjoelker X-Mailer: Apple Mail (2.2070.6) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: FreeBSD Hackers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 13:04:34 -0000 > On Apr 7, 2015, at 5:15 PM, Jilles Tjoelker wrote: >=20 > On Mon, Apr 06, 2015 at 04:18:09PM -0500, Guy Helmer wrote: >> Recently an application I use switched from using flock() for = advisory >> file locking to lockf() in the code that protects against concurrent >> writes to a file that is being shared and updated by multiple >> processes (not threads in a single process). The code seems reliable = =E2=80=94 >> a lock manager class opens the file & obtains the lock, then the >> read/update method opens the file using a separate file descriptor & >> reads/writes the file, flushes & closes the second file descriptor, >> and then destroys the lock manager object which unlocks the file & >> closes the first file descriptor. >=20 >> Surprisingly this simple change seems to have made the code = unreliable >> by allowing concurrent writers to the file and corrupting its >> contents: >=20 >> - if (flock(fd, LOCK_EX) !=3D 0) >> + if (lockf(fd, F_LOCK, 0) !=3D 0) >> throw std::runtime_error("Failed to get a lock of " + = filename); >=20 >> . . . >> if (fd !=3D -1) { >> - flock(fd, LOCK_EX); >> + lockf(fd, F_ULOCK, 0); >> close(fd); >> fd =3D -1; >> } >=20 >> =46rom my reading of the lockf(3) man page and reviewing the >> implementation in lib/libc/gen/lockf.c, and corresponding code in >> sys/kern/kern_descrip.c, it appears the lockf() call should be >> successfully obtaining an advisory lock over the whole file like a >> successful flock() did. However, I have a stress test that quickly >> corrupts the target file using the lockf() implementation, and the >> test fails to cause corruption using the flock() implementation. = I=E2=80=99ve >> instrumented the code, and it's clear that multiple processes are >> simultaneously in the block of code after the =E2=80=9Clockf(fd, = F_LOCK, 0)=E2=80=9D >> line. >=20 >> Am I missing something obvious? Any ideas? >=20 > With lockf/fcntl locks, the close of the second file descriptor = actually > already unlocks the file. If there is another close and open in there, > it would explain your problem. Both the lockf(3) and the fcntl(2) man > pages mention these strange semantics, but only fcntl(2) clearly warns > about them. With flock locks, opening the file another time will not > cause problems. >=20 > The second thing that will not work with lockf/fcntl locks is having a > child process inherit them. >=20 > Changing flock() to lockf() seems like a bad idea, particularly in a > reusable "lock manager" class, since it is then harder to see what > operations must be avoided to avoid losing the lock. >=20 > There is a proposal in the Austin Group for the next version of POSIX = to > add a form of file lock that allows both range locking and proper > (flock-style) semantics. Thank you! I had forgotten the text in the fcntl(2) page about = fcntl/lockf semantics (haven=E2=80=99t read that for a while). I have = verified the method in question uses the lock manager to lock the file, = opens the file to read the contents, closes the file [thus loosing the = lockf lock], does some work, and then opens & writes the file to save = the updates. Regards, Guy From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 9 17:16:29 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1C05CE05; Thu, 9 Apr 2015 17:16:29 +0000 (UTC) Received: from mail-vn0-x22d.google.com (mail-vn0-x22d.google.com [IPv6:2607:f8b0:400c:c0f::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE5769AA; Thu, 9 Apr 2015 17:16:28 +0000 (UTC) Received: by vnbg7 with SMTP id g7so23407222vnb.10; Thu, 09 Apr 2015 10:16:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=aPpT/UoKKEAFyNtH8RhmjqeRj9c8ix+k9bP6QXT/hY0=; b=DrDNq1Rjc8TllP10/rhNyjGvKhTq63PP27iM1dwXvZ6ln3wDFnyiza0/OTd3O/uCrK FKAyMY9/Daw4ixZRc9w62FQ2hwygwujRsNRjcZ2D18ppp6gf/zRJe9JQmcIpgXjzAuhB uj3oHqATOU3+FBHESA69Q2reFT8asxevqgMVkTTW/NwtnxYCaekGi7EGl4VsQbpbphwm cfmDzVBZHCAJkaazZyXTyzSKUKEdAEhehUlcq17Qpvh28W5Zspn4Otedry5d25dlG6Hz /CEg0QFpdY/JIOpIjpqscLIzbU7+1Ldj5j9K9cx/ap7SgKsU7zGQr2lEM/dmGnNsLXyC 950w== MIME-Version: 1.0 X-Received: by 10.140.83.19 with SMTP id i19mr36305647qgd.97.1428599787758; Thu, 09 Apr 2015 10:16:27 -0700 (PDT) Received: by 10.140.38.73 with HTTP; Thu, 9 Apr 2015 10:16:27 -0700 (PDT) In-Reply-To: References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> Date: Thu, 9 Apr 2015 10:16:27 -0700 Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Jim Harris To: Konstantin Belousov Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-hackers@freebsd.org" , Tobias Oberstein , Michael Fuckner , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 17:16:29 -0000 On Wed, Apr 1, 2015 at 2:34 PM, Jim Harris wrote: > > > Yes - this is exactly the problem. > > nvme does use MSI-X if it can allocate the vectors (one per core). With > 48 cores, > I suspect we are quickly running out of vectors, so NVMe is reverting to > INTx. > > Could you actually send vmstat -ia (I left off the 'a' previously) - just > so we can > see all allocated interrupt vectors. > > As an experiment, can you try disabling hyperthreading - this will reduce > the > number of cores and should let you get MSI-X vectors allocated for at least > the first couple of NVMe controllers. Then please re-run your performance > test on one of those controllers. > > sys/x86/x86/local_apic.c defines APIC_NUM_IOINTS to 191 - it looks like > this > is the actual limit for MSI-X vectors, even though NUM_MSI_INTS is set to > 512. > Update on this vector allocation discussion: The DC P3*00 SSDs have a 32-entry MSI-X vector table, which is fewer than the 48 cores on your system, so it falls back to INTx. I committed r281280 which will make it fall back to 2 MSI-X vectors (one for admin queue, one for a single I/O queue) which is better than INTx. In the future this driver will be improved to utilize the available number of interrupts rather than falling back to a single I/O queue. (Based on your ramdisk performance data, it does not appear that lack of per-CPU NVMe I/O queues is the cause of the performance issues on this system - but I'm working to confirm on a system in my lab.) I also root caused a vector allocation issue and filed PR199321. In summary, driver initialization occurs before APs are started, so MSI-X allocation code only allocates IRQs from the BSP's lapic. The BSP's lapic can only accommodate 191 (APIC_NUM_IOINTS) total vectors. The allocation code is designed to round-robin allocations across all lapics, but does not allow it until after the APs are started which is after driver initialization during boot. Drivers that are loaded after boot should round-robin IRQ allocation across all physical cores, up to the NUM_MSI_INTS (512) limit. > > >> >> I have the following patch for a long time, it allowed to increase pps >> in iperf and similar tests when DMAR is enabled. In your case it could >> reduce the rate of the DMAR interrupts. >> >> diff --git a/sys/x86/iommu/intel_ctx.c b/sys/x86/iommu/intel_ctx.c >> index a18adcf..b23a4c1 100644 >> --- a/sys/x86/iommu/intel_ctx.c >> +++ b/sys/x86/iommu/intel_ctx.c >> @@ -586,6 +586,18 @@ dmar_ctx_unload_entry(struct dmar_map_entry *entry, >> bool free) >> } >> } >> >> +static struct dmar_qi_genseq * >> +dmar_ctx_unload_gseq(struct dmar_ctx *ctx, struct dmar_map_entry *entry, >> + struct dmar_qi_genseq *gseq) >> +{ >> + >> + if (TAILQ_NEXT(entry, dmamap_link) != NULL) >> + return (NULL); >> + if (ctx->batch_no++ % dmar_batch_coalesce != 0) >> + return (NULL); >> + return (gseq); >> +} >> + >> void >> dmar_ctx_unload(struct dmar_ctx *ctx, struct dmar_map_entries_tailq >> *entries, >> bool cansleep) >> @@ -619,8 +631,7 @@ dmar_ctx_unload(struct dmar_ctx *ctx, struct >> dmar_map_entries_tailq *entries, >> entry->gseq.gen = 0; >> entry->gseq.seq = 0; >> dmar_qi_invalidate_locked(ctx, entry->start, entry->end - >> - entry->start, TAILQ_NEXT(entry, dmamap_link) == NULL ? >> - &gseq : NULL); >> + entry->start, dmar_ctx_unload_gseq(ctx, entry, >> &gseq)); >> } >> TAILQ_FOREACH_SAFE(entry, entries, dmamap_link, entry1) { >> entry->gseq = gseq; >> diff --git a/sys/x86/iommu/intel_dmar.h b/sys/x86/iommu/intel_dmar.h >> index 2865ab5..6e0ab7f 100644 >> --- a/sys/x86/iommu/intel_dmar.h >> +++ b/sys/x86/iommu/intel_dmar.h >> @@ -93,6 +93,7 @@ struct dmar_ctx { >> u_int entries_cnt; >> u_long loads; >> u_long unloads; >> + u_int batch_no; >> struct dmar_gas_entries_tree rb_root; >> struct dmar_map_entries_tailq unload_entries; /* Entries to >> unload */ >> struct dmar_map_entry *first_place, *last_place; >> @@ -339,6 +340,7 @@ extern dmar_haddr_t dmar_high; >> extern int haw; >> extern int dmar_tbl_pagecnt; >> extern int dmar_match_verbose; >> +extern int dmar_batch_coalesce; >> extern int dmar_check_free; >> >> static inline uint32_t >> diff --git a/sys/x86/iommu/intel_drv.c b/sys/x86/iommu/intel_drv.c >> index c239579..e7dc3f9 100644 >> --- a/sys/x86/iommu/intel_drv.c >> +++ b/sys/x86/iommu/intel_drv.c >> @@ -153,7 +153,7 @@ dmar_count_iter(ACPI_DMAR_HEADER *dmarh, void *arg) >> return (1); >> } >> >> -static int dmar_enable = 0; >> +static int dmar_enable = 1; >> static void >> dmar_identify(driver_t *driver, device_t parent) >> { >> diff --git a/sys/x86/iommu/intel_utils.c b/sys/x86/iommu/intel_utils.c >> index f696f9d..d3c3267 100644 >> --- a/sys/x86/iommu/intel_utils.c >> +++ b/sys/x86/iommu/intel_utils.c >> @@ -624,6 +624,7 @@ dmar_barrier_exit(struct dmar_unit *dmar, u_int >> barrier_id) >> } >> >> int dmar_match_verbose; >> +int dmar_batch_coalesce = 100; >> >> static SYSCTL_NODE(_hw, OID_AUTO, dmar, CTLFLAG_RD, NULL, ""); >> SYSCTL_INT(_hw_dmar, OID_AUTO, tbl_pagecnt, CTLFLAG_RD, >> @@ -632,6 +633,9 @@ SYSCTL_INT(_hw_dmar, OID_AUTO, tbl_pagecnt, >> CTLFLAG_RD, >> SYSCTL_INT(_hw_dmar, OID_AUTO, match_verbose, CTLFLAG_RWTUN, >> &dmar_match_verbose, 0, >> "Verbose matching of the PCI devices to DMAR paths"); >> +SYSCTL_INT(_hw_dmar, OID_AUTO, batch_coalesce, CTLFLAG_RW | CTLFLAG_TUN, >> + &dmar_batch_coalesce, 0, >> + "Number of qi batches between interrupt"); >> #ifdef INVARIANTS >> int dmar_check_free; >> SYSCTL_INT(_hw_dmar, OID_AUTO, check_free, CTLFLAG_RWTUN, >> > > From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 9 21:08:11 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18A8BB6B; Thu, 9 Apr 2015 21:08:11 +0000 (UTC) Received: from mail-wg0-x22c.google.com (mail-wg0-x22c.google.com [IPv6:2a00:1450:400c:c00::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8FACB97; Thu, 9 Apr 2015 21:08:10 +0000 (UTC) Received: by wgso17 with SMTP id o17so21037156wgs.1; Thu, 09 Apr 2015 14:08:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=7ej1afHRCxGtf7PhbWMU1/n2+c2ji4s4Ljj2hu6DiOc=; b=Z8JrNRp0Vdhf4saFuQu/nHA7nYU5XubZ0AzB/D1uI89AWIe3f7+P2L21eC2PyyH0FS zM26tjXyzGc9yFqxtUQVxAOmNArMxWjX9vSQsP3mmoarj74eUyjOvictSFJL7Xop/bVE zwEn0u2ULZU3l2Cu9Xvc0pw7kc6RM0G90gz8igkhc/iHwrI6/wMPsv63MXD8C3QvVHRS 0fKpszHlpnEO6Nk+akkOWxYOS431ADS5lhX/gO0NORxVrh8jg6VZuHEUmAPzCW9CeIJB Y9WbbQV+DOVzhj9Rdwnt7VUCrk/1PUF67JKQq4KM30J6ujDDfGeuB7zxiReBCC7w43b3 REhA== X-Received: by 10.194.200.194 with SMTP id ju2mr55227947wjc.61.1428613689138; Thu, 09 Apr 2015 14:08:09 -0700 (PDT) Received: from [192.168.43.73] ([89.204.139.142]) by mx.google.com with ESMTPSA id l3sm21780206wiv.18.2015.04.09.14.08.03 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Apr 2015 14:08:04 -0700 (PDT) Message-ID: <5526EA33.6090004@gmail.com> Date: Thu, 09 Apr 2015 23:08:03 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Jim Harris , Konstantin Belousov Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hackers@freebsd.org" , Michael Fuckner , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 21:08:11 -0000 Hi Jim, thanks for coming back to this and your work / infos - highly appreciated! > (Based on your ramdisk performance data, it does not > appear that lack of per-CPU NVMe I/O queues is the cause of the performance > issues on this system - My unscientific gut feeling is: it might be related to NUMA in general. The memory performance https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_memperf.md#results-48-core-numa-machine is slower than a E3 single socket Xeon https://github.com/oberstet/scratchbox/blob/master/freebsd/cruncher/results/freebsd_memperf.md#results-small-xeon-machine The E3 is Haswell at 3.4 GHz, whereas the E7 is one gen. older and 3.0 GHz, but I don't think this explains the very large difference. The 4 socket box should have an aggregate main memory bandwidth of 4 x 85GB/s = 340 GB/s The measured numbers are orders smaller. > but I'm working to confirm on a system in my lab.) FWIW, the box I am testing is http://www.quantaqct.com/Product/Servers/Rackmount-Servers/4U/QuantaGrid-Q71L-4U-p18c77c70c83c79 The box is maxed out on RAM, CPU (mostly), internal SSDs, as well as PCIe cards (it has 10 slots). There are very few x86 systems with bigger scale-up. Tops out with the SGI Ultraviolet UV2000. But this is totally exotic, whereas above is pure Intel design. How about Intel donating such a baby to FBSD foundation to get NUMA and everything sorted out? Street price is roughly 150k, but given most of the components are made by Intel, should be cheaper for Intel;) == Sadly, given the current state of affairs, I couldn't support targeting FreeBSD on this system any longer. Customer wants to go to production soonish. We'll be using Linux / SLES12. Performance at block device level there is as expected from Intel datasheets. Means: massive! We now "only" need to translate those millions of IOPS from block device to filesystem level and then database (PostgreSQL). Ha, will be fun;) And I will miss ZFS and all the FreeBSD goodies =( Cheers, /Tobias From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 9 21:22:11 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 01FF65B5; Thu, 9 Apr 2015 21:22:11 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B620FD85; Thu, 9 Apr 2015 21:22:10 +0000 (UTC) Received: by iget9 with SMTP id t9so3067897ige.1; Thu, 09 Apr 2015 14:22:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=YWs0VzRaPc0PW+dbh3oC47JLHUaQe588YZ/eNts5lwQ=; b=0pm/IdoccPzAAk8Ac+axaqzOI1HrlTVNOEkJHj3lvFOKZ7wyGPcpozf4kwufgCgY4E ZfMszkQqrYq7U3+/e8Rcok0E4m0qGEEzOeXGbdJWcHZIEzbNhkYr00OOUvVzhvB3piDr sO+Uin9wQ0iDHCMTZqicnPlWQkQEWhgbaLLYmfsTA1AeMpXOx7dMC54L16itfrWYEgoy 0QBK4W1KtQeKh74vW5z68kAp1QJhS5knGPSvJ5cyDZuZRFntbTnt5JozGn10wHLnNb1R QOvpgBnZYDG8z2vLCFLJJb+YfDs84bnyEGvB4+19tOifjZAW7cOrf1p+BxD8eNRGMzsh 12UQ== MIME-Version: 1.0 X-Received: by 10.107.136.25 with SMTP id k25mr48571426iod.88.1428614530187; Thu, 09 Apr 2015 14:22:10 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Thu, 9 Apr 2015 14:22:09 -0700 (PDT) In-Reply-To: <5526EA33.6090004@gmail.com> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <5526EA33.6090004@gmail.com> Date: Thu, 9 Apr 2015 14:22:09 -0700 X-Google-Sender-Auth: dmdv4AWO5FJup_ZGlsXH3SjcOV0 Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Adrian Chadd To: Tobias Oberstein Content-Type: text/plain; charset=UTF-8 Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 21:22:11 -0000 Hi, Dell has graciously loaned me a bunch of hardware to continue doing NUMA development on, but I have no NVMe hardware. I'm hoping people at Intel can continue kicking along any desires for NUMA that they require. (Which they have, fwiw.) -adrian From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 10 16:07:52 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9435FFBE; Fri, 10 Apr 2015 16:07:52 +0000 (UTC) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1CE15DCE; Fri, 10 Apr 2015 16:07:52 +0000 (UTC) Received: by wiun10 with SMTP id n10so2218781wiu.1; Fri, 10 Apr 2015 09:07:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=xHxDm2NVV49XqVwEfpdGF8RFpyyEpHxV9KvangNlvfs=; b=xutWrkTnCELg8VaChEM9vFVV9G/bcRABCSSIy6OptBlcBZ9AkBaLLwaxYwEah1eKmr 7/+LjJDXgZSZYgmEG18Ex2VsWrX/M2XvYp6jI0HhYOH9xM9yrq4GV6kJqFyGC6Q/vFgE KAlFYTwYM10YYDigSqychosMVGhxi3Q/pbpujBVkmE/b4mrmK/VY4MX0uh8pddRPeXpK x+UEaETCrx7cm1Ptlgepk+O3YT2dlEKCJOGxfJmDxlTUm6Ob2SXMnWW/cX+n61kxGPVG kyaryZsaZw1zzUG78URRFSQPqd4WxheFdDhznsABGR8cvuDJiZvAl4GHsR1ZocTjm6TA V4qg== X-Received: by 10.180.102.130 with SMTP id fo2mr16445201wib.30.1428682070542; Fri, 10 Apr 2015 09:07:50 -0700 (PDT) Received: from [192.168.43.73] ([89.15.236.65]) by mx.google.com with ESMTPSA id w8sm3621220wja.4.2015.04.10.09.07.49 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Apr 2015 09:07:49 -0700 (PDT) Message-ID: <5527F554.2030806@gmail.com> Date: Fri, 10 Apr 2015 18:07:48 +0200 From: Tobias Oberstein User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Adrian Chadd Subject: Re: NVMe performance 4x slower than expected References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <5526EA33.6090004@gmail.com> In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" , Michael Fuckner , Jim Harris , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 16:07:52 -0000 Hi Adrian, > Dell has graciously loaned me a bunch of hardware to continue doing FWIW, Dell has a roughly comparable system: Dell R920. But they don't have Intel NVMe's on their menu, only Samsung (and FusionIO, but that's not NVMe). > NUMA development on, but I have no NVMe hardware. I'm hoping people at The 8 NVMe PCIe SSDs in the box we're deploying are a key feature of this system (will be a data-warehouse). A single NVMe probably won't have triggered (all) issues we experienced. We are using the largest model (2TB), and this amounts to 50k bucks for all eight. The smallest model (400GB) is 1.5k, so 12k in total. > Intel can continue kicking along any desires for NUMA that they > require. (Which they have, fwiw.) It's already awesome that Intel has senior engineers working on FreeBSD driver code! And it would underline Intel's Open-source commitment and tech leadership if they donated a couple of these beefy NVMes. The specs are incredible, but extracing all the performance is non-trivial .. Cheers, /Tobias From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 10 16:58:03 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8E682B31; Fri, 10 Apr 2015 16:58:03 +0000 (UTC) Received: from mail-qk0-x229.google.com (mail-qk0-x229.google.com [IPv6:2607:f8b0:400d:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E332392; Fri, 10 Apr 2015 16:58:03 +0000 (UTC) Received: by qkhg7 with SMTP id g7so37269846qkh.2; Fri, 10 Apr 2015 09:58:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=h08uLZVGKITrUQcopce8baambKS6hqiwFVVfetzxL8c=; b=IGNghlS7J2isTnR3xzcIBprZ1/fJi1YZgbJEgGS2tYh4RDBJAfJqj57onmskvx/3ll CNPgxfVjGHETLkDOhDXk2n5ZVnEE9hJkmp1JA7Kh+7Kbsdj9uegHxmLF0UzpUbooaKEb cm1hzMIsZSXJ4s8AMHusK6qtz3Dj8laZDFbll6Q6/KXtA/TIksbvPwdRV79lxZKxw9pD vBCX8XSxOgQ6b3WT2rdUlWVa8yKowrnESvMazgZzXTO3INHHj4UrGqYOSc6a83MyHr/t 5i0Zbi5ciovymU5CYwFq7NBmnQPC32TTjKAomNHiw62GdWsTdgV7MFX1T/3n+/xQNXCo PCzg== MIME-Version: 1.0 X-Received: by 10.55.53.137 with SMTP id c131mr4772699qka.102.1428685082463; Fri, 10 Apr 2015 09:58:02 -0700 (PDT) Received: by 10.140.38.73 with HTTP; Fri, 10 Apr 2015 09:58:02 -0700 (PDT) In-Reply-To: <5527F554.2030806@gmail.com> References: <551BC57D.5070101@gmail.com> <551C5A82.2090306@gmail.com> <20150401212303.GB2379@kib.kiev.ua> <5526EA33.6090004@gmail.com> <5527F554.2030806@gmail.com> Date: Fri, 10 Apr 2015 09:58:02 -0700 Message-ID: Subject: Re: NVMe performance 4x slower than expected From: Jim Harris To: Tobias Oberstein Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: Konstantin Belousov , "freebsd-hackers@freebsd.org" , Adrian Chadd , Michael Fuckner , Alan Somers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 16:58:03 -0000 On Fri, Apr 10, 2015 at 9:07 AM, Tobias Oberstein < tobias.oberstein@gmail.com> wrote: > Hi Adrian, > > > Dell has graciously loaned me a bunch of hardware to continue doing > > FWIW, Dell has a roughly comparable system: Dell R920. But they don't have > Intel NVMe's on their menu, only Samsung (and FusionIO, but that's not > NVMe). > > NUMA development on, but I have no NVMe hardware. I'm hoping people at >> > > The 8 NVMe PCIe SSDs in the box we're deploying are a key feature of this > system (will be a data-warehouse). A single NVMe probably won't have > triggered (all) issues we experienced. > > We are using the largest model (2TB), and this amounts to 50k bucks for > all eight. The smallest model (400GB) is 1.5k, so 12k in total. > > Intel can continue kicking along any desires for NUMA that they >> require. (Which they have, fwiw.) >> > > It's already awesome that Intel has senior engineers working on FreeBSD > driver code! And it would underline Intel's Open-source commitment and tech > leadership if they donated a couple of these beefy NVMes. > Intel has agreed to send DC P3700 samples to the FreeBSD Foundation to put in the cluster for this kind of work - we are working on getting these through the internal sample distribution process at the moment. -Jim From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 10 19:19:29 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EDA9389D; Fri, 10 Apr 2015 19:19:29 +0000 (UTC) Received: from mail-vn0-x233.google.com (mail-vn0-x233.google.com [IPv6:2607:f8b0:400c:c0f::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A46EC8C6; Fri, 10 Apr 2015 19:19:29 +0000 (UTC) Received: by vnbg7 with SMTP id g7so7864484vnb.11; Fri, 10 Apr 2015 12:19:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=vzTm65/h1i4DdtmHAQHzQEivAfNoXWQ7BVpl006+Dr0=; b=wVAaGqV8LTEtQ+pqZvIIxul7XPS/7J5R0DpjqTBHFx7OcPUfSNN8ZSGpYrWYw5HEKQ t4wNq7D0eKn9ObCUrCW57Qpbp5D5SoAGyzckX88+Uh4zCvQL+PFBPItLgF/zki14eB0R ZBxDWn87fSdIDPeGfPkFoNQynQPrHMOTGNJqH+fzB5nMLJ6J1kut4t7XyWlxgWqhqIea 7XEJK+sAoPg8utdCaRdudyPnN9gzFpUjrH755hoj4dNeDBUiqRRIrvlgVRZOm2WMKohK dr+x0qBz7hlx4MhUfUDg/OO8qum+Rebi+zk87+2clkHojOEhEHvi/XeszcWtvtW7+aX8 DBMQ== X-Received: by 10.52.78.35 with SMTP id y3mr3090046vdw.5.1428693568804; Fri, 10 Apr 2015 12:19:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.61.100 with HTTP; Fri, 10 Apr 2015 12:19:08 -0700 (PDT) From: Pratik Singhal Date: Sat, 11 Apr 2015 00:49:08 +0530 Message-ID: Subject: pmap_demote_section: No l2_bucket for wired mapping panic To: freebsd-hackers , "freebsd-arm@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 19:19:30 -0000 Hello all, I am trying to boot into FreeBSD 11.0 current on my CubieBoard 1 . I have compiled the kernel in two ways along with "options ARM_NEW_PMAP" as well as without it. Whenever , I try to boot into the system the kernel panics . I am pasting the stack trace and the boot log for the compilation with ARM_NEW_PMAP enabled. People have reported previously that on R-pi if we compile with the "options ARM_NEW_PMAP" the panic issue is resolved, but it seems to have no effect on Cubieboard 1. Any ideas what could be the problem ? boot log and stack trace :- http://pastie.org/10085231 -- Regards, Pratik Singhal From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 10 21:45:37 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA9F2794 for ; Fri, 10 Apr 2015 21:45:37 +0000 (UTC) Received: from mail-ie0-x232.google.com (mail-ie0-x232.google.com [IPv6:2607:f8b0:4001:c03::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B158FBFF for ; Fri, 10 Apr 2015 21:45:37 +0000 (UTC) Received: by iejt8 with SMTP id t8so27879997iej.2 for ; Fri, 10 Apr 2015 14:45:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:content-type; bh=LqlRVIO5On0+hl511n9R5jiLatbtoP8EblTa53LjKqk=; b=SUA0I49yODVtB0E5zec/kunR0NwD/b7zPE27wNs6CJkrqaFO7K7rgZwsmG3Wkg2ZCA 2+A/wG9/9FoBD4ldBKJC+S/yUMIoN4Kk0Tv9/zabti/LR8DhbFNqz/SErHBfOjmdNHS+ 6duEtHaN/hbDStK18MTsdJoNXB8hb9N6x7Mst6g5enOize3MaBiea07jT7lZB7it+ZHC nhAOtPGzXeYOn3f0aaJvFiOCcTrHMXMe24A/YnpwZi7y1h1KNUhyEBngXsTUoipZoRk8 ggJNHFh1Ds3JLiyIInIxXwl6K5G+C3C8+B3LwnkxXm5Fdmt71TFi0X05G2z8Sy8H2IqI eIew== X-Received: by 10.42.171.8 with SMTP id h8mr6275293icz.25.1428702336837; Fri, 10 Apr 2015 14:45:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.131.194 with HTTP; Fri, 10 Apr 2015 14:45:16 -0700 (PDT) From: Alex Merritt Date: Fri, 10 Apr 2015 17:45:16 -0400 Message-ID: Subject: Page Coloring Manipulation To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 21:45:38 -0000 Hello hackers, I am interested in playing with the page-coloring support within the virtual memory subsystem. Specifically: i. Methods to specify the number of colors a task receives ii. Methods to change the quantity of colors at runtime iii. Understand what areas of memory coloring governs, e.g., just task data/instructions, file-backed/anon, or even task page-table entries created by the kernel? Great if there are user-level methods, but not necessary. Does the kernel determine these values automatically? Is it disabled by default or always exercised? Here is what I found thus far -- // during page fault, in vm_fault_hold() #if VM_NRESERVLEVEL > 0 if ((fs.object->flags & OBJ_COLORED) == 0) { fs.object->flags |= OBJ_COLORED; fs.object->pg_color = atop(vaddr) - fs.pindex; } #endif // and in vm_object_init #if VM_NRESERVLEVEL > 0 kernel_object->flags |= OBJ_COLORED; kernel_object->pg_color = (u_short)atop(VM_MIN_KERNEL_ADDRESS); #endif So it seems an object must be explicitly marked as "colorable", which is true always? Please help me interpret. I am new to FreeBSD hacking ;) and appreciate any pointers for me to study further. Thanks, Alex From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 10 23:01:05 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C8EAC2B1; Fri, 10 Apr 2015 23:01:05 +0000 (UTC) Received: from mail-ig0-x22c.google.com (mail-ig0-x22c.google.com [IPv6:2607:f8b0:4001:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F2C730E; Fri, 10 Apr 2015 23:01:05 +0000 (UTC) Received: by ignm3 with SMTP id m3so22807441ign.0; Fri, 10 Apr 2015 16:01:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=T/q4zjo/GeL+7vg7RM+xP4sA2cVb+q8//52CXUs0YzQ=; b=wMoinXlQRkSdXMydtuX3S8X3JLyMQyFZNxTgQ+wbxmxxiL63+G+kXnGNsN33Kfp+r4 YlDyfuGQxoVNJE5DuEDEXqNjvFgnkqxytm/oRiljCdh1wAJuiCRkJn4nEL+Oxd0nS63O vo8iaEifFB8K2CzfjHIRyKxxLO/a+ysBRIjU0l/A353HRA9VMmqCAZJ7RtCrMOmAqLWH h9dCYVuI75Y+YRQpgk5uN1jbf3hrLXvTqvcdgj4MqQIOgf5K3E2fEKmVuCe7cmtwDFMm STSvi7wUWnTHYIXaIpXTDuVyP584RbYQnMXHQTDWv0eFf2p6XianZr6OEx1fxeXP6/aN cGBQ== MIME-Version: 1.0 X-Received: by 10.42.27.14 with SMTP id h14mr6497227icc.19.1428706865058; Fri, 10 Apr 2015 16:01:05 -0700 (PDT) Received: by 10.64.28.43 with HTTP; Fri, 10 Apr 2015 16:01:04 -0700 (PDT) In-Reply-To: References: Date: Sat, 11 Apr 2015 01:01:04 +0200 Message-ID: Subject: Re: pmap_demote_section: No l2_bucket for wired mapping panic From: Svatopluk Kraus To: Pratik Singhal Content-Type: text/plain; charset=UTF-8 Cc: freebsd-hackers , "freebsd-arm@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Apr 2015 23:01:05 -0000 The point of ARM_NEW_PMAP option is that there are no l2_buckets in new armv6 pmap. So, as long as you get same panic, you are not running kernel compiled with ARM_NEW_PMAP option. Svatopluk Kraus On Fri, Apr 10, 2015 at 9:19 PM, Pratik Singhal wrote: > Hello all, I am trying to boot into FreeBSD 11.0 current on my CubieBoard 1 > . I have compiled the kernel in two ways along with "options ARM_NEW_PMAP" > as well as without it. > > Whenever , I try to boot into the system the kernel panics . I am pasting > the stack trace and the boot log for the compilation with ARM_NEW_PMAP > enabled. > > People have reported previously that on R-pi if we compile with the > "options ARM_NEW_PMAP" the panic issue is resolved, but it seems to have no > effect on Cubieboard 1. > > Any ideas what could be the problem ? > > boot log and stack trace :- http://pastie.org/10085231 > > -- > Regards, > Pratik Singhal > _______________________________________________ > freebsd-arm@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 11 01:28:51 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE4F69CA; Sat, 11 Apr 2015 01:28:51 +0000 (UTC) Received: from mail-vn0-x22d.google.com (mail-vn0-x22d.google.com [IPv6:2607:f8b0:400c:c0f::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7C01C283; Sat, 11 Apr 2015 01:28:51 +0000 (UTC) Received: by vnbf1 with SMTP id f1so9933184vnb.5; Fri, 10 Apr 2015 18:28:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Fl+jlTwfEXOzLHRWtl93+7T8JgNdV79t79p/d/GGj1Q=; b=wlXIc5X6IgkkYl2u4d9BoPdiZIdKZ88vaCeXdmqtw/vd6tb94MFl1nWxpEgPjusY6t MoVM+r4NJZ7Hp1NszszZOAIcuvekhEUOyT6ItVxIw/4y0TQNFoAbeFeLvgSt1e+d+LQ2 RSdoEkc7S8ouYJ7Aqwz32ctoQ/L4yswsoKTuh9WGm6WMeLpZODnd0LaySoKxrQayjxTF Kjo+lbqqJOF43eSVP3l/qsiObNjea6f6GMxxkYDeMrovvwYVJ9WBged1ZugCHrEhHGzM mDbnYsYZ220UEReBGh2sn9AF3RaGwQL82naOmzx9x2BZdck4mgOFMViMMmZQn31DZUqT 2yNw== X-Received: by 10.52.14.200 with SMTP id r8mr4514714vdc.79.1428715729766; Fri, 10 Apr 2015 18:28:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.61.100 with HTTP; Fri, 10 Apr 2015 18:28:29 -0700 (PDT) In-Reply-To: References: From: Pratik Singhal Date: Sat, 11 Apr 2015 06:58:29 +0530 Message-ID: Subject: Re: pmap_demote_section: No l2_bucket for wired mapping panic To: Svatopluk Kraus Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-hackers , "freebsd-arm@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 01:28:51 -0000 I have double checked everything, recompiled kernel from scratch imade sure that I ncluded the line "options ARM_NEW_PMAP" in the kernel configuration file and I am still getting the same panic. Is there something at runtime that I can do to make sure that the kernel I am running is compiled with ARM_NEW_PMAP ? On Sat, Apr 11, 2015 at 4:31 AM, Svatopluk Kraus wrote: > The point of ARM_NEW_PMAP option is that there are no l2_buckets in > new armv6 pmap. So, as long as you get same panic, you are not running > kernel compiled with ARM_NEW_PMAP option. > > Svatopluk Kraus > > > On Fri, Apr 10, 2015 at 9:19 PM, Pratik Singhal wrote: > > Hello all, I am trying to boot into FreeBSD 11.0 current on my > CubieBoard 1 > > . I have compiled the kernel in two ways along with "options > ARM_NEW_PMAP" > > as well as without it. > > > > Whenever , I try to boot into the system the kernel panics . I am pasting > > the stack trace and the boot log for the compilation with ARM_NEW_PMAP > > enabled. > > > > People have reported previously that on R-pi if we compile with the > > "options ARM_NEW_PMAP" the panic issue is resolved, but it seems to have > no > > effect on Cubieboard 1. > > > > Any ideas what could be the problem ? > > > > boot log and stack trace :- http://pastie.org/10085231 > > > > -- > > Regards, > > Pratik Singhal > > _______________________________________________ > > freebsd-arm@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > -- Regards, Pratik Singhal From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 11 02:36:02 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C0F123B5; Sat, 11 Apr 2015 02:36:02 +0000 (UTC) Received: from relay.mailchannels.net (si-002-i39.relay.mailchannels.net [184.154.112.204]) by mx1.freebsd.org (Postfix) with ESMTP id D401FADC; Sat, 11 Apr 2015 02:36:00 +0000 (UTC) X-Sender-Id: duocircle|x-authuser|hippie Received: from smtp6.ore.mailhop.org (ip-10-213-14-133.us-west-2.compute.internal [10.213.14.133]) by relay.mailchannels.net (Postfix) with ESMTPA id 05D5E1D0AA3; Sat, 11 Apr 2015 02:20:13 +0000 (UTC) X-Sender-Id: duocircle|x-authuser|hippie Received: from smtp6.ore.mailhop.org (smtp6.ore.mailhop.org [10.45.8.167]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:2500 (trex/5.4.8); Sat, 11 Apr 2015 02:20:13 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: duocircle|x-authuser|hippie X-MailChannels-Auth-Id: duocircle X-MC-Loop-Signature: 1428718813428:1587006772 X-MC-Ingress-Time: 1428718813427 Received: from c-73-34-117-227.hsd1.co.comcast.net ([73.34.117.227] helo=ilsoft.org) by smtp6.ore.mailhop.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1Ygl1n-0007jV-Tb; Sat, 11 Apr 2015 02:20:12 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t3B2K9Hp009055; Fri, 10 Apr 2015 20:20:09 -0600 (MDT) (envelope-from ian@freebsd.org) X-Mail-Handler: DuoCircle Outbound SMTP X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@duocircle.com (see https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information for abuse reporting information) X-MHO-User: U2FsdGVkX1/Z7zGJvfgk12j4JfBDyl1g Message-ID: <1428718809.1182.31.camel@freebsd.org> Subject: Re: pmap_demote_section: No l2_bucket for wired mapping panic From: Ian Lepore To: Pratik Singhal Date: Fri, 10 Apr 2015 20:20:09 -0600 In-Reply-To: References: Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.12.10 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-AuthUser: hippie Cc: freebsd-hackers , "freebsd-arm@freebsd.org" , Svatopluk Kraus X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 02:36:02 -0000 Reply at the bottom, where it belongs. On Sat, 2015-04-11 at 06:58 +0530, Pratik Singhal wrote: > I have double checked everything, recompiled kernel from scratch imade sure > that I ncluded the line "options ARM_NEW_PMAP" in the kernel configuration > file and I am still getting the same panic. > > Is there something at runtime that I can do to make sure that the kernel I > am running is compiled with ARM_NEW_PMAP ? > > > > On Sat, Apr 11, 2015 at 4:31 AM, Svatopluk Kraus wrote: > > > The point of ARM_NEW_PMAP option is that there are no l2_buckets in > > new armv6 pmap. So, as long as you get same panic, you are not running > > kernel compiled with ARM_NEW_PMAP option. > > > > Svatopluk Kraus > > > > [...] > > > >From the announcement of the ARM_NEW_PMAP option... If you need to check whether the new or old code is running on a system, use "sysctl vm.pmap.pte1.promotions", if that gives an "unknown oid" error you're running the old code. Other evidence: revolution > grep l2_bucket pmap-v6-new.c revolution > You can't be getting a panic about l2_bucket stuff from code that doesn't have anything in it about l2 buckets. Something must have gone wrong with the way you built and installed the new kernel. -- Ian From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 11 09:14:26 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BB0C69E; Sat, 11 Apr 2015 09:14:26 +0000 (UTC) Received: from mail-vn0-x231.google.com (mail-vn0-x231.google.com [IPv6:2607:f8b0:400c:c0f::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39496832; Sat, 11 Apr 2015 09:14:26 +0000 (UTC) Received: by vnbf1 with SMTP id f1so10636891vnb.5; Sat, 11 Apr 2015 02:14:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=0tgh5m+Y5ayJDVTtHp2ORpmfKq7Duu67Bc1EmSGCRss=; b=a+qNB/zTvlXkj6jHne7QtPAGLxS3qwEfUNQ9dyoggcdJINCVsFEza8JGXeO8zVv2np XxqLoE2MzkyGtrxaUiwPifUhHSmaM+YyaZnygoMIXHxQ3Fd/6XIgoxdo2O6gWPxUxQ8c D9GorrZVdzMVhOYPSoofAASuGhq9iTwibB2t/Mj35gi8+XAPECnP2y0IyLxO3A2HLyhv 87Sw8BhwoZToczPOA+gGeguRFuIiNLp3K87iWlpqCNzJvkDr0BwbSO//3+njKQdJDOxz AmCemAOp6LN1RyZAC6O0uO2jxiEDWi6vIW1Ca7DE0OBm8Hj8BksRTa6qPmYF3h8Kudh6 XpHQ== X-Received: by 10.52.78.35 with SMTP id y3mr6060073vdw.5.1428743665277; Sat, 11 Apr 2015 02:14:25 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.61.100 with HTTP; Sat, 11 Apr 2015 02:14:05 -0700 (PDT) In-Reply-To: <20150411084630.e424b59c1c327add498721e2@ulrich-grey.de> References: <1428718809.1182.31.camel@freebsd.org> <20150411081427.2f4fc14feb08cee948734d66@ulrich-grey.de> <20150411084630.e424b59c1c327add498721e2@ulrich-grey.de> From: Pratik Singhal Date: Sat, 11 Apr 2015 14:44:05 +0530 Message-ID: Subject: Re: pmap_demote_section: No l2_bucket for wired mapping panic To: Ulrich Grey Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-hackers , "freebsd-arm@freebsd.org" , Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 09:14:26 -0000 It seems that the issue is resolved for now, as many people suspected I was using the wrong kernel. I was making the changes in the different config file than the one from which the kernel was being compiled. I again recompiled the kenrel, this time correctly and the issue is resolved now. Regards, Pratik Singhal On Sat, Apr 11, 2015 at 2:16 PM, Ulrich Grey wrote: > > sysctl vm.pmap.pte1.promotions > > It's the figure "1" not the letter "l"! > ------------------------------------------ > On Sat, 11 Apr 2015 08:14:27 +0000 > Ulrich Grey wrote: > > > This kernel is running ARM_NEW_PMAP: > > > > root@wqtest:/usr/home/gwgpi # sysctl vm.pmap. > > vm.pmap.pv_entry_spare: 10379 > > vm.pmap.pv_entry_allocs: 1486593921 > > vm.pmap.pv_entry_frees: 1486577084 > > vm.pmap.pc_chunk_tryfail: 0 > > vm.pmap.pc_chunk_frees: 5598313 > > vm.pmap.pc_chunk_allocs: 5598394 > > vm.pmap.pc_chunk_count: 81 > > vm.pmap.pv_entry_count: 16837 > > vm.pmap.pte1.promotions: 3530 > > vm.pmap.pte1.p_failures: 20352 > > vm.pmap.pte1.mappings: 721715 > > vm.pmap.pte1.demotions: 508 > > vm.pmap.sp_enabled: 1 > > vm.pmap.nkpt2pg: 32 > > vm.pmap.shpgperproc: 200 > > vm.pmap.pv_entry_max: 1745184 > > root@wqtest:/usr/home/gwgpi # > > ---------------------------------- > > On Fri, 10 Apr 2015 20:20:09 -0600 > > Ian Lepore wrote: > > > > > Reply at the bottom, where it belongs. > > > > > > On Sat, 2015-04-11 at 06:58 +0530, Pratik Singhal wrote: > > > > I have double checked everything, recompiled kernel from scratch > imade sure > > > > that I ncluded the line "options ARM_NEW_PMAP" in the kernel > configuration > > > > file and I am still getting the same panic. > > > > > > > > Is there something at runtime that I can do to make sure that the > kernel I > > > > am running is compiled with ARM_NEW_PMAP ? > > > > > > > > > > > > > > > > On Sat, Apr 11, 2015 at 4:31 AM, Svatopluk Kraus > wrote: > > > > > > > > > The point of ARM_NEW_PMAP option is that there are no l2_buckets in > > > > > new armv6 pmap. So, as long as you get same panic, you are not > running > > > > > kernel compiled with ARM_NEW_PMAP option. > > > > > > > > > > Svatopluk Kraus > > > > > > > > > > > > > [...] > > > > > > > > > > > > From the announcement of the ARM_NEW_PMAP option... > > > > > > If you need to check whether the new or old code is running on > a > > > system, use "sysctl vm.pmap.pte1.promotions", if that gives an > > > "unknown oid" error you're running the old code. > > > > > > Other evidence: > > > > > > revolution > grep l2_bucket pmap-v6-new.c > > > revolution > > > > > > > You can't be getting a panic about l2_bucket stuff from code that > > > doesn't have anything in it about l2 buckets. Something must have gone > > > wrong with the way you built and installed the new kernel. > > > > > > -- Ian > > > > > > > > > _______________________________________________ > > > freebsd-arm@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > > > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-arm@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-arm@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > -- Regards, Pratik Singhal From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 11 08:15:13 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 014E198E; Sat, 11 Apr 2015 08:15:12 +0000 (UTC) Received: from mo6-p00-ob.smtp.rzone.de (mo6-p00-ob.smtp.rzone.de [IPv6:2a01:238:20a:202:5300::4]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.smtp.rzone.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 38301FFA; Sat, 11 Apr 2015 08:15:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1428740082; l=2306; s=domk; d=ulrich-grey.de; h=Content-Transfer-Encoding:Content-Type:Mime-Version:References: In-Reply-To:Subject:Cc:To:From:Date; bh=bV9NmMBMEQC18pJp5DEZGmFDQuSpd6Y8YNX1v4ctknw=; b=RI9A1E3nwoKhoI6JxSRlwKfJ/XPJYmRb141dwVpZgRVjgDn/sVibcXdY3Yp9Q7VJUBP F6LpBQk0C44ScYHS+MWjZLwpmFSfKzQ7h5FSrnNkNrSdsxhFB3jSvlqQvkBmZ+AcXXCGU HhWQDrXcXbX7HP2KaUnPTbpq8cq7LvbgSlo= X-RZG-AUTH: :OX8Be0W8W+pMC3rDLL/lo2xV/LZTbZkYhOcjg8suic3iYr/B8J9Lzp3TJg46scv/pkmS X-RZG-CLASS-ID: mo00 Received: from quad (p5486816B.dip0.t-ipconnect.de [84.134.129.107]) by smtp.strato.de (RZmta 37.5 DYNA|AUTH) with ESMTPSA id I010cfr3B8ESBEw (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (curve sect571r1 with 571 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Sat, 11 Apr 2015 10:14:28 +0200 (CEST) Date: Sat, 11 Apr 2015 08:14:27 +0000 From: Ulrich Grey To: Ian Lepore Subject: Re: pmap_demote_section: No l2_bucket for wired mapping panic Message-Id: <20150411081427.2f4fc14feb08cee948734d66@ulrich-grey.de> In-Reply-To: <1428718809.1182.31.camel@freebsd.org> References: <1428718809.1182.31.camel@freebsd.org> Organization: - X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.25; armv6-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 11 Apr 2015 11:31:28 +0000 Cc: freebsd-hackers , "freebsd-arm@freebsd.org" , Pratik Singhal X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 08:15:13 -0000 This kernel is running ARM_NEW_PMAP: root@wqtest:/usr/home/gwgpi # sysctl vm.pmap. vm.pmap.pv_entry_spare: 10379 vm.pmap.pv_entry_allocs: 1486593921 vm.pmap.pv_entry_frees: 1486577084 vm.pmap.pc_chunk_tryfail: 0 vm.pmap.pc_chunk_frees: 5598313 vm.pmap.pc_chunk_allocs: 5598394 vm.pmap.pc_chunk_count: 81 vm.pmap.pv_entry_count: 16837 vm.pmap.pte1.promotions: 3530 vm.pmap.pte1.p_failures: 20352 vm.pmap.pte1.mappings: 721715 vm.pmap.pte1.demotions: 508 vm.pmap.sp_enabled: 1 vm.pmap.nkpt2pg: 32 vm.pmap.shpgperproc: 200 vm.pmap.pv_entry_max: 1745184 root@wqtest:/usr/home/gwgpi # ---------------------------------- On Fri, 10 Apr 2015 20:20:09 -0600 Ian Lepore wrote: > Reply at the bottom, where it belongs. > > On Sat, 2015-04-11 at 06:58 +0530, Pratik Singhal wrote: > > I have double checked everything, recompiled kernel from scratch imade sure > > that I ncluded the line "options ARM_NEW_PMAP" in the kernel configuration > > file and I am still getting the same panic. > > > > Is there something at runtime that I can do to make sure that the kernel I > > am running is compiled with ARM_NEW_PMAP ? > > > > > > > > On Sat, Apr 11, 2015 at 4:31 AM, Svatopluk Kraus wrote: > > > > > The point of ARM_NEW_PMAP option is that there are no l2_buckets in > > > new armv6 pmap. So, as long as you get same panic, you are not running > > > kernel compiled with ARM_NEW_PMAP option. > > > > > > Svatopluk Kraus > > > > > > > [...] > > > > > > From the announcement of the ARM_NEW_PMAP option... > > If you need to check whether the new or old code is running on a > system, use "sysctl vm.pmap.pte1.promotions", if that gives an > "unknown oid" error you're running the old code. > > Other evidence: > > revolution > grep l2_bucket pmap-v6-new.c > revolution > > > You can't be getting a panic about l2_bucket stuff from code that > doesn't have anything in it about l2 buckets. Something must have gone > wrong with the way you built and installed the new kernel. > > -- Ian > > > _______________________________________________ > freebsd-arm@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 11 08:47:17 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90D33C61; Sat, 11 Apr 2015 08:47:17 +0000 (UTC) Received: from mo6-p00-ob.smtp.rzone.de (mo6-p00-ob.smtp.rzone.de [IPv6:2a01:238:20a:202:5300::8]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.smtp.rzone.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C616535E; Sat, 11 Apr 2015 08:47:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1428742007; l=2869; s=domk; d=ulrich-grey.de; h=Content-Transfer-Encoding:Content-Type:Mime-Version:References: In-Reply-To:Subject:Cc:To:From:Date; bh=x0W0hwx2w7DDUUoj29heI+WvHD7FhhJGBdmaxK1p9Dc=; b=tqw4aqwsKOubnxak9vdJzXABPapwPL27r02Syo8c3FcPF5adYeMnaf3PLjzEiCdhRHD AjAldnWU00j40b4PbSa257nDwA82N+otm7rKOaS15/t2chjkwGegWnn5GVRSMcESlfd/9 cosyRPIoGpaN1KH/8XEUoEkXEKtp8PfD9NA= X-RZG-AUTH: :OX8Be0W8W+pMC3rDLL/lo2xV/LZTbZkYhOcjg8suic3iYr/B8J9Lzp3TJg46scv/pkmS X-RZG-CLASS-ID: mo00 Received: from quad (p5486816B.dip0.t-ipconnect.de [84.134.129.107]) by smtp.strato.de (RZmta 37.5 DYNA|AUTH) with ESMTPSA id c06599r3B8kVAWa (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (curve sect571r1 with 571 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Sat, 11 Apr 2015 10:46:31 +0200 (CEST) Date: Sat, 11 Apr 2015 08:46:30 +0000 From: Ulrich Grey To: Ulrich Grey Subject: Re: pmap_demote_section: No l2_bucket for wired mapping panic Message-Id: <20150411084630.e424b59c1c327add498721e2@ulrich-grey.de> In-Reply-To: <20150411081427.2f4fc14feb08cee948734d66@ulrich-grey.de> References: <1428718809.1182.31.camel@freebsd.org> <20150411081427.2f4fc14feb08cee948734d66@ulrich-grey.de> Organization: - X-Mailer: Sylpheed 3.4.2 (GTK+ 2.24.25; armv6-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Sat, 11 Apr 2015 11:45:56 +0000 Cc: freebsd-hackers , "freebsd-arm@freebsd.org" , Ian Lepore X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 08:47:17 -0000 sysctl vm.pmap.pte1.promotions It's the figure "1" not the letter "l"! ------------------------------------------ On Sat, 11 Apr 2015 08:14:27 +0000 Ulrich Grey wrote: > This kernel is running ARM_NEW_PMAP: > > root@wqtest:/usr/home/gwgpi # sysctl vm.pmap. > vm.pmap.pv_entry_spare: 10379 > vm.pmap.pv_entry_allocs: 1486593921 > vm.pmap.pv_entry_frees: 1486577084 > vm.pmap.pc_chunk_tryfail: 0 > vm.pmap.pc_chunk_frees: 5598313 > vm.pmap.pc_chunk_allocs: 5598394 > vm.pmap.pc_chunk_count: 81 > vm.pmap.pv_entry_count: 16837 > vm.pmap.pte1.promotions: 3530 > vm.pmap.pte1.p_failures: 20352 > vm.pmap.pte1.mappings: 721715 > vm.pmap.pte1.demotions: 508 > vm.pmap.sp_enabled: 1 > vm.pmap.nkpt2pg: 32 > vm.pmap.shpgperproc: 200 > vm.pmap.pv_entry_max: 1745184 > root@wqtest:/usr/home/gwgpi # > ---------------------------------- > On Fri, 10 Apr 2015 20:20:09 -0600 > Ian Lepore wrote: > > > Reply at the bottom, where it belongs. > > > > On Sat, 2015-04-11 at 06:58 +0530, Pratik Singhal wrote: > > > I have double checked everything, recompiled kernel from scratch imade sure > > > that I ncluded the line "options ARM_NEW_PMAP" in the kernel configuration > > > file and I am still getting the same panic. > > > > > > Is there something at runtime that I can do to make sure that the kernel I > > > am running is compiled with ARM_NEW_PMAP ? > > > > > > > > > > > > On Sat, Apr 11, 2015 at 4:31 AM, Svatopluk Kraus wrote: > > > > > > > The point of ARM_NEW_PMAP option is that there are no l2_buckets in > > > > new armv6 pmap. So, as long as you get same panic, you are not running > > > > kernel compiled with ARM_NEW_PMAP option. > > > > > > > > Svatopluk Kraus > > > > > > > > > > [...] > > > > > > > > > From the announcement of the ARM_NEW_PMAP option... > > > > If you need to check whether the new or old code is running on a > > system, use "sysctl vm.pmap.pte1.promotions", if that gives an > > "unknown oid" error you're running the old code. > > > > Other evidence: > > > > revolution > grep l2_bucket pmap-v6-new.c > > revolution > > > > > You can't be getting a panic about l2_bucket stuff from code that > > doesn't have anything in it about l2 buckets. Something must have gone > > wrong with the way you built and installed the new kernel. > > > > -- Ian > > > > > > _______________________________________________ > > freebsd-arm@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-arm@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arm > To unsubscribe, send any mail to "freebsd-arm-unsubscribe@freebsd.org" From owner-freebsd-hackers@FreeBSD.ORG Sat Apr 11 22:05:36 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3E988847 for ; Sat, 11 Apr 2015 22:05:36 +0000 (UTC) Received: from puchar.net (puchar.net [188.252.31.250]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "puchar.net", Issuer "puchar.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B94B4D3D for ; Sat, 11 Apr 2015 22:05:35 +0000 (UTC) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.14.9/8.14.9) with ESMTP id t3BM5PUf061674 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sun, 12 Apr 2015 00:05:26 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from laptop.wojtek.intra (localhost [127.0.0.1]) by laptop.wojtek.intra (8.14.9/8.14.9) with ESMTP id t3BM5P5o001081 for ; Sun, 12 Apr 2015 00:05:25 +0200 (CEST) (envelope-from wojtek@puchar.net) Received: from localhost (wojtek@localhost) by laptop.wojtek.intra (8.14.9/8.14.9/Submit) with ESMTP id t3BM5Kwi001078 for ; Sun, 12 Apr 2015 00:05:20 +0200 (CEST) (envelope-from wojtek@puchar.net) X-Authentication-Warning: laptop.wojtek.intra: wojtek owned process doing -bs Date: Sun, 12 Apr 2015 00:05:20 +0200 (CEST) From: Wojciech Puchar X-X-Sender: wojtek@laptop.wojtek.intra To: freebsd-hackers@freebsd.org Subject: Help with 8TB Seagate Archive Message-ID: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (puchar.net [10.0.1.1]); Sun, 12 Apr 2015 00:05:26 +0200 (CEST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 22:05:36 -0000 got it and connected in logs i see (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 00 40 00 00 00 00 46 00 (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 (aprobe0:ahcich0:0:0:0): Retrying command (aprobe0:ahcich0:0:0:0): SETFEATURES SET TRANSFER MODE. ACB: ef 03 00 00 00 40 00 00 00 00 46 00 (aprobe0:ahcich0:0:0:0): CAM status: ATA Status Error (aprobe0:ahcich0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) (aprobe0:ahcich0:0:0:0): RES: 51 04 00 00 00 40 00 00 00 46 00 (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted and it doesn't connect. any ideas?