From owner-freebsd-hackers@freebsd.org Sat Oct 1 18:02:20 2016 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47062C0AC6E for ; Sat, 1 Oct 2016 18:02:20 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 15B8A6ACDF for ; Sat, 1 Oct 2016 18:02:19 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 32BCB2A028; Sat, 1 Oct 2016 20:02:09 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id BNFRXc3NzlDD; Sat, 1 Oct 2016 20:02:08 +0200 (CEST) Received: from [192.168.10.10] (asus [192.168.10.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 6D9D22A027 for ; Sat, 1 Oct 2016 20:02:08 +0200 (CEST) To: FreeBSD Hackers From: Willem Jan Withagen Subject: Kqueue and threading Message-ID: <111e0c35-7a4b-b6c7-ef1d-1a0d85112e61@digiware.nl> Date: Sat, 1 Oct 2016 20:02:09 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Oct 2016 18:02:20 -0000 Hi, Ceph uses a lot of threading, and for any part of it communication it uses epoll(), which FreeBSD does not use. For that there was already a EvenKqueue implementation. But I think I'm now running into: The kqueue() system call creates a new kernel event queue and returns a descriptor. The queue is not inherited by a child created with fork(2). However, if rfork(2) is called without the RFFDG flag, then the descriptor table is shared, which will allow sharing of the kqueue between two processes. Kqueue descriptors are created and events are added, but then the process starts other threads and expects the kqueue-id to be valid there. However adding more events returns an error, also waiting on the ID for events to happen returns an error (descriptor invalid) Threading is done with 2 different constructions: std::thread and creating Workers Would any of these qualify with the quoted warning? and invalidate the descriptor table? If not, how can I (easily) debug the reason why my descriptors go invalid? thanx, --WjW