From owner-soc-status@freebsd.org Mon Jun 25 18:49:02 2018 Return-Path: Delivered-To: soc-status@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0413A101AF27 for ; Mon, 25 Jun 2018 18:49:02 +0000 (UTC) (envelope-from aniket.ezio41@gmail.com) Received: from mail-ed1-f46.google.com (mail-ed1-f46.google.com [209.85.208.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 609018915F; Mon, 25 Jun 2018 18:49:01 +0000 (UTC) (envelope-from aniket.ezio41@gmail.com) Received: by mail-ed1-f46.google.com with SMTP id l23-v6so3374436edq.0; Mon, 25 Jun 2018 11:49:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=p89HhWtD3jOak5qtenbrWPY+UwJCadebFVBMPT7YGE8=; b=NPeH+/vZvBb+3Q1WNkcBfK8shKGYvov1fkCYaahzAsBcFHVS6sq/dz8TjJdJFsaUDF P2eZ0F6tc5VFTyHlPdh+y0CfojemWthpkaeAoKJId74OJNLSjwCu0/Zd6NY9KyQ3yjfz jqlJ24sGleCXfETZeFIBvRdIDpG9EzPGU9BUxQaA0KmHTbsL31J56/l/e7oRZRzJZNkc pg1uocft2x7R/dpUDtalFIS6xqIbyXA6VQI+exMhgnLwKpznPaxzNPuT1ocQQLf5EycR otzz3gnuP/o/NInrUhAK+3m4IK2AmZtq0PuwaRpsrJ9FHzJ/F2CS6KJGe9YTby2aoGXA p7cw== X-Gm-Message-State: APt69E3HCUJK4TvCP4LIET5BIJnjPY87yQxHIOKc/GSkroXL67NSA4qD EZceE+kH3yfhaLdXHemsnOa1Obg2 X-Google-Smtp-Source: ADUXVKIJK6UwkWCcjKFRuYLHVaHd3/Lb/EGpTAKji3iionkUBfV3odlqbbvUmI+RWk2lqoEhtAU2Ww== X-Received: by 2002:a50:cb0d:: with SMTP id g13-v6mr12232462edi.81.1529952058810; Mon, 25 Jun 2018 11:40:58 -0700 (PDT) Received: from mail-wr0-f175.google.com (mail-wr0-f175.google.com. [209.85.128.175]) by smtp.gmail.com with ESMTPSA id a12-v6sm5919112edr.89.2018.06.25.11.40.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 25 Jun 2018 11:40:58 -0700 (PDT) Received: by mail-wr0-f175.google.com with SMTP id a12-v6so14704987wro.1; Mon, 25 Jun 2018 11:40:58 -0700 (PDT) X-Received: by 2002:adf:a6ec:: with SMTP id t99-v6mr11013323wrc.51.1529952057888; Mon, 25 Jun 2018 11:40:57 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a5d:4146:0:0:0:0:0 with HTTP; Mon, 25 Jun 2018 11:40:57 -0700 (PDT) From: Aniket Pandey Date: Tue, 26 Jun 2018 00:10:57 +0530 X-Gmail-Original-Message-ID: Message-ID: Subject: [GSoC-18] Regression Test-Suite for Audit Framework [Week-6] To: soc-status@freebsd.org Cc: Alan Somers , George Neville-Neil , robert.watson@cl.cam.ac.uk Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2018 18:49:02 -0000 Hello All, This week, I primarily focused on getting the final bit of code merged in FreeBSD src and resolving the issues revealed therein. I also encountered a few interesting bugs and had to think out of the box to resolve them. 1) In the case of system calls concerned with manipulating SysV semaphore operations, there is an option to #define _WANT_SEMUN to access the semun union which is used for various commands of semctl(2). As defined in "sys/security/audit/audit_klib.c", each of those 'commands' is a unique audit event so it is recommended to test them all individually. But for some reason, the import of union semun wasn't working properly and it resulted in multiple incorrect forward declarations. To resolve the issue, I had to recompile the kernel from source as this change was introduced in r330299 [1] by brooks@ and apparently, I did not have the updated version of 12-CURRENT. 2) The wait4(2) issue with process control groups of system calls, about which I had discussed a few emails back, turned out to be a false setback. Usually, whenever I run my tests, I always have a clone of /dev/auditpipe open in a separate window to monitor the actual execution of system calls (Note: dtrace can also be used here). Now the auditpipe(4) device instance "waits" for syscalls to enter the execution context and thus, this extraneous wait4(2) tampers with tests by interfering with the syscall in concern. Although this could lead to a problem if someone has both the activities ongoing, however, Alan pointed out that these ATF tests would rarely be run in a production environment so that wouldn't affect the overall regression testing. 3) For setpgrp(2), I had to execute the tests from a forked child process as for some reason, Kyua denied modifying the process group ID of the main test-case process (with EPERM). 4) For some administrative syscalls like auditctl(2) and acct(2), I had to figure out how to get them to audit successfully without tampering with the system accounting records and the event auditing at /var/audit. For that, I followed the approach of confirming whether these utilities are enabled in the system or not. If so, then we might have to temporarily disrupt the system auditing/accounting by configuring a different file path to allow the tests to execute independently. Once done, restart the respective daemons. For acct(2), I had to use sysctlbyname(3) to get the accounting status as the normal system(3) utility outputs the exit status of the command ("ken.acct_configured" in our case) rather than the value in STDOUT. 5) I'm yet to figure out how to get mount(2) and nmount(2) to execute successfully. I tried mounting some dummy filesystems like nullfs, tmpfs, and even UFS. But all efforts were in vain. I'll try some workarounds for this issue. If anybody has any suggestions in this regard, I'll be glad to give it a try! Apart from the issues mentioned above, the week was productive in general. Almost all of my proposed work is now either in FreeBSD Head or is accepted and waiting to be landed. Few miscellaneous syscalls' tests remain, and I'll give them a run this week. Also, I've updated the documentation of my project repository [2]. Since the tests won't be in FreeBSD 11 Stable, I'll try to create a custom installation script for users who want to run the tests in the older stable version. Thank you, With best regards, Aniket Pandey [1] https://reviews.freebsd.org/rS330299 [2] https://github.com/aniketp/AuditTestSuite Project wiki: https://wiki.freebsd.org/SummerOfCode2018Projects/RegressionTestSuiteForAuditFramework From owner-soc-status@freebsd.org Mon Jun 25 19:18:08 2018 Return-Path: Delivered-To: soc-status@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6EAF101BF23 for ; Mon, 25 Jun 2018 19:18:07 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-x243.google.com (mail-lj1-x243.google.com [IPv6:2a00:1450:4864:20::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 357398A201; Mon, 25 Jun 2018 19:18:07 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-x243.google.com with SMTP id u6-v6so3750611lju.13; Mon, 25 Jun 2018 12:18:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=R6qy28xfibe9D9M54L8vJb3U/wqsORk4weLu/YCIfSc=; b=A5EPRh+yzo/9vS3ILIS+qYeOLs9OPDQk91+yUlP0vhiIzYjMfLCc+eMPm+zjUEO2GR NZOOOeUAOuak8Wip60IoPutFf0iVw63Sg8/mzmUiU5Xg77OmBc6kASC0QqByY4UIhufl J04WwoQRsdVMjTajqouOL9qZ1QrV7t2uXZkcvvfQoCUggCjfdtml67UFwBxvIGNLh/x+ JdTryqQ+2gLgk89XJn9PgAScrpz2Cti6X65OfUVkpb6MUd2+wMgECHuHYfowvg+iJIYw CuedVEp5MdD76zgmhtehqwHYdparK7zT/7GRKMAY6UjPX7aZUQt8dTXMAQfB9WGhg2ca sT4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=R6qy28xfibe9D9M54L8vJb3U/wqsORk4weLu/YCIfSc=; b=aiF/oVCtzRmK1qHuUxzImfP31zj5iCE0QvLSUH48NkxPpjfhAjojWaYt79pf79XeUm La2SlTqrMHBDX+SO3PZGdIRXLyGjS+nWRZWWaV6avLcQ/e9IpD7m3EGI/A7Eyj5xCyKz q7Wg4ApkVERbNCxiGgBFScYJvv0m8jInlSfwvq3pFBgp1Qmy+1kKmdnIZa7/THHRbmF4 RbBZiB/ixqR2KjG/TNd+FJOES2ipT6bNmE0lVv3JpWV0AL/ziEJi/VwPP67UGau2qcJ/ CAr6/VMYGs+AHlb1T8/cL38yYSALVq27P+/ZRexpiNh455UTHQ/924J48fpvT6poRdcm 9pxQ== X-Gm-Message-State: APt69E22RQFnlccLCRffoN5PH9O4YKRA3gXW315BtL1XIplB/j9UHOR6 k9PBtxJZRTWVslECe657doH5kfUSL1y1BKaBLjc= X-Google-Smtp-Source: ADUXVKILapUN0JZBzXgu5isGFzbkClpOYQOK75xjPcGiBnZvgjaE6a57Hdf9avkE50OKbxQg2/Q+KraGmrfIDBlArbA= X-Received: by 2002:a2e:6d11:: with SMTP id i17-v6mr8747745ljc.116.1529954285350; Mon, 25 Jun 2018 12:18:05 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:1b91:0:0:0:0:0 with HTTP; Mon, 25 Jun 2018 12:18:04 -0700 (PDT) In-Reply-To: References: From: Alan Somers Date: Mon, 25 Jun 2018 13:18:04 -0600 X-Google-Sender-Auth: twTx4-t26iT2gr4h7-RZ7inakeo Message-ID: Subject: Re: [GSoC-18] Regression Test-Suite for Audit Framework [Week-6] To: Aniket Pandey Cc: soc-status@freebsd.org, George Neville-Neil , Robert Watson Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2018 19:18:08 -0000 On Mon, Jun 25, 2018 at 12:40 PM, Aniket Pandey wrote: > Hello All, > > This week, I primarily focused on getting the final bit of code merged in > FreeBSD src and resolving the issues revealed therein. I also encountered a > few interesting bugs and had to think out of the box to resolve them. > > 1) In the case of system calls concerned with manipulating SysV semaphore > operations, there is an option to #define _WANT_SEMUN to access the > semun union which is used for various commands of semctl(2). As defined in > "sys/security/audit/audit_klib.c", each of those 'commands' is a unique > audit event so it is recommended to test them all individually. But for > some reason, the import of union semun wasn't working properly and it > resulted in multiple incorrect forward declarations. To resolve the issue, > I had to recompile the kernel from source as this change was introduced > in r330299 [1] by brooks@ and apparently, I did not have the updated > version of 12-CURRENT. > Recompiling the kernel can't have helped. Brooks' change doesn't EXPOSE semun if you DO define _WANT_SEMUN. Rather, it HIDES semun if you DON"T define _WANT_SEMUN. Something else must've been the problem. > > 2) The wait4(2) issue with process control groups of system calls, about > which I had discussed a few emails back, turned out to be a false setback. > Usually, whenever I run my tests, I always have a clone of /dev/auditpipe > open in a separate window to monitor the actual execution of system calls > (Note: dtrace can also be used here). Now the auditpipe(4) device instance > "waits" for syscalls to enter the execution context and thus, this > extraneous wait4(2) tampers with tests by interfering with the syscall in > concern. Although this could lead to a problem if someone has both the > activities ongoing, however, Alan pointed out that these ATF tests would > rarely be run in a production environment so that wouldn't affect the > overall regression testing. > So the problem is that your test was completing or advancing too soon, because it was matching an event generated from outside of Kyua? That sounds like a bug, and a very difficult one to debug. You should add more details to the regex to ensure that it doesn't happen again. BTW, auditpipe(4) can't "wait" for syscalls. Only userland processes can call "wait" or its variants. And wait doesn't wait for syscalls; it waits for processes to change their running state. > > 3) For setpgrp(2), I had to execute the tests from a forked child process > as for some reason, Kyua denied modifying the process group ID of the main > test-case process (with EPERM). > > 4) For some administrative syscalls like auditctl(2) and acct(2), I had to > figure out how to get them to audit successfully without tampering with the > system accounting records and the event auditing at /var/audit. For that, I > followed the approach of confirming whether these utilities are enabled in > the system or not. If so, then we might have to temporarily disrupt the > system auditing/accounting by configuring a different file path to allow > the tests to execute independently. Once done, restart the respective > daemons. For acct(2), I had to use sysctlbyname(3) to get the accounting > status as the normal system(3) utility outputs the exit status of the > command ("ken.acct_configured" in our case) rather than the value in > STDOUT. > sysctlbyname is also far faster than system(3). > > 5) I'm yet to figure out how to get mount(2) and nmount(2) to execute > successfully. I tried mounting some dummy filesystems like nullfs, tmpfs, > and even UFS. But all efforts were in vain. I'll try some workarounds for > this issue. If anybody has any suggestions in this regard, I'll be glad to > give it a try! > > Apart from the issues mentioned above, the week was productive in general. > Almost all of my proposed work is now either in FreeBSD Head or is accepted > and waiting to be landed. Few miscellaneous syscalls' tests remain, and > I'll give them a run this week. > > Also, I've updated the documentation of my project repository [2]. Since > the tests won't be in FreeBSD 11 Stable, I'll try to create a custom > installation script for users who want to run the tests in the older stable > version. > This is probably more work than it's worth. It's likely that some of your tests won't work on 11, because of changes to the syscalls involved. If you really want to run your tests on stable/11, then we should just go through the normal MFC process. I can show you how to do that. > > Thank you, > With best regards, > Aniket Pandey > > [1] https://reviews.freebsd.org/rS330299 > [2] https://github.com/aniketp/AuditTestSuite > Project wiki: > https://wiki.freebsd.org/SummerOfCode2018Projects/ > RegressionTestSuiteForAuditFramework > > From owner-soc-status@freebsd.org Mon Jun 25 22:54:26 2018 Return-Path: Delivered-To: soc-status@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 293D410240D9 for ; Mon, 25 Jun 2018 22:54:26 +0000 (UTC) (envelope-from duostefano93@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9FDE571A98 for ; Mon, 25 Jun 2018 22:54:25 +0000 (UTC) (envelope-from duostefano93@gmail.com) Received: by mail-it0-x236.google.com with SMTP id u4-v6so14907743itg.0 for ; Mon, 25 Jun 2018 15:54:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=2/nqdXJuh4yTyrCrtsM/0uvlzn/i2O/E2KNdX1L7z40=; b=CbuFYxX9L0PQ8FDXshp/YFUzK3q0+KWI4mLUEkAp8gA+XQWxviv6iimcS2W9I/24E+ 78tpPRinUuAgo/tgdlFyaR/uF8B8NgR7PRlcEz+V3hIAh9RskK0bhxKiY5gbM6ZQngrU qHPJOTal6YXF/XGmYFkHJ1Tg4xkFuvKB/M1QDpRd3ylMHrdtlBx9bugiqgukh7XLUvgl 6f0Dr819V0qcOPFv7v8r8I4a4DIGr4CCqKcmSLEvCpXemxTCOWWchjhHRNz55a2MqlFz UhfuYsX2T34fEidi6oe2en5B38PSJonryN4iT+t/e940V5Tw/oXwjNkBhxtRIeAPchB+ qBjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=2/nqdXJuh4yTyrCrtsM/0uvlzn/i2O/E2KNdX1L7z40=; b=Y0yNfXNg3vgIQ2ZC3d3hlamuEjT9CLiYcPz6HFYYohtDT/sbSWAzWtiaEcc5DXP94A +CY0jJhtIHejs/aX7dujUivNp8FeQAIpszBlkHCLd4hERfh8bTq7/Zb0nmPRURFGozNP sU6/sugnA7/qdyJqNLbf133EaRUdIP0KdWmMbUVR26eJ/9gNXGdQ75EvqDow166hr7D4 ruC7OE4KD/jibGgERuxSufDpejSyQphL2PJBhn/W6IA0SjCIVn9bKwbUpLdAfGoxp/Jb p/kp1zeCcBDJsEz9dLAHYMYKXglHryri231SJxn8gURusdC0EdHz8laCCK2ruMMvdt1F jSgg== X-Gm-Message-State: APt69E0lTP+f9hN1cWTcgOw9lb2wq6l0JI5rQG60Sl8UwOwVv8VwAfHT Ugox213ygCP/h+2TL/ydTEN/4++0/tqsD40JELabLA== X-Google-Smtp-Source: AAOMgpe+42NM1VNv6n+hbOdk2bANjTONkEDjcojRKWNDMKBucTzMwpJZj7V7kiwmRhNYo9eeAu8JHnCm1iiSsjofrKo= X-Received: by 2002:a02:4142:: with SMTP id x63-v6mr11683979jaa.46.1529967264775; Mon, 25 Jun 2018 15:54:24 -0700 (PDT) MIME-Version: 1.0 From: Stefano Duo Date: Tue, 26 Jun 2018 00:54:16 +0200 Message-ID: Subject: Extensions to the netmap framework - Week 6 To: soc-status@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2018 22:54:26 -0000 Hi, during the sixth i extended functional.c, implementing the following: - netmap interface file descriptors can be requested from the fd_server i wrote the previous week. - the fd_server can be started (and stopped) through cli arguments. When it is active, it acts as a daemon, logging stuff to /var/log/syslog. - source and destination MAC addresses of the frame can be specified during a send and receive action. - multiple frames can be sent (and received) though a single ioctl(). - when sending (or receiving) multiple frame, it's possible to change the fill character, following a deterministic pattern, after each send (or receive). This way it's possible to check that frames are receive in-order. Project wiki: https://wiki.freebsd.org/SummerOfCode2018Projects/ExtensionsToNetmap SVN repository: https://svnweb.freebsd.org/socsvn/soc2018/sduo/ Stefano. From owner-soc-status@freebsd.org Mon Jun 25 23:59:38 2018 Return-Path: Delivered-To: soc-status@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B70631025A8A for ; Mon, 25 Jun 2018 23:59:37 +0000 (UTC) (envelope-from lakhanshiva@gmail.com) Received: from mail-ot0-f194.google.com (mail-ot0-f194.google.com [74.125.82.194]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3887F73437 for ; Mon, 25 Jun 2018 23:59:37 +0000 (UTC) (envelope-from lakhanshiva@gmail.com) Received: by mail-ot0-f194.google.com with SMTP id d19-v6so17052113oti.8 for ; Mon, 25 Jun 2018 16:59:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=5y4LJyuY6ss1irr1inQXlQcIuBVorEE5QNBZ5XrP4HE=; b=SKq8eVagZPOw2KFgdY0rp/3sWyeaVoqPywZiqa4ZUmIrcZxDKtmGrDlJeFTPa5uXNf ygiaVT/USJSO8KGHRcMAStIXFp72kg5XG+T0ymz7DqfbxCjmO0u886Ar0VDTKkVWoCvy Nu4sapZYRczueBLyL6EcJ093RcIxYOUa7BVQm30CGSKnTMWosqHKScuiWLeLqCcpa3HC JQlnW6Yo7866+/sRuFCRO+UuT8plBXvHLGLJUu7Q2gbAJ+tFnHOh54oHRBkIr2mLPrtc YStD9IrqreaCvsUsuYjtZiMLT4wA0bsM1Rgu+eaz1aPet3+HEY3oVjrc81WN5BLSHA9r fS5A== X-Gm-Message-State: APt69E2i/w5VE0XpKZsqmJHgJOSN00FlP7bChfWBeoaLtVF3uKK0B8wO kYo3Xy6hrMfu35ZTpbbvDgPBtX4y X-Google-Smtp-Source: AAOMgpfIoIcbGF42unt7HQce7ZPHAxVW7z+Do0m9WkDc/cgb3zQjNWfgUDP1UP1RTsIGLnWEmiXmGQ== X-Received: by 2002:a9d:74a:: with SMTP id 68-v6mr95036ote.43.1529969744504; Mon, 25 Jun 2018 16:35:44 -0700 (PDT) Received: from mail-ot0-f170.google.com (mail-ot0-f170.google.com. [74.125.82.170]) by smtp.gmail.com with ESMTPSA id g8-v6sm208208otc.68.2018.06.25.16.35.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 25 Jun 2018 16:35:44 -0700 (PDT) Received: by mail-ot0-f170.google.com with SMTP id i19-v6so17010834otk.10 for ; Mon, 25 Jun 2018 16:35:43 -0700 (PDT) X-Received: by 2002:a9d:10e:: with SMTP id 14-v6mr5785345otu.330.1529969743829; Mon, 25 Jun 2018 16:35:43 -0700 (PDT) MIME-Version: 1.0 From: Lakhan Shiva Kamireddy Date: Tue, 26 Jun 2018 05:05:32 +0530 X-Gmail-Original-Message-ID: Message-ID: Subject: [GSOC-18] Convert PCI drivers to be table driven - Week 6 To: soc-status@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Jun 2018 23:59:38 -0000 Hi All, As you are aware, my project is to convert PCI drivers to be table driven. This week I have worked on other PCI drivers. I converted imcsb, isab, ignore, ix, ixv, ixgb, ips, iwm, iir PCI drivers. I worked with hard drivers as well. [1] -[3] I have added the Plug and Play information for them and found them to be used by devmatch to match them with modules automatically. I resolved all build issues. We have migrated the reviews to Phabricator and see that it is pretty cool. I was having a problem with updating my differential review. The process gets frozen on a step called querydiffs. We contacted the Phabricator team on FreeBSD side. Hopefully we will get to some solution eventually. Phabricator reviews can be found here. [4]-[6] We need to convert the drivers in a certain way so that kldxref can find it and record it in linker hints. I am working on PCI drivers so as to make it easy for kldxref to find them. In a couple of drivers I have added tables and modified the probe function. At the same time, I am also learning the internals of the kernel, this will help our goal ultimately - Automatic module loading. [7] Please find my updated project homepage here. [8] Thank you, Lakhan [1] https://github.com/bsdimp/freebsd/pull/4 [2] https://github.com/bsdimp/freebsd/pull/6 [3] https://github.com/bsdimp/freebsd/pull/7 [4] https://reviews.freebsd.org/D15995 [5] https://reviews.freebsd.org/D15996 [6] https://reviews.freebsd.org/D15979 [7] http://bsdimp.blogspot.in/2016/01/details-on-coming-automatic-module.html [8] https://wiki.freebsd.org/SummerOfCode2018Projects/ConvertPCIdriverAttachmentsToTables From owner-soc-status@freebsd.org Fri Jun 29 01:49:17 2018 Return-Path: Delivered-To: soc-status@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68AF8102F370 for ; Fri, 29 Jun 2018 01:49:17 +0000 (UTC) (envelope-from mateus@mateus.tech) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-bl2nam02on0126.outbound.protection.outlook.com [104.47.38.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8864E776A5; Fri, 29 Jun 2018 01:49:16 +0000 (UTC) (envelope-from mateus@mateus.tech) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mateusrodrigueslive.onmicrosoft.com; s=selector1-mateus-tech; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qNzId1VqmoUkiu2DUYZdQxklgLztDEDE8LOiciTe7KQ=; b=Bisqi+wPyvyZTodIyLEQmgJ9o4yiJGnrFdcfIc6YA1KpHgsUIFMrBv+gJ200mVH7EmlFHTVsoUI477gMcgRD18x4pt+iL31eY4qp7NqbO3s9ciEKd6t8A3GYRcPQXLpY2aP/+2YjakDDTiRAsezB/rZYBE2xbsbKcM1J2lRXGRI= Received: from CP2P152MB1124.LAMP152.PROD.OUTLOOK.COM (10.171.53.11) by CP2P152MB0193.LAMP152.PROD.OUTLOOK.COM (10.171.24.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.906.23; Fri, 29 Jun 2018 01:49:10 +0000 Received: from CP2P152MB1124.LAMP152.PROD.OUTLOOK.COM ([fe80::e956:3e1f:d21d:e054]) by CP2P152MB1124.LAMP152.PROD.OUTLOOK.COM ([fe80::e956:3e1f:d21d:e054%8]) with mapi id 15.20.0906.023; Fri, 29 Jun 2018 01:49:10 +0000 From: Mateus Rodrigues de Morais To: "soc-status@freebsd.org" CC: David Naylor , "theraven@freebsd.org" Subject: [Week 6] Porting Microsoft's CoreCLR, CoreFX, and PowerShell to FreeBSD Thread-Topic: [Week 6] Porting Microsoft's CoreCLR, CoreFX, and PowerShell to FreeBSD Thread-Index: AQHUD0lgPEiJx3g1JUid5Osca9HFyA== Date: Fri, 29 Jun 2018 01:49:10 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=mateus@mateus.tech; x-originating-ip: [177.89.43.129] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; CP2P152MB0193; 7:/rMcLaissJnik/XGAIMITqV3ASkuO7n3shIJJORXtyTVZhLUkiIwqDfO5QhHYxkABdVuXXk9s57VSk+i1bVt+xvEnXPmoUJmFmAbmEIL0KRBCeyEdNYf+FHlwDNjms0bGC2dEDPCZkQUSQ8dHLUT2ZjmgiBoZ5HgP/ox2ZBGhcfkh+4EvQriPfQNdWsWr2NF9hqkQKNbQJXwaOw+jXFpIz54dEGkOpLPzmOqgKztKO0ak7p513VlcklIwAN4Fr9Y x-ms-exchange-antispam-srfa-diagnostics: SOS; x-ms-office365-filtering-correlation-id: e072bcdd-70e2-48f3-1d63-08d5dd628263 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652037)(7021125)(8989117)(4534165)(7022125)(4603075)(4627221)(201702281549075)(8990107)(7048125)(7024125)(7027125)(7028125)(7023125)(5600026)(711020)(2017052603328)(7153060)(7193020); SRVR:CP2P152MB0193; x-ms-traffictypediagnostic: CP2P152MB0193: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(166708455590820)(21748063052155); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(3231254)(2017060905117)(944501410)(52105095)(10201501046)(3002001)(93006095)(93001095)(149027)(150027)(6041310)(20161123564045)(2016111802025)(20161123560045)(20161123558120)(20161123562045)(6072148)(6043046)(201708071742011)(7699016); SRVR:CP2P152MB0193; BCL:0; PCL:0; RULEID:; SRVR:CP2P152MB0193; x-forefront-prvs: 0718908305 x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(376002)(366004)(136003)(39830400003)(396003)(346002)(47530400004)(53754006)(199004)(189003)(45080400002)(86362001)(97736004)(2906002)(81166006)(8676002)(26005)(7736002)(8936002)(14454004)(99286004)(81156014)(102836004)(476003)(53936002)(54906003)(4326008)(478600001)(74316002)(186003)(966005)(486006)(7696005)(316002)(5640700003)(2351001)(5250100002)(106356001)(450100002)(52230400001)(6916009)(54896002)(2501003)(6306002)(2900100001)(6116002)(3846002)(68736007)(25786009)(105586002)(14444005)(256004)(33656002)(9686003)(6436002)(5660300001)(66066001)(55016002)(6506007); DIR:OUT; SFP:1102; SCL:1; SRVR:CP2P152MB0193; H:CP2P152MB1124.LAMP152.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: mateus.tech does not designate permitted sender hosts) x-microsoft-antispam-message-info: Ar9gOPpoKjs+Jct15wcWu5Uh/QTUfibmI4WQUc08OsMrxUz8wILRrMj99RNfIkj7VthB2ri14aH/CfASzZvdS6lFM/aIg+mRYfKODjaUl8ktyTasCJ6sfydCRnoAiAL16rujG3LRZ6ZdBJz2WqMMIzLmfCSLiMKJDRASvhGbM3yJ2HGk0wJUMWnpeM1SAfbvSgTWhl3eGCTLBmY9JyVvpHNETUmvHProa7trF46iNluhq6neEEbZ16hu6ohWvUnwpDA43YZl6ubEMLjAA8+KYCxOPVLHxagLHEyEqgVG6qXiFPYHfTBfPTA4ppxqucie/yao1BwK8+Ni4u3mgHNEQFs//H+bBrYjGiHzydv6r84= spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-OriginatorOrg: mateus.tech X-MS-Exchange-CrossTenant-Network-Message-Id: e072bcdd-70e2-48f3-1d63-08d5dd628263 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Jun 2018 01:49:10.5989 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 3cb2d6d7-8d6e-44b8-954d-3085729d2030 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CP2P152MB0193 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: soc-status@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Summer of Code Status Reports and Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2018 01:49:17 -0000 Hello all, During this 6th week of work, I started working on getting a PowerShell bui= ld on FreeBSD. As I researched how its code is organized, I found out that = on Linux and macOS, PowerShell Core relies on a native layer called libpsl = that provides missing functionality from .NET Core as kernel system calls. = As a result, this native layer also had to be built and ported over to Free= BSD. No code changes were necessary to get it built, however, out of the 32= unit tests, 3 were not passing. I forked the PowerShell repo over to my GitHub account [1], fixed some code= to work on FreeBSD and, as of right now, 2 out of these 3 tests are now pa= ssing. I=92m working on the last one. PowerShell itself is written in C# and runs on top of .NET. Therefore, I us= ed the bootstrapped SDK from previous weeks to build it on FreeBSD and the = steps [2] complete successfully. However, I see some misbehaviors while run= ning PowerShell that I believe to be caused by managed code and I=92ll inve= stigate those further as soon as I get done with the native tests. As of the CoreFX tests from last week, I=92m still on the process of invest= igating those as well. Maybe some of them relate to the PowerShell misbehav= iors or maybe they don=92t. [1] https://github.com/mateusrodrigues/PowerShell [2] https://github.com/mateusrodrigues/gsoc18-progress/tree/master/powershe= ll Best, -- Mateus de Morais