Date: Thu, 13 Jul 2017 17:38:34 +0000 From: Shivansh Rai <shivansh@freebsd.org> To: "soc-status@freebsd.org" <soc-status@freebsd.org> Cc: Alan Somers <asomers@freebsd.org>, Brooks Davis <brooks@freebsd.org>, "Ngie Cooper (yaneurabeya)" <yaneurabeya@gmail.com> Subject: [GSOC17] Smoke testing of all base utilities - Week 6 Message-ID: <CAF%2Bp1HujTMjm8Wg=p3=wR%2BuAK5Ar_ViVefuZStfSTbFjgCaW9w@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello all, Sending an early progress report to cover up the lag in the weekly schedule= . Previous 4 days were spent on polishing some corner cases in the automation tool. A summary of updates can be viewed here [1]. To demonstrate the tests which the tool can generate, the utilities date(1), ln(1) and stdbuf(1) were used. These tests are available here [2]. The following table summarizes the type of test-cases produced for these 3 utilities. +----------------+-----------------+-----------------+ | Test | (+ve) testcases | (-ve) testcases | +----------------+-----------------+-----------------+ | date_test.sh | 5 | 2 | +----------------+-----------------+-----------------+ | ln_test.sh | 0 | 11 | +----------------+-----------------+-----------------+ | stdbuf_test.sh | 1 | 0 | +----------------+-----------------+-----------------+ Some key-points worth noting : 1.) date_test.sh is time/timezone dependent, hence all the test-cases apart from invalid_usage will fail. The aim for using this utility is to demonstrate that the tool can generate positive test-cases along with negative ones. It also demonstrates the need to sanity check the tests. 2.) ln_test.sh demonstrates that if the utility produces a common usage message for every invalid usage, then the tool can define a variable containing this message and avoid duplication while generating the test-cases. Also, all the negative tests are collated under a single testcase named "invalid_usage". 3.) The groff script for stdbuf(1) is slightly different than others, hence the tool couldn=E2=80=99t generate any negative test-case in stdbuf_test.sh= as it couldn't pick up the supported options. This scenario will be taken into account next. The aim for using this utility is to demonstrate that the tool will always produce atleast one positive/negative testcase. Some updates were also made to the documentation [3]. The project wiki will be updated next. The plan for this week is to start working on generating tests for all the utilities which currently don=E2=80=99t have any test coverage. The automat= ion tool will be worked upon and improved during the process. As always, suggestions are most welcome! [1]: https://github.com/shivrai/smoketestsuite/compare/1a7e26e...2b3e2a5 [2]: https://github.com/shivrai/smoketestsuite/tree/master/tool/generated_tests [3]: https://github.com/shivrai/smoketestsuite/blob/master/README.md Thank you. With best regards, Shivansh Rai =E2=80=8B
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAF%2Bp1HujTMjm8Wg=p3=wR%2BuAK5Ar_ViVefuZStfSTbFjgCaW9w>