Date: Mon, 21 May 2018 03:51:34 +0530 From: Aniket Pandey <aniketp@freebsd.org> To: soc-status@freebsd.org Cc: asomers@freebsd.org, robert.watson@cl.cam.ac.uk, gnn@neville-neil.com, gavin@freebsd.org Subject: [GSoC-18] Regression Test-Suite for Audit Framework [Week-1] Message-ID: <CADsqe1B78opDbReLB3MfcnZDhCJhRM9c6bfjp4zBhsAUOTkjJw@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello All,
This project aims to develop a set of atf-c(3) test-cases for FreeBSD's
audit subsystem such that it will be easier to automate the testing of
Audit framework along with the entire operating system. The automation will
be done by Kyua(1).
I've been working on this project from late January and have made a
considerable amount of progress since then. I'll list out the important
highlights, breakthroughs and issues I've encountered along the way.
* Initial attempt was to create a set of source files which would contain
basic implementation of all common and recurring system calls in the audit
records. Then I would develop an automation script which would check the
presence of the system call along with the arguments, attributes, return
status and possibly some other information. However, this approach was
deprecated since FreeBSD already had Kyua(1) which could do the automation
for me and had some really nice utility functions available.
* On recommendation of Kristof and Alan, I switched to creating atf-c(3)
test-cases for all system calls in various scenarios. However, considering
the vast number of OpenBSM supported system calls and for the scope of the
projectc, I planned on testing "success" and "failure" modes of each system
call to ensure greater coverage.
* Each test case would be independent, would launch auditd(8) if needed,
would set local mode auditing using the defined preselection-parameters,
check auditpipe(4) if it emitted the system call, and then conclude the
tests. Detailed implementation details can be found in the Project Wiki.
* Test-Suite organisation is to have one test-program per audit class, and
have all its audit events within. (e.g file-create.c,
file-attribute-access.c etc)
Milestones:
---------------
So far, I've been able to create 306 ATF test cases spanning over 6
test-programs for 62 system calls of ("fc", "fw", "fr", "fd", "fm", "fa"
audit classes) and 5875 SLOC.
The tests are currently passing and there general output can be seen here:
https://pastebin.com/XHfAP6nH
Bugs/Issues encountered:
----------------------------------
* Initially we had trouble with getting auditd(8) to properly start and
basically it was creating a race condition within the test-cases. To
resolve this, we had to poll(2) for "audit startup" and once it was
confirmed, proceed with normal syscall testing.
* After introducing the audit startup check, we encountered another problem
with one of the libbsm(3) APIs, au_read_rec(3) which for some reason,
returned corrupted records when checking for successful audit startup as a
lot of other events were being generated too. To resolve this, we had to
explicitly set the audit mask for local mode auditing as "ad" (for audit
startup)
* On trying to run the tests in FreeBSD-12 CURRENT when the auditd(8) is
not already running, we encountered a reproducible kernel panic (
https://pastebin.com/u3DqGTS1), which was fixed by Alan in D15381
<https://reviews.freebsd.org/D15381>.
* Some system calls like "lgetfh(2)" and "chflagsat(2)" were not being
audited while syscalls like "getfh(2)" were audited as "nfs_getfh(2)".
I've created a differential revision D15286
<https://reviews.freebsd.org/D15286>, which introduces the initial set of
tests (24) for audit system. It is currently under review.
I'll keep adding more test cases in the next week and hope to finish the
remaining File-I/O audit events and complete tests for socket API.
Bugs Reported
--------------------
1) https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228374
2) https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226713
Differential Revisions
----------------------------
1) https://reviews.freebsd.org/D15286
2) https://reviews.freebsd.org/D15381 (@asomers)
ProjectWiki:
https://wiki.freebsd.org/SummerOfCode2018Projects/RegressionTestSuiteForAuditFramework
TestSuite Repo: https://github.com/aniketp/AuditTestSuite
Socsvn branch: https://svnweb.freebsd.org/socsvn/soc2018/aniketp/head/
Thank You,
With Best Regards,
Aniket Pandey
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CADsqe1B78opDbReLB3MfcnZDhCJhRM9c6bfjp4zBhsAUOTkjJw>