[Ksummit-2012-discuss] [ATTEND] writeback and kernel testing

Grant Likely grant.likely at secretlab.ca
Tue Jun 19 20:38:37 UTC 2012

On Sat, 16 Jun 2012 07:07:25 -0700, Guenter Roeck <linux at roeck-us.net> wrote:
> On Sat, Jun 16, 2012 at 08:44:18PM +0800, Fengguang Wu wrote:
> > Greetings,
> > 
> > I'd like to attend this year's kernel summit.
> > 
> > I may talk about the technical challenges and trade-offs on writeback and memcg
> > and IO controllers with anyone interested, perhaps in some breakout session.
> > 
> > And I would like a chance to talk about doing kernel tests in a timely fashion:
> > whenever one pushes new commits to git.kernel.org, build/boot/stress tests will
> > be kicked off and possible errors be notified back to the author within hours.
> > 
> > This fast develop-test feedback cycle is enabled by running a test
> > backend that is able to build 25000 kernels and runtime test 3000
> > kernels (assuming 10m boot+testing time for each kernel) each day.
> > Just capable enough to outrace our patch creation rate ;-)
> > 
> > On an average day, 1-2 build errors are caught in the 160 monitored kernel trees.
> > 
> > The runtime tests are still in active development and I'd like to ask for your
> > inputs on best practices and test methodology for every possible subsystems. The
> > target is to catch _common_ bugs early, so that a) less people are impacted and
> > b) when bisecting a specific bug, it's not confused by unrelated but common bugs.
> > 
> > I'll need your help and feedback to run this system well. In return, you'll be
> > able to take better advantage of it, once got some basic understandings on how
> > it runs for you. Hopefully, someday, these diligent machines may bring a little
> > happiness to our stressed life. As is the secret of happiness for us kernel
> > developers: if a bug is caught and fixed in my own tree, Cheers!  Otherwise if
> > the bug has been merged in the upstream tree, OMG, it's out of control and may
> > well impact 1000 commits after it.. regrets, sadness, guilty, embarrassments,
> > bad commits with my Signed-off-by carved in stone, forever ...
> > 
> I am running a nightly sequence of builds on my tree, for as many targets as
> possible. allyesconfig, allmodconfig, and a large number of randconfigs. That
> helps me find most of the problems I had early on, which only show up in some
> configurations. That combined with a personal rule to only push code upstream
> which has been in my local tree for at least one test cycle helps a lot to avoid
> the embarrassment of breaking linux-next or Linus' tree.
> I would love to be able to expand that to runtime tests, specifically for
> hardware monitoring functionality, but that is a bit more than I can afford
> and/or maintain on my own.

Actually, this should be doable *without* any special hardware or
setup.  QEMU has models for all the major architectures.  It would be
useful to have a tool in the kernel tree that builds and boots a set
of kernel configs on QEMU, assuming that the user has the relevant
cross compilers installed and that there is are a set of rootfs images
to use for each architecture.

If done right, a maintainer (or submitter) can fire off a command to
sanity-test their kernel tree before pushing it out to linux-next.
I've been thinking about this for a while now, but haven't had the
time to hack on it.


More information about the Ksummit-2012-discuss mailing list