Ensuring quality of OSDL releases

Overview

This document concentrates on some common quality rules which should be enforced by the OSDL project, on a best-effort basis.

Some coding conventions

Some of them can be found in Ceylan's coding conventions (some parts in French).

Testing

We do our best to torture-test, as far as possible, the whole LOANI's tool chain before releasing a new version. With this test suite, both the auto-installation script LOANI and the Ceylan and OSDL C++ libraries are validated, thanks to the numerous executables tests that are built alongside.

We run through the whole process on the following hardware and software configurations, which happen to be varied and, we hope, representative of the users' configurations (various technological generations, processors, video cards, kernels, and distributions):

Test configurations
Description Processor Video card OS & kernel version Distribution / Version
Very Old PC Intel PII 300 MHz ATI Rage 128 GNU/Linux 2.4.21 Debian Woody
Very Old PC Intel PIII 733 GHz nVidia GNU/Linux 2.4.9 Red Hat 7.1
Old PC AMD Athlon XP 2800+ ATI Radeon 9200 GNU/Linux 2.6.10 Gentoo
Old DELL Inspiron 8100 laptop Intel PIII 1 GHz nVidia GNU/Linux 2.6.11 Gentoo
Toshiba M60-176 laptop Intel Centrino 1,7 GHz ATI Mobility Radeon x700 GNU/Linux 2.6.12 Ubuntu (Breezy Badger to Jaunty)
Toshiba M60-176 laptop Intel Centrino 1,7 GHz ATI Mobility Radeon x700 Windows XP Home Edition
HP/Compaq NC6400 laptop Intel Celeron Mobility 440 at 1,86 GHz Intel 945GM Express GNU/Linux 2.6 Debian Etch
HP/Compaq NC6400 laptop Intel Celeron Mobility 440 at 1,86 GHz Intel 945GM Express Windows XP Professional Edition, Version 2002, Service Pack 2
Workstation Two Intel Xeon 2,66 GHz with hyperthreading nVidia NV34GL (Quadro FX 500) GNU/Linux 2.6.8.1 Debian Woody-based
Nintendo DS (Lite) ARM9 and ARM7 none, but two rendering cores none none
Alienware M15X laptop Core i7 720QM nVidia GeForce GTX 260M GNU/Linux 2.6.31-14 Ubuntu 9.10 (Karmic Koala)
Alienware M15X laptop Core i7 720QM nVidia GeForce GTX 260M Windows Seven 64 bit, Home Premium

We have access to two other sets of computers as well, even though we test our developments on their platforms less frequently. These are two compile farms, kindly provided by:

Finally, we use the VirtualBox virtualizer to build and run OSDL on various vanilla operating systems.

Thanks to all these structures, we believe we have access to most operating system/system type/processor for UNIX-based architectures.

Full testing process

For each new test, our process is to create a brand new user (adduser tester) and to behave exactly as a new LOANI user would behave (su - tester): install everything thanks to LOANI, and run the full test suite thanks to playTest.sh.

That way, we can get rid of side-effects such as configuration files (ex: .*rc) or shell environment (ex: does the user include '.' in his PATH ?). It allows us to feel a little bit more confident with our tests, since we closely emulate OSDL's average user.

With tests made regularly with different other GNU/Linux machines, we believe that this platform is well supported. We fully ported Ceylan and OSDL to Windows (notably XP), and we fix from time to time the Windows build (due to the new code coming from Linux), notably before new release. We ported a while ago both libraries to BSD systems (FreeBSD, OpenBSD mainly).

For the other platforms (we target Mac OS X and others), a port is still to be done, but it should mostly impact Ceylan, since OSDL relies on cross-platform tools. If you would have liked testing Ceylan and OSDL on those platforms, send us a mail!

From time to time we try to use Valgrind on Linux x86 to spot potential bugs. We should do that more often though.

Other quality concerns

Website

From time to time, we check our website for dead links, thanks to the W3C Link Checker tool, with a recursion depth of 10.

Please react!

If you have information more detailed or more recent than those presented in this document, if you noticed errors, neglects or points insufficiently discussed, or if you would like to contribute and help us, even a little bit, drop us a line!


[Top]

Last update: Sunday, January 17, 2010