March 23, 2012

Jenkins + CMake/CTest

Some time ago I setup Jenkins CI to compile & test our code. Installation itself was straightforward, matched to documentation on official site (besides official documentation, there is pretty good book "Jenkins: The Definitive Guide"). The only problem was that it didn't work with OpenJDK, so I was need to install Sun JDK.
For C++ components we're using CMake. Out of box, Jenkins has no support for CMake, but there is plugin in repository, that you can install separately. This plugin allows to specify different options for CMake, and will perform compilation and installation of code. (Although, at the end, I setup configuration that explicitly run CMake with needed options, and perform compilation via GNU Make with some additional options). The main problem with CMake (CTest really) was that tests results are in format, that isn't acceptable by default test analyzer that is designed to work with JUnit logs. Plugin repository has xUnit plugin, that should work with other test frameworks, but it also doesn't support CTest log format. Solution was found on StackOverflow - we need to write custom XSL that will convert CTest results into JUnit format. I found stylesheet at StackOverflow and slightly modified it. So, build step for testing now looks following way:
cd Debug
TRES=0
ctest -T test --no-compress-output || true
if [ -f Testing/TAG ] ; then
   xsltproc /var/lib/jenkins/userContent/xunit/CMake/2.8/ctest2junix.xsl Testing/`head -n 1 < Testing/TAG`/Test.xml > CTestResults.xml
fi

I was need to rewrite 3rd line this way, because Jenkins aborted job if ctest returned non-0 result if one of tests failed, so other commands weren't executed, and we had no testing results in dashboard.
So now, all our components are compiled & tested immediately after check-in, and we're able to catch error early.

Update: last line will cause build be marked as broken if any test is failing. You can remove it, and in this case, Jenkins will mark build as unstable if any test is failing - it will take this information from tests results

2 comments:

Tim Sutton said...

Hi

I just wanted to say thanks very much for this great post - it made it really easy for me to set up jenkins against our CMake/CTest based project. I used a slighly modified version of your commands which I list below in case it is of any help to anyone. Our app is a gui app so the commands let the tests run in gui mode.

---------------------------
cd build
cmake ..
xvfb-run --auto-servernum --server-num=1 \
--server-args="-screen 0 1024x768x24" \
make Experimental || true
if [ -f Testing/TAG ] ; then
xsltproc ../tests/ctest2junix.xsl \
Testing/`head -n 1 < Testing/TAG`/Test.xml > \
CTestResults.xml

fi
---------------------------

Regards

Tim

Alex Ott said...

thank you for information...
i'm try to argue that we can also run GUI tests for java programs on jenkins...