|How to do host based… on How to do host based testing p…|
|kyranto on Welcome to the MeeGo QA-tools…|
QA-tools development and whatnot
On this post I will go through setting up environment, image creation and some basic test packaging to show how you can do host based testing on MeeGo running in Nokia N900. Although I’m using N900 as the device under test, this can be also done with other devices too. The idea is to take some existing tests for MeeGo, put them in an image and execute the tests from a host machine through ssh. This could be done in a simpler way but I’m doing this as an hands-on-example in a similar way how open test system (OTS) does automated testing to illustrate that test automation is not black magic, etc. Of course this example is lacking full automation present in OTS but should give some idea on what it does behind the scenes anyway. You need to have Linux to try this. I’ve tested it with Ubuntu 10.10 but it can also be done with other Linux distributions.
Creating testable image
First order of business is to have some tests to run. For this example took X tests from MeeGo core test suite and packaged it in the community obs. The test package contains XML file that describes how the tests are executed, files needed by the tests and dependencies to other packages that the tests need. One example for a test package dependency is the used test framework. For more information on creating test packages see the test packaging wiki page.
Now that we have tests we need to create image containing them. For that we need to create our own MeeGo image. If you are not familiar with image creation, see this wiki page.
repo –name=home-timoph –baseurl=http://repo.pub.meego.com/home:/timoph/meego_current_extras_handset/ –save –debuginfo –source –gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-meego
The second change is to add the test package to the %packages section
# Tests – edit according to your needs
Now we are ready to create the image
sudo mic-image-creator –run-mode=0 –cache=mycachedir –format=raw –arch=armv7l –save-kernel –config=meego-handset-n900-host-based-testing.ks
After the image creation finishes, install it to your N900 using these instructions and boot the device.
Preparing the environment
Before the tests can be executed we need to install testrunner-lite from Tools:Testing repository to the host PC and setup the connection between the device and the host PC. First connect the device to your PC with a USB cable and configure usb0 network interface.
sudo ifconfig usb0 192.168.2.14 up
If you have network manager running and it causes issues with the USB network connection, you can safely disable it while doing this. Next we need to enable passwordless logins to the device through ssh.
We could install the ssh key used in test automation to avoid having to manually add the key to the device but that would introduce a security risk so we will use your own key. If you don’t have a ssh key for your user generate one with ssh-keygen command. Copy your public key to the device’s authorized keys list. The password for the device’s root user is ‘meego’ unless you changed it from the kickstart file.
cat ~/.ssh/id_rsa.pub | ssh email@example.com “cat >> ~/.ssh/authorized_keys”
Next test that the passwordless login work by connecting to the device and confirm that there’s no password prompt
To execute all tests found in the device create a directory, create the following run script (e.g. get-and-run-tests.sh) in it and set executable bit to the script (chmod +x get-and-run-tests-sh).
# copy test.xmls from dut
scp -r firstname.lastname@example.org:/usr/share/*-tests .
# run tests
for i in `ls`
# remove the –automatic switch if you want also to run manual cases
testrunner-lite -f $i/tests.xml -v -o $i-results.xml -t email@example.com –automatic
The script fetches all the tests.xml files from the device that are found following the test packaging rules and executes them with testrunner-lite. Results from the tests are written to xml files named test-package-name-results.xml
Now you can start the script and see inside the test automation black box
I hope this gave some insight on how automated host based test execution works. On part 2 of this host based testing thing, I will do pretty much the same thing but in a more user friendly way using a graphical tool called testrunner-ui to execute the tests which is better suited for manual testing than this shell script.