MeeGo QA-Tools

QA-tools development and whatnot

How to do host based testing part 1

On this post I will go through setting up environment, image creation and some basic test packaging to show how you can do host based testing on MeeGo running in Nokia N900. Although I’m using N900 as the device under test, this can be also done with other devices too. The idea is to take some existing tests for MeeGo, put them in an image and execute the tests from a host machine through ssh. This could be done in a simpler way but I’m doing this as an hands-on-example in a similar way how open test system (OTS) does automated testing to illustrate that test automation is not black magic, etc. Of course this example is lacking full automation present in OTS but should give some idea on what it does behind the scenes anyway. You need to have Linux to try this. I’ve tested it with Ubuntu 10.10 but it can also be done with other Linux distributions.

Creating testable image

First order of business is to have some tests to run. For this example took X tests from MeeGo core test suite and packaged it in the community obs. The test package contains XML file that describes how the tests are executed, files needed by the tests and dependencies to other packages that the tests need. One example for a test package dependency is the used test framework. For more information on creating test packages see the test packaging wiki page.

Now that we have  tests we need to create image containing them. For that we need to create our own MeeGo image. If you are not familiar with image creation, see this wiki page.

I used this kickstart file as a template to create the image. To get the test package in the image the kickstart file needs two changes. First change is to add the used repository

repo –name=home-timoph     –baseurl=http://repo.pub.meego.com/home:/timoph/meego_current_extras_handset/ –save –debuginfo –source –gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-meego

The second change is to add the test package to the %packages section

# Tests – edit according to your needs
mcts-x-tests

Now we are ready to create the image

sudo mic-image-creator –run-mode=0 –cache=mycachedir –format=raw –arch=armv7l –save-kernel –config=meego-handset-n900-host-based-testing.ks

After the image creation finishes, install it to your N900 using these instructions and boot the device.

Preparing the environment

Before the tests can be executed we need to install testrunner-lite from Tools:Testing repository to the host PC and setup the connection between the device and the host PC. First connect the device to your PC with a USB cable and configure usb0 network interface.

sudo ifconfig usb0 192.168.2.14 up

If you have network manager running and it causes issues with the USB network connection, you can safely disable it while doing this. Next we need to enable passwordless logins to the device through ssh.

We could install the ssh key used in test automation to avoid having to manually add the key to the device but that would introduce a security risk so we will use your own key. If you don’t have a ssh key for your user generate one with ssh-keygen command. Copy your public key to the device’s authorized keys list. The password for the device’s root user is ‘meego’ unless you changed it from the kickstart file.

cat ~/.ssh/id_rsa.pub | ssh root@192.168.2.15 “cat >> ~/.ssh/authorized_keys”

Next test that the passwordless login work by connecting to the device and confirm that there’s no password prompt

ssh root@192.168.2.15

To execute all tests found in the device create a directory, create the following run script (e.g. get-and-run-tests.sh) in it and set executable bit to the script (chmod +x get-and-run-tests-sh).

#!/bin/sh
# copy test.xmls from dut
scp -r root@192.168.2.15:/usr/share/*-tests .
# run tests
for i in `ls`
do
# remove the –automatic switch if you want also to run manual cases
testrunner-lite -f $i/tests.xml -v -o $i-results.xml -t root@192.168.2.15 –automatic
done

return $?

The script fetches all the tests.xml files from the device that are found following the test packaging rules and executes them with testrunner-lite. Results from the tests are written to xml files named test-package-name-results.xml

Now you can start the script and see inside the test automation black box

./get-and-run-tests.sh

I hope this gave some insight on how automated host based test execution works. On part 2 of this host based testing thing, I will do pretty much the same thing but in a more user friendly way using a graphical tool called testrunner-ui to execute the tests which is better suited for manual testing than this shell script.

About these ads

One response to “How to do host based testing part 1

  1. Pingback: How to do host based testing part 2 « MeeGo QA-Tools

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: