Android, Robotium and Spoon: A Practical Guide for Testing on Android Devices

Developing for Android is a lot of fun. It has actually become a niche for me here at SEP. Android has changed quite a bit over the past 3 years since I first started hacking on it – action bars, more screen sizes, and probably the biggest addition…fragments.

But one thing certainly has not changed. Testing on real devices is not easy.

I’m not going to go into detail about the importance of unit testing and functional testing. I am assuming that since you are reading this, you already know that both types of testing are important.

Instead, this is a practical guide showing how my teammates and I are using Robotium and Spoon to perform functional testing on real Android devices.

Robotium

Robotium is a testing framework that works with Android’s “Instrumentation” tests.

The biggest benefit of using a tool like Robotium is that you can write tests as if you were a user (with very little knowledge about the implementation).
For example, your tests might look like the following:


robo.enterText(0, "user-name");
robo.enterText(1, "password");
robo.clickButton(0);

At first pass, this looks terrible. I can still hear myself saying, out loud, “why do I have to use these awful indexes instead of IDs!?!” Honestly, though, that is the secret sauce.
The above example simply looks for the first text box, and types in the string “user-name”…same for the password. Then Robotium looks for the first button, and triggers the “performClick()” action on that button.

There are many other great benefits from using Robotium. It takes care of a lot of the gross/hard work…like how long do I have to wait before I assert that a new Activity started? Or, how can I assert that the text is actually visible to the user?

Setup is super simple. Include the “robotium-solo-*.jar” and create a new instance of “Solo” for use in your testing project.


robo = new Solo(getInstrumentation());

Now that you are writing functional tests, you need a way to get feedback about how those functional tests behave on a multitude of devices.
Meet Spoon.

Spoon

The people at Square released a wonderful tool called Spoon.

Spoon itself is a separate testrunner which can be run from the command line. Spoon then runs your functional tests on all of the devices connected to the computer (including emulators running). Spoon will then record the results, save off the logcat logs, and take screenshots of the app during testing.

Tips

Both Robotium and Spoon have greatly improved our functional testing. We have a stable setup that the entire team has grown to trust in.

We did have some issues come up as we went.  Below are some additional tips that we have learned during our adventure.

  •  make sure you follow the OEM USB Drivers documentation in order to enable USB Debugging on all devices and platforms
  •  take the Spoon screenshot, right before you assert – this helps so that you can see the screen right before the assert…if the test fails, you still can see what potentially went wrong.

robo.clickOnText("List Item 1");
takeScreenshot("list_item_1_detail_view");
assertTrue(robo.searchText("List Item 1 Details"));
    • use “waitForIdleSync()” to ensure the views are fully loaded

public void takeScreenshot(String imageName) {
    instrumentation.waitForIdleSync();
    Spoon.screenshot(getCurrentActivity(), imageName);
}
  • disable the lock screen (Settings -> Security -> Screen lock -> None)
    • by disabling the lock screen, you don’t need to include the extra permissions for Spoon to unlock the device
  • dim the brightness, to protect from draining the batteries while plugged into a USB port (USB is typically a slow charger)
  • enable the “Stay Awake” setting to keep the screen turned on (Settings -> Developer options -> Stay awake)

There is no doubt that functional testing is important.  Robotium and Spoon have both helped us to write better functional tests.

Next up…getting Monkey and MonkeyRunner to play well with our application.

The following two tabs change content below.
Comments
  • karthik

    I tried to run the test on my normal Robotium code.

    All I did is this.

    1. Added all the libraries (Runner and the client of spoon)
    2. Added the code which takes screenshot.
    3. Tried to execute the test via eclipse (Not the command prompt)

    But its throwing me error.

    I know, I am missing something (Not running in command prompt), but still I am not fully good with the concept of spoon.

    Can you better explain this with a sample code snippet or downloadable project.

    Thanks,
    Karthik KK

  • Matt Terry

    Hello Karthik – thank you for reaching out to me!

    I am certainly happy to post a snippet of what our tests look like, I will do that in the short future.

    In the mean-time, can you tell me what error you are getting?
    I am assuming that you are running the tests through Eclipse, so when do you get the error? Is it immediately, or after the tests are finished running, or sometime in between?

    Talk to you soon!
    -MaTT

  • karthik

    Actually I am getting error immediately after running the test.

    It’s a weird error which occurs even if I add the two Jar files in my build path as reference.

    I am sure it’s problem with my config or the way l am doing it.

    Just post a very simple code bundle which will be helpful for me and all a lot.

    Thanks
    Karthikkk

  • karthik

    Hy Matt,

    I am getting exception as
    java.lang.RuntimeException: Unable to capture screenshot.

    Here is the detailed log
    04-15 03:20:54.313 info TestRunner started: testDialog(com.example.calcapplication.test.CalcTestCase)
    04-15 03:20:54.672 debug dalvikvm GC_FOR_ALLOC freed 480K, 21% free 2618K/3288K, paused 129ms, total 136ms
    04-15 03:20:55.632 debug dalvikvm GC_FOR_ALLOC freed 20K, 12% free 2902K/3288K, paused 28ms, total 29ms
    04-15 03:20:56.012 debug gralloc_goldfish Emulator without GPU emulation detected.
    04-15 03:20:56.622 debug dalvikvm GC_EXPLICIT freed 41K, 10% free 3135K/3468K, paused 4ms+7ms, total 151ms
    04-15 03:20:56.732 debug dalvikvm GC_EXPLICIT freed 8K, 10% free 3127K/3468K, paused 3ms+6ms, total 90ms
    04-15 03:20:56.732 info TestRunner failed: testDialog(com.example.calcapplication.test.CalcTestCase)
    04-15 03:20:56.742 info TestRunner —– begin exception —–
    04-15 03:20:56.782 info TestRunner java.lang.RuntimeException: Unable to capture screenshot.
    04-15 03:20:56.782 info TestRunner at com.squareup.spoon.Spoon.screenshot(Spoon.java:54)
    04-15 03:20:56.782 info TestRunner at com.example.calcapplication.test.CalcTestCase.testDialog(CalcTestCase.java:37)
    04-15 03:20:56.782 info TestRunner at java.lang.reflect.Method.invokeNative(Native Method)
    04-15 03:20:56.782 info TestRunner at java.lang.reflect.Method.invoke(Method.java:525)
    04-15 03:20:56.782 info TestRunner at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
    04-15 03:20:56.782 info TestRunner at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
    04-15 03:20:56.782 info TestRunner at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
    04-15 03:20:56.782 info TestRunner at junit.framework.TestCase.runBare(TestCase.java:134)
    04-15 03:20:56.782 info TestRunner at junit.framework.TestResult$1.protect(TestResult.java:115)
    04-15 03:20:56.782 info TestRunner at junit.framework.TestResult.runProtected(TestResult.java:133)
    04-15 03:20:56.782 info TestRunner at junit.framework.TestResult.run(TestResult.java:118)
    04-15 03:20:56.782 info TestRunner at junit.framework.TestCase.run(TestCase.java:124)
    04-15 03:20:56.782 info TestRunner at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
    04-15 03:20:56.782 info TestRunner at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
    04-15 03:20:56.782 info TestRunner at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
    04-15 03:20:56.782 info TestRunner at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
    04-15 03:20:56.782 info TestRunner Caused by: java.lang.NullPointerException
    04-15 03:20:56.782 info TestRunner at com.squareup.spoon.Spoon.obtainScreenshotDirectory(Spoon.java:106)
    04-15 03:20:56.782 info TestRunner at com.squareup.spoon.Spoon.screenshot(Spoon.java:49)
    04-15 03:20:56.782 info TestRunner … 15 more
    04-15 03:20:56.782 info TestRunner —– end exception —–
    04-15 03:20:56.842 info TestRunner finished: testDialog(com.example.calcapplication.test.CalcTestCase)

    Is there any config file, where I can define the path or location of where to store the screenshot, please give your comments !!!

    Thanks,
    Karthik KK

  • Matt Terry

    Hey Karthik – there is not any config file that I am aware of for defining the path for the screenshots.

    Are you experiencing this exception on a physical device, or on an emulator?

    If it is an emulator, make sure that you have enough storage (either an SD Card with at least 200 MiB, or “internal memory” of at least 200 MiB).

    If you are using a physical device, make sure that it has an SD Card installed, and that you are able to read/write from that SD Card (take some pictures via the camera, for example).

    Feel free to email me and we can discuss further – mcterry@sep.com

  • karthik

    Hy Matt,

    Thanks for your help.

    I actually figured out the mistake I was making with my code :)

    The problem for the error is I did not pass the correct reference of my Activity to Spoon.screenshot, hence I used the Solo.getCurrentActivity() method and passed it as the parameter and it worked fine !!!

    Thanks,
    Karthik KK

  • Matt Terry

    Oh good, I’m glad you got it figured out!

    Happy Testing, Karthik!

  • lau

    i run success this testcase,log“capture success”,but
    spoon-output/image/devicename/classname/ not have picture,
    do you know the picture path on the phone or PC ?

  • Matt Terry

    Hello lau – just to confirm, are you explicitly calling the method “Spoon.Screenshot()”?

    The path that you have shared is where I would have expected the images to show up.

  • Name (required)

    Hi Matt,

    Thank you for a nice article. I wanted to use spoon with Android. So I downloaded spoon-client and spoon-runner and added them to libs of my project ( copy them libs in my project). so when I compile I am getting this issue

    [2014-09-26 15:57:31 – Dex Loader] Unable to execute dex: Multiple dex files define Lcom/squareup/spoon/Chmod$1;
    [2014-09-26 15:57:31 – SmartyApp26Test] Conversion to Dalvik format failed: Unable to execute dex: Multiple dex files define Lcom/squareup/spoon/Chmod$1;

    I tried the solution given like change the VM in eclipse.ini and stuff, but still it has an issue. Can you please let me know Where I am causing an issue..?

    Chandra

  • Chandra

    Hi Matt,

    Thank you for a nice article. I wanted to use spoon with Android. So I downloaded spoon-client and spoon-runner and added them to libs of my project ( copy them libs in my project). so when I compile I am getting this issue

    [2014-09-26 15:57:31 – Dex Loader] Unable to execute dex: Multiple dex files define Lcom/squareup/spoon/Chmod$1;
    [2014-09-26 15:57:31 – SmartyApp26Test] Conversion to Dalvik format failed: Unable to execute dex: Multiple dex files define Lcom/squareup/spoon/Chmod$1;

    I tried the solution given like change the VM in eclipse.ini and stuff, but still it has an issue. Can you please let me know Where I am causing an issue..?

    Chandra

  • Rajesh

    @Chandra, Eclipse Project >> Properties >> Build Path >> Remove Android Private Libraries

  • Khushabu

    Hii
    Matt
    I am beginner for the testing ,can you guide me for this tool .How to use this?

  • Name (required)

    com.squareup.spoon.SpoonTestRunListener.testFailed(Lcom/android/ddmlib/testrunner/TestIdentifier;Ljava/lang/String;)V

    Hi guys, I am using “androidTestCompile ‘com.squareup.spoon:spoon-client:1.1.9′” dependency in my build.gradle file for generating report using Spoon. during Spoon.screenshot(getActivity(),”screenName”), test execution fails and gives error

    com.squareup.spoon.SpoonTestRunListener.testFailed(Lcom/android/ddmlib/testrunner/TestIdentifier;Ljava/lang/String;)V

  • Piyush

    com.squareup.spoon.SpoonTestRunListener.testFailed(Lcom/android/ddmlib/testrunner/TestIdentifier;Ljava/lang/String;)V

    Hi guys, I am using \”androidTestCompile \’com.squareup.spoon:spoon-client:1.1.9\’\” dependency in my build.gradle file for generating report using Spoon. during Spoon.screenshot(getActivity(),\”screenName\”), test execution fails and gives error

    com.squareup.spoon.SpoonTestRunListener.testFailed(Lcom/android/ddmlib/testrunner/TestIdentifier;Ljava/lang/String;)V

  • Lloyd

    Hy Matt,

    I am getting exception as

    java.lang.RuntimeException: Unable to capture screenshot.
    at com.squareup.spoon.Spoon.screenshot(Spoon.java:80)
    at com.squareup.spoon.Spoon.screenshot(Spoon.java:53)
    at com.tudouapp.test.N15_Noticefacation.test_noticefacation(N15_Noticefacation.java:81)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
    at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
    at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
    at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
    at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
    at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:554)
    at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1701)
    Caused by: java.lang.NullPointerException
    at com.squareup.spoon.Spoon.screenshot(Spoon.java:72)
    … 16 more

    please give your comments !!!

    Thanks,

  • Name (required)

    Hello Matt,
    The screenshot taken by the Soon library is stored on the device? What if I want it to be in a folder on Jenkins
    ?

    Thnks,
    Rathna

  • Matt Terry

    Hello Rathna,

    Spoon will pull the images off of the devices by default…I recommend relying on the spoon output if possible. Perhaps a solution like this one will be useful for you…

    http://stackoverflow.com/a/24624262

    If using the spoon output with Jenkins isn’t giving you what you want, let me know and together we can figure out a solution that works for you!

    Happy testing!!

Add your comment

Your email address will not be published.