Introduction
Recently I had a few people ask me, “How do I start with testing in Android?”
As much as I wanted to say, “It depends”, I decided on a simpler answer, it was “Start with User Acceptance Testing, either with Espresso or Robotium“.
This is because UAT testing is generally easier for people to understand and because Android apps often contain much more “view logic” then business logic.
Luckily for me, the question also inspired me to have another go at getting Gradle to run all of the different test types.
Like a lot of people, I still haven’t made the switch from Eclipse to Android Studio but as a former Java developer I am quite familiar with the advantages of a good build tool especially when it comes to my current favorite thing, Continuous Delivery.
For those of you that are not familiar with the concepts of Continuous Delivery or tools like Jenkins, the important thing to know here is this.
You can use a tool like Jenkins to monitor your Source Code Management (SCM) system for changes and then when it detects one, it will check out the code, run some Gradle commands and quite a lot of other stuff too.
Cool right? I thought so when I discovered it. Now that I am doing Android I find it even more important. Anyone that has tried to write and run a set of User Acceptance Tests for Android will tell you with sadness just how slow they run. So slow in fact, that I recommend people almost never run the entire test suite themselves. By letting Jenkins take care of it and then notify me if something breaks I can save many hours of time that I would otherwise spend drinking coffee (aka waiting for the UAT tests to finish).
Here is a diagram of how to “leverage” continuous delivery and how with (in this case jenkins) you can get from raw source code to a testable APK that has already passed all the automated checks and tests.
File Structure
Now the reason that I mentioned that I still use Eclipse is because the “standard” Gradle/Android Studio project structure cannot be used in Eclipse and as such the best option is to structure the projects for eclipse and then modify your Gradle build files to match.
Based on my experience and some good ideas from others (here and here) this is what I have come up with:
I have built myself a sample project so that I can just clone something at the start of a project and get started. You are welcome to fork or clone it from here.
(One warning however, I like to use AndroidAnnotations so the project is set up for that. While I recommend it, if you don’t want to use it, removing all references to AA from the .gradle files should be all that’s needed)
Exploring the Gradle Files
Lets review the build.gradle files included in the sample project.
First is the build.gradle file for the entire project, there is nothing too funky here but a couple of things of note:
- Lines 25-38: These lines provide a central place for managing versions and other config, definitely easier then changing it multiple places.
- Lines 41-46: Nice little trick I picked up for setting the repositories in all of the sub-projects.
- Lines 50-88: This is a standard local config for Sonar, it is the base project config and it also defines the sub-modules. This will allow us to see the project as a whole instead of several separate parts.
The following is the build.gradle for the app (android part) of the project. Things to note:
- Lines 27-30: These are the dependencies for testing (currently Robotium and Mockito)
- Lines 48-51: These are required for testing and define the package and the runner.
- Lines 68-76: These lines provide the link from the app project to the test code. This is really the key, by placing the test config in this file instead of a build.gradle in the test directory Gradle is able to “see” both the test code and the app code at the same time.
- Lines 12-15, 23-24, 83-103: These lines are needed in Android Annotations, if you don’t want to use that you can just delete those lines.
- Lines 106-117: This is the specific sonar config for the sub-module.
This file is pretty much a standard Gradle config for testing a vanilla java project. The only thing to note is that the test code is inside the project instead of outside like the android test code.
The Gradle commands you are going to need are:
Please note that all of these commands should be run in the base project directory.
To build use (This also runs the tests in the lib project):
gradle clean build
To run sonar analysis:
gradle sonarRunner
To run Android Tests (Robotium, etc)
gradle connectedInstrumentTest
Other features of the sample project include:
- The Android test project is already configured with Robotium and Mockito and contains a few simple sample tests.
- The lib project is already configured as a dependency of the app project.
- The lib project is already configured with standard jUnit4 and contains a simple sample test.
- It’s all release under Apache 2 license so you can pretty much do anything you like with it.
- It includes a sample config for a local sonar installation.
If you like it, please fork/star/share the project or don’t agree with the setup please let me know, or better yet send in a pull request.
Running the tests in Eclipse:
To run the java tests for the lib project, you can right click on the project and “Run As JUnit Test”.
To run the android (and Robotium) tests in the test project, you can right click on the project and “Run As Android JUnit Test”.
Aspects of the Robotium Test:
Below I have included the simplest of Robotium tests, a simple verification that the expected content is displayed on screen.
As you can see on line 7 we are searching for a particular string but I have used the android resource instead of a raw string.
A raw string will work but I personally find that using an Android resource wherever possible reduces the amount of test breakages that occur from minor changes, especially copy writing changes.
It is possible to argue that this test falls into the “too simple to break” category and therefore shouldn’t be written. While this might be true, this test is very quick and easy to write and will hardly ever break. But when it does break, it usually is an indication that any related tests are going to break, so it can be useful.
For the second test, we are doing a simple test for the onClick() event of the button. In it we are simply clicking the button and checking for the expected outcome. In this case it is a Toast.
Generally you should have at least one test per onClick() event, but you will generally find that you often have 2 or more. One for the “happy path” or successful outcome and one for the “error” outcome. If you have multiple error outcomes you should have a test for each common outcome and you should let your experience (and bug reports) be your guide as to how many of these error outcomes you need to to verify.
If you are having trouble figuring out how to test error outcomes, you should look into Stubs and Mocks. With these you will be able to “inject” fake implementations of external components and/or resources that allows you to control the outcome.
Wrapping it up:
As with all my posts, feedback is welcome and encouraged (positive and constructively negative).
If you want more background on testing and continuous delivery please check out my presentation to KL GDG DevFest 2013 the slides below (don’t forget to turn on “Speaker notes”).
