using Kotlin and command-line via Gradle

Evan Fang

Sep 2, 2018·2 min read

Recently I’m running an Android project. In order to make sure it’s quality and stability, we’ll usually write unit tests and instrumented tests for every feature we implemented. If you have some experience in testing method knowledge, you might hear about RAT(Release Acceptance Test), which runs on each release build to check if the build is stable for further testing. I’m faced with a problem that I wanted to run RAT tests ONLY on Jenkins triggered after each merge request being accepted.

I created a new Kotlin project with default login page on Github for example, you can clone it as a local project to run:
Now, here comes four test cases for testing login flow:

Use command line to cd to project path and run
./gradlew connectedAndroidTest
It should run all the tests successfully.

At this moment the QA(Quality Assurance) of your team might say that: “Hey! Evan, could we pick some tests for RAT? Because there are too many tests in our project, it takes a long time to run. I just wonder if we can make a quick test .”

Okay, now we need to figure out how to run specific tests only, and here is the solution:

Step 1) Create a custom annotation

@Target specifies the possible kinds of elements which can be annotated with the annotation (classes, functions, properties, expressions etc.)

We used it for annotate some functions, so we use AnnotationTarget.FUNCTION

@Retention specifies whether the annotation is stored in the compiled class files and whether it’s visible through reflection at runtime (by default, both are true).

AnnotationRetention.RUNTIME makes sure that the Rat annotation is visible to the test runner during the runtime.

Step 2) Annotate @Rat on the tests you want to run

Step 3) Use gradlew to run only RAT tests

./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.annotation=com.mutant.androidtestcustomannotations.annotation.Rat

Congratulations, we finally run the tests with @Rat annotation.

Android framework provides many command line to filter tests while using adb shell am instrument. But if we want to run automation tests, we need to install the test app first and then execute adb shell am instrument, that is a little complicated. Fortunately, Gradle plugin added support for specifying instrumentation test-runner arguments from the command line via Gradle after version 1.3.0.

Which means

./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.<argName>=<argValue>

is equivalent to running

adb shell am instrument -w -e <argName> <argValue> com.mutant.androidtestcustomannotations.test/android.support.test.runner.AndroidJUnitRunner

We can run only large size tests with command line:

./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.size=large

or run single test like this:

./gradlew connectedAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.mutant.androidtestcustomannotations.ExampleInstrumentedTest#invalidEmail_showError

I hope this article will help you, and here is the full project used in this article

Github example — AndroidTestCustomAnnotations

and the references I surveyed on the Internet:

Kotlin annotations

Google Testing Templates project on Github