Fully automated Android build pipeline

My starting point was quite sound, indeed. Colleagues had already automated the installation of the Jenkins server, the required packages - including Java, as well as provided reusable Puppet classes for job creations. Jenkins jobs rely on a single config.xml file, that is an assembly of different sections. Each section is handled by a dedicated template. At this point, I thought creating a simple Gradle job would be akin to a walk in the park, that it would take a few days at most and that I would soon be assigned another task.

The first step was easy enough: just update an existing Puppet manifest to add the Gradle plugin to Jenkins.

Regular readers of this blog know my opinion about Gradle. However, I admit that guaranteeing a build that works regardless of the installed tool version is something that Maven lacks - and should have. To achieve that, Gradle provides a so-called wrapper mechanism through a JAR, a shell script and a properties file, the latter containing the URL to the Gradle ZIP distribution. All three needs to be stored in the SCM.

This was the beginning of my troubles. Having to download in an enterprise environment means going through and authenticate to the proxy. The simplest option would be to set everything in the job configuration…​ including the proxy credentials. However, going this way is not very sound from a security point of view, as anyone having access to the Jenkins interface or the filesystem would be able to read those credentials. There’s a need for another option.

The customer already has a working Nexus repository with a configured proxy. It was easy as pie to upload the required Gradle distribution there and update the gradle.properties to point to it.

The Android SDK is just a ZIP. I reused the same tactics: download it then upload it to Nexus. At this point, an existing Puppet script took care of downloading it, extracting it and setting the right permissions.

This step is the beginning of the real problems, however. Android developers know that the Android SDK is just a manager: one has to manually check the desired platforms and tools to download them on the local filesystem. What is a simple step for an Android developer on his machine is indeed a nightmare to automate, though there’s a command-line equivalent to install/update packages through the SDK (with the --no-ui parameter). For a full description, please check this link.

Google engineers fail to provide 2 important parameters:

There’s a lot of non-working answers on the Web, the most alluring being a configuration file. I found none of them to work. However, I found a creative solution using the expect command. Expect is a nifty command that reads the standard output and fill in the standard input accordingly. The good thing about expect is that it accepts regexp. So, when it asks for proxy login, your type the login, when it asks for the password, you do likewise, and when it asks for license acceptance, you type 'y'. It’s pretty straightforward - though it took me a lot of trial and error to achieve the desired results.

My initial design was to have all necessary Android packages installed with Puppet as part of the server provisioning. For standard operations, such as file creation or system package installation, Puppet is able to determine if provisioning is necessary, e.g. if the file exists, there’s no need to create it and likewise if the package is installed. In the end, Puppet reports each operation it performed in a log. At first, I tried to implement this sort of caching by telling Puppet about which packages was created during the provisioning, since the Android SDK creates one folder per package. The first problem is that Puppet only accepts a single folder to verify. Then, for some packages, there’s no version information (for example, this is the case for Google Play services).

Thus, a colleague had the idea to move from this update from Puppet provisioning to a pre-step in each job. This fixes the non-idempotent issue. Besides, it makes running the update configurable per job.

At this point, I thought it would have been done. Unfortunately, it wasn’t the case, due to a library - Robolectric.

I didn’t know about Robolectric at this time, I just knew it was a testing library for Android that provided a way to run tests without a connected physical device. While trying to run the build on Jenkins, I stumbled upon an "interesting" issue: although Roboletric provides a POM complete with dependencies, the MavenDependencyResolver class hard-codes the repository where to download from.

The only provided workaround is to extend the above class to hack your own implementation. Mine used the enterprise Nexus repository mentioned above.

The end of the implementation was relatively easy. Only were missing the upload of the artifacts to the Nexus repository and the tag of the release in the SCM.

In order to achieve the former, I just added a custom Gradle task to get Nexus settings from the settings.xml (provisioned by Puppet). Then I managed for the upload task to depend on this one. Finally, for every flavor of assemble task execution, I added the output file to the to-be-uploaded artifacts set. This way, the following command would upload only flavours XXX and YYY regardless of what flavour are configured in the build file:

For the release, it’s even simpler: the only required thing was to set this Gradle plugin, which adds a release task, akin to Maven’s deploy.

As a backend developer, I’m used to Continuous Integration setup and was nearly sure I could handle Android CI process in a few days. I’ve been quite surprised at the lack of maturity of the Android ecosystem regarding CI. Every step is painful, badly documented (if at all) and solutions seem more like hacks than anything else. If you want to go down this path, you’ve been warned…​ and I wish you the best of luck.