Peer review with SonarQube
Most of the time we are under time pressure to deliver a product. As a consequence bad coding habits strike back. It happen also if the programmer is not yet enough experienced. Peer review can solve this problem. Still it is not easy if the product is growing fast and nobody has a full sight on whole project or if you are alone on some part of it. Solution I was searching for was tool that automatically gives a descriptive overview of code quality. I wanted to get rid of most obvious code duplications in the project and stick to the DRY principle.
There are quite a few statical analysis tools that can help with that. PMD, Simian, Atomiq, Lint, FindBugs..
Sonarlint
SonarLint Is IDE plugin that acts as on fly first line of defense agains the bugs we ignorant developers produce on daily basis. It can be integrated with Eclipse, IntelliJ, Microsoft IDE and even Atom. You can even connect it to the SonarQube server to share the ruleset.
Integration of SonarLint into Gradle task.
SonarQube™
I gave a try to SonarQube. It is an open source quality management platform, dedicated to continuously analyzing and measuring the technical quality of source code. It is really meant for everybody who wants to manage code quality. It is coming form Switzerland, so it has to be good. Realy nice feature is the graphical interface that enables visual representation of bugs, and potential bugs in code. It also can find code duplications and shows it line by line with vertical bars. At this time the latest version is 7.1, released on 18 Apr. 2018.
We are provided with Demo SonarQube instance where some open source projects can be viewed. To manage to analyze out project like we can see on demonstration, we need to install SonarQube server, database to save data from analysis, and set up the project to enable scanning.
Presentation
Short introductory presentation about SonarQube that I put together to easily convince my collegues to give a shot with SonarQube.
Docker
To take from installation process as much environmental inconsistency as possible, I chose to deploy everything in docker. Prerequisite is an working version of docker. If you do not have an running instance already, take a look at the official page on how to get started. Running SonarQube in docker requires few commands.
SonarQube image
docker pull sonarqube
It is build on top of java:openjdk-8u45-jdk, and has opened port 9000. Here is the line to run the SonarCube server with the latest 7.1 version.
docker run -d --name sonarqube -p 9000:9000 -p 9092:9092 sonarqube:latest
Database image
Per default SonarQube uses H2 Database Engine which is embedded into server and gets fresh installed on every upgrade. It is only meant to be used for testing the functionality. Among supported databases I chose to try out Postgres and MySql.
Postgres
Run Postgres image with provided default environmental variables for this virtual machine. In this example I am using simple credentials.
docker pull postgres
docker run --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=sonar -e POSTGRES_USER=sonar -d postgres:9.4
Starting SonarQube server with Postgres database for data persistence. The production server can be configured with following environmental variables: SONARQUBE_JDBC_USERNAME, SONARQUBE_JDBC_PASSWORD and SONARQUBE_JDBC_URL.
docker run -d --name sonarqube -p 9000:9000 -p 9092:9092 -e SONARQUBE_JDBC_USERNAME=sonar -e SONARQUBE_JDBC_PASSWORD=sonar -e SONARQUBE_JDBC_URL=jdbc:postgresql://postgres/sonar --link postgres:postgres sonarqube:5.2
The line -p 9000:9000 is mapping port 9000 from container to docker-machine and it is the entry point to send results to server and view them later.
In case you forgot the password of admin account. You can reset it back to default “admin” one with following command:
UPDATE users SET crypted_password = '88c991e39bb88b94178123a849606905ebf440f5', salt='6522f3c5007ae910ad690bb1bdbf264a34884c6d' WHERE login = 'admin';
MySql
The same as for Postgres.
docker pull mysql
docker run --name mysql -p 5432:5432 -e MYSQL_ROOT_PASSWORD=trFm9-3Q -e MYSQL_DATABASE=sonar -e MYSQL_USER=sonar -e MYSQL_PASSWORD=sonar -d mysql:5.5
And it is equivalent to creating a new database in MySql virtual machine:
CREATE DATABASE sonar CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE USER 'sonar' IDENTIFIED BY 'sonar';
GRANT ALL ON sonar.* TO 'sonar'@'%' IDENTIFIED BY 'sonar';
GRANT ALL ON sonar.* TO 'sonar'@'localhost' IDENTIFIED BY 'sonar';
FLUSH PRIVILEGES;
Starting SonarQube server with MySql database connection.
docker run -d --name sonarqube -p 9000:9000 -p 9092:9092 -e SONARQUBE_JDBC_USERNAME=sonar -e SONARQUBE_JDBC_PASSWORD=sonar -e SONARQUBE_JDBC_URL=jdbc:mysql://mysql:3306/sonar?useUnicode=true\&characterEncoding=utf8\&rewriteBatchedStatements=true --link mysql:mysql sonarqube:5.2
Gradle configuration
My goal was to analyze Android project developed in Android Studio with Gradle build tool. SonarQube provides us with detailed documentation for official Gradle plugin. Settings in the concrete top build.gradle file:
plugins {
id "org.sonarqube" version "1.1"
}
sonarqube {
properties {
property "sonar.host.url", "http://192.168.99.100:9000/"
property "sonar.projectName", "client_android"
property "sonar.projectKey", "client_android"
}
}
project(":android") {
sonarqube {
properties {
property "sonar.language", "java"
property "sonar.sources", "src"
property "sonar.binaries", "build"
}
}
}
The sonar.host.url property tells us the url of the server. If it is running in docker we can get it from docker-machine, with mapped port of container.
docker-machine ls
This command reveals to us the url of docker machine and default sonar port is 9000 which is data needed in above
property.
sonar.projectName is the name displayed in the web page.
sonar.projectKey is the id of the project, because there can have multiple projects on one server.
On “:android” module level we tell the analyzer that the language is java, source is in src folder and the binaries
are in build folder. Now the new Gradle task “sonarqube” is available. It can be run with:
gradle sonarcube
After analysis is finished the results are available on the same url as above; http://192.168.99.100:9000/.
default admin account credentials:
user:admin
pwd:admin
Maven configuration
We have to specify sonarqube plugin group and profile with the URL pointing to sonar server in maven settings that will be applied to build system. We also have to specify profile with the URL pointing to sonar server.
<settings xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd">
...
<profiles>
<profile>
<id>sonar</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<sonar.host.url>
https://sonarqube.best-it-service-company-ever.com
</sonar.host.url>
</properties>
</profile>
...
</profiles>
<pluginGroups>
<pluginGroup>org.sonarsource.scanner.maven</pluginGroup>
...
</pluginGroups>
</settings>
Add Sonar maven plugins to main pom.xml. JaCoCo is used to gnerate coverage reports.
<properties>
<sonar-maven-plugin-version>3.4.0.905</sonar-maven-plugin-version>
<jacoco-maven-plugin-version>0.8.1</jacoco-maven-plugin-version>
</properties>
..
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<runOrder>random</runOrder>
</configuration>
</plugin>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>${sonar-maven-plugin-version}</version>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco-maven-plugin-version}</version>
<executions>
<execution>
<id>agent</id>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
<execution>
<id>default-report</id>
<phase>prepare-package</phase>
<goals>
<goal>report</goal>
</goals>
</execution>
</executions>
</plugin>
To generate Sonar report and send it to the server execute maven sonar:sonar goal. For example
mvn clean install sonar:sonar.
Maven with passed argLine
TL;DR; Maven argLine parameter is overwriting JaCoCo default argLine for destfile. As a result jacoco.exec file is missing for ceration of coverage reports in Sonar.
I had a case in which running maven locally, sonar analysis was generated successfully, including coverage reports. But when executing maven inside docker for the same program, test coverage was not passed to Sonar server. When checking logs I noticed following line:
[INFO] --- jacoco-maven-plugin:0.8.1:prepare-agent (agent) @ inventory ---
[INFO] argLine set to -javaagent:/root/.m2/repository/org/jacoco/org.jacoco.agent/0.8.1/org.jacoco.agent-0.8.1-runtime.jar=destfile=/opt/path/to/the/jacoco.exec
Additional argLine was set for destfile. Actual line from where I got a hint what is going on is here:
[INFO] --- jacoco-maven-plugin:0.8.1:report (default-report) @ inventory ---
[INFO] Skipping JaCoCo execution due to missing execution data file.
JaCoCo could not find jacoco.exec file. It was not generated during analysis process.
I compared mvn commands, and noticed that the only difference was that in docker image we were passing additional argLine to maven like following.
RUN mvn clean package sonar:sonar -U -DargLine="-Dxxx=yyy"
When I removed argLine parameter form maven command then the coverage reports were created, but then other pieces of program were not working correctly as of the missing argument. So it clearly shows that the argLine that is passed as command line to maven overwrites the one JaCoCo plugin is generating.
All help that I could find was on the Surefire help page and on Stack Overflow. Using @{...} placeholder to late replace argLine. Now in Docker file instead of
RUN mvn clean package sonar:sonar -U -DargLine="-Dxxx=yyy"
you should use
RUN mvn clean package sonar:sonar -U -DargLine="@{argLine} -Dxxx=yyy"
I think that the surefire plugin will replace @{argLine} placeholder with the argline form JaCoCo plugin. At the end both properties will be used. Now jacoco.exec report is created in target dir and coverage reports are uploaded to Sonar server.
Missing language plugins
I dont know exactly what was the reason but somehow the plugins were remove from the SonarQube server. Then in the analysis following message was thrown from maven build. No quality profiles have been found, you probably don’t have any language plugin installed. To resolve it I had to re-install Java and XML language plugins that were used in the project form SonarQube marketplace.
