Android Appium Automated Tests with BrowserMob MITM proxy
In this blog
Test project
To better understand and learn this concept, I have developed the below sample application for which we are going to automate UI tests using Appium.
"Let's find you something to do." If you are getting bored, the app will share some unique and random finds that can help you get rid of your boredom. For this, we are using The Bored API.
Note: We are not touching on Android development for the above project in this article.
Appium project setup
For Appium, we are going to use Kotlin with Maven, and we are using Intellij IDEA CE for development. The selection of language and IDE can be different based on personal preference.
Let's start with creating a new Maven project.
Create a new Maven project with Kotlin as the language
Add dependencies in pom.xml
<dependency>
<groupId>io.appium</groupId>
<artifactId>java-client</artifactId>
<version>7.0.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.14.3</version>
<scope>test</scope>
</dependency>Setup base classes
Initially, we will setup some reusable base classes, which usually help when the project gets bigger and help keep the code clean.
Capabilities class
Base class with the required Appium configuration. Every test class will implement this so that we don't need to reconfigure Appium setup in all test classes.
package com.example.gotbored.base
import com.example.gotbored.utils.getFormattedDate
import com.example.gotbored.utils.getProjectCurrentPath
import io.appium.java_client.AppiumDriver
import io.appium.java_client.MobileElement
import io.appium.java_client.android.AndroidDriver
import io.appium.java_client.remote.AutomationName
import io.appium.java_client.remote.MobileCapabilityType
import org.junit.jupiter.api.TestInstance
import org.openqa.selenium.remote.DesiredCapabilities
import org.testng.annotations.*
import java.io.File
import java.net.URL
import java.util.*
import java.util.concurrent.TimeUnit
@Listeners(TestListener::class)
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
open class Capabilities {
@BeforeSuite
open fun setup() {
println("BeforeSuite setup...")
val capabilities = DesiredCapabilities()
val userDir = System.getProperty("user.dir")
val serverAddress = URL("http://127.0.0.1:4723/wd/hub")
val androidPlatformVersion = "14"
capabilities.setCapability(MobileCapabilityType.VERSION, "1.7.1")
capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "Android")
capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, AutomationName.ANDROID_UIAUTOMATOR2)
capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, androidPlatformVersion)
capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "Android")
capabilities.setCapability(
MobileCapabilityType.APP,
"${getProjectCurrentPath()}/app-debug.apk"
)
capabilities.setCapability("appPackage", "com.example.gotbored")
capabilities.setCapability("appActivity", "com.example.gotbored.regular.HomeActivity")
capabilities.setCapability("shouldTerminateApp", true)
capabilities.setCapability("forceAppLaunch", true)
driver = AndroidDriver(serverAddress, capabilities)
driver?.let {
it.manage()?.timeouts()?.implicitlyWait(30, TimeUnit.SECONDS)
}
}
@BeforeMethod
fun startApp() {
println("BeforeMethod Start App...")
driver?.launchApp()
}
@AfterMethod
fun closeApp() {
println("AfterMethod Close App...")
driver?.closeApp()
}
@AfterSuite
open fun tearDown() {
println("AfterSuite tearDown...")
}
companion object{
var driver: AppiumDriver<MobileElement>? = null
}
}TestListener class
Listener is defined as an interface that modifies the default TestNG's behaviour. As the name suggests, listeners "listen" to the event defined in the selenium script and behave accordingly.
We are implementing this to better track test behaviour with proper logging.
package com.example.gotbored.base
import com.aventstack.extentreports.Status
import com.aventstack.extentreports.markuputils.ExtentColor
import com.aventstack.extentreports.markuputils.MarkupHelper
import com.example.gotbored.base.Capabilities.Companion.driver
import com.example.gotbored.utils.copyScreenshot
import com.example.gotbored.utils.getProjectCurrentPath
import org.openqa.selenium.OutputType
import org.testng.ITestContext
import org.testng.ITestListener
import org.testng.ITestResult
import java.io.File
import java.text.SimpleDateFormat
import java.util.*
class TestListener : ITestListener {
override fun onTestStart(result: ITestResult) {
val testName: String = result.method.methodName
println("onTestStart...$testName")
Capabilities.extentTest = Capabilities.extentReport.createTest(testName)
}
override fun onTestSuccess(result: ITestResult) {
val testName: String = result.method.methodName
println("onTestSuccess...$testName")
try {
Capabilities.extentTest.log(
Status.PASS,
MarkupHelper.createLabel("$testName Test Case PASSED", ExtentColor.GREEN)
)
} catch (e: Exception) {
e.printStackTrace()
}
}
override fun onTestFailure(result: ITestResult) {
val testName: String = result.method.methodName
println("onTestFailure...$testName")
try {
val screenshotPath = driver!!.getScreenshotAs(OutputType.FILE)
val savedPath = copyScreenshot(screenshotPath, testName, Capabilities.reportPath)
Capabilities.extentTest.log(
Status.FAIL,
MarkupHelper.createLabel("$testName Test Case FAILED", ExtentColor.RED)
)
Capabilities.extentTest.log(
Status.FAIL,
MarkupHelper.createLabel("Reason for Failure: " + result.throwable.toString(), ExtentColor.RED)
).addScreenCaptureFromPath(savedPath)
} catch (e: Exception) {
e.printStackTrace()
}
}
override fun onTestSkipped(result: ITestResult) {
val testName: String = result.method.methodName
println("onTestSkipped...$testName")
try {
Capabilities.extentTest.log(
Status.SKIP,
MarkupHelper.createLabel("$testName Test Case SKIPPED", ExtentColor.ORANGE)
)
} catch (e: Exception) {
e.printStackTrace()
}
}
override fun onTestFailedButWithinSuccessPercentage(result: ITestResult) {
}
override fun onStart(p0: ITestContext?) {
}
override fun onFinish(testContext: ITestContext?) {
println("onFinish...")
}
}BasePage class
Base class to initialize Android elements.
package com.example.gotbored.base
import io.appium.java_client.AppiumDriver
import io.appium.java_client.pagefactory.AppiumFieldDecorator
import org.openqa.selenium.support.PageFactory
abstract class BasePage(driver: AppiumDriver<*>) {
init {
PageFactory.initElements(AppiumFieldDecorator(driver), this)
}
}CommonUtils class
Utils class contains some commonly required functions.
package com.example.gotbored.utils
import org.apache.commons.io.FileUtils
import java.nio.file.Paths
import org.apache.commons.io.IOUtils
import java.io.BufferedReader
import java.io.File
import java.lang.Exception
import java.text.SimpleDateFormat
import java.util.*
fun getProjectCurrentPath() = Paths.get("").toAbsolutePath().toString()
fun getProjectSrcPath() =
Paths.get("").toAbsolutePath().toString() + "/src/test/kotlin/com/example/gotbored/"Setup test cases
Let's set up actual Appium tests to validate application behavior.
HomePage class
Page containing all required UI elements with some validation functions to validate data. These functions will be called from the Test class.
package com.example.gotbored.home
import com.example.gotbored.base.BasePage
import io.appium.java_client.AppiumDriver
import io.appium.java_client.MobileElement
import io.appium.java_client.pagefactory.AndroidFindBy
import org.junit.Assert
class HomePage(driver: AppiumDriver<*>) : BasePage(driver) {
@AndroidFindBy(id = "activityInfoView")
private lateinit var activityInfoView: MobileElement
@AndroidFindBy(id = "activityNameTV")
private lateinit var activityName: MobileElement
@AndroidFindBy(id = "activityTypeTV")
private lateinit var activityType: MobileElement
@AndroidFindBy(id = "participantsTV")
private lateinit var participants: MobileElement
@AndroidFindBy(id = "errorView")
private lateinit var errorView: MobileElement
@AndroidFindBy(id = "errorText")
private lateinit var errorMessage: MobileElement
@AndroidFindBy(id = "retryButton")
private lateinit var retryButton: MobileElement
fun validateActivityDetails() {
Assert.assertTrue(activityInfoView.isDisplayed)
Assert.assertEquals("Some dummy social activity!!!", activityName.text)
Assert.assertEquals("Social", activityType.text)
Assert.assertEquals("2", participants.text)
}
fun validateErrorPage() {
Assert.assertTrue(errorView.isDisplayed)
Assert.assertEquals("Something went wrong!\nPlease try again.", errorMessage.text)
Assert.assertTrue(retryButton.isDisplayed)
}
}HomeTest class
Test class for validating happy use case.
package com.example.gotbored.home
import com.example.gotbored.base.Capabilities
import com.example.gotbored.utils.getStubActivityInfoResponse
import org.testng.annotations.BeforeSuite
import org.testng.annotations.Test
class HomeTest : Capabilities() {
@BeforeSuite
override fun setup() {
super.setup()
}
@Test
fun `validate activity details`() {
val page = HomePage(driver!!)
page.run {
validateActivityDetails()
}
}
}HomeErrorTest class
Test class for validating negative use case.
package com.example.gotbored.home
import com.example.gotbored.base.Capabilities
import com.example.gotbored.utils.getStubActivityInfoResponse
import org.testng.annotations.BeforeSuite
import org.testng.annotations.Test
class HomeErrorTest : Capabilities() {
@BeforeSuite
override fun setup() {
super.setup()
}
@Test
fun `validate error page when API fails`() {
val page = HomePage(driver!!)
page.run {
validateErrorPage()
}
}
}We are all set up now to run our test cases.
Run the test cases
Before running the test case, start the Appium server. But our test fails every time we run because the data, we receive from the API is random, and with each request, it is going to always change.
So, what to do now? How can we automate this so that the test suite will always receive similar responses?
The solution is to use a man-in-the-middle proxy concept.
MITM
A man in the middle (MITM) attack is a general term for when a perpetrator positions himself in a conversation between a user and an application—either to eavesdrop or to impersonate one of the parties, making it appear as if a normal exchange of information is underway.
Man-in-the-middle proxy
A proxy of this sort is a program that we insert between the device and the internet. We tell the device to route all requests through our proxy, which records the traffic as it passes through. Not only does it record it, but the proxy we will be using can also be configured to modify the traffic. This opens up more potential uses in testing: we can test the failure cases when certain URLs can't be reached, and we can modify responses to suit our needs for consistent data.
With a proxy, we can even modify the network responses our device receives, allowing us to test with fewer random values. We can also use this technique to force the app into a particular state we want to test. All from within our test code, involving no changes to the app.
For this purpose, we are going to use "Browsermob Proxy."
BrowserMob proxy integration and setup
BrowserMob Proxy allows you to manipulate HTTP requests and responses and capture HTTP content.
Steps to integrate the BrowserMob proxy.
Add dependency
<dependency>
<groupId>net.lightbody.bmp</groupId>
<artifactId>browsermob-core</artifactId>
<version>2.1.5</version>
<scope>test</scope>
</dependency>Helper class to configure proxy
package com.example.gotbored.base
import com.example.gotbored.utils.getStubActivityInfoResponse
import io.netty.handler.codec.http.HttpMethod
import io.netty.handler.codec.http.HttpResponse
import io.netty.handler.codec.http.HttpResponseStatus
import net.lightbody.bmp.BrowserMobProxy
import net.lightbody.bmp.BrowserMobProxyServer
import net.lightbody.bmp.core.har.Har
import net.lightbody.bmp.proxy.CaptureType
import net.lightbody.bmp.util.HttpMessageContents
import net.lightbody.bmp.util.HttpMessageInfo
class BrowserMobHelper {
fun stopProxy() {
if (proxy != null) {
proxy!!.stop()
proxy = null
println("=== Proxy is stopped ===")
}
}
fun startRecording(): BrowserMobHelper {
proxy!!.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT)
proxy!!.newHar()
return this
}
fun stopRecording(): Har? {
return if (proxy!!.har != null) {
proxy!!.endHar()
} else null
}
companion object {
private val ourInstance = BrowserMobHelper()
private var proxy: BrowserMobProxy? = null
val instance: BrowserMobHelper
get() {
if (proxy == null) {
proxy = BrowserMobProxyServer()
(proxy as BrowserMobProxyServer).setTrustAllServers(true)
(proxy as BrowserMobProxyServer).start(9999)
println("=== Proxy has been started ===")
}
return ourInstance
}
}
}Overwriting response
There are 4 methods to support request and response interception:
- addRequestFilter
- addResponseFilter
- addFirstHttpFilterFactory
- addLastHttpFilterFactory
See more details in "HTTP Request Manipulation" section of BMP docs. For demo purposes, we'll use addResponseFilter.
Create fun within the BrowserMobHelper class, which will be used to override API responses.
var getActivityResponse = Pair(false, "")
fun overwriteApiResponse(): BrowserMobHelper {
println("overwriteApiResponse...")
proxy!!.addResponseFilter { response: HttpResponse?, contents: HttpMessageContents, messageInfo: HttpMessageInfo ->
println("***** Overriding http response *****")
println("URL - ${messageInfo.url}")
println("Response - ${response!!.status}")
when {
messageInfo.originalRequest.method == HttpMethod.GET && messageInfo.url.endsWith("boredapi.com/api/activity") -> {
if (getActivityResponse.first) {
response.status = HttpResponseStatus(400, "")
} else {
response.status = HttpResponseStatus(200, "")
}
contents.textContents = getActivityResponse.second
}
else -> {}
}
}
return this
}We have created the "getActivityResponse" variable, which we will use to set the required response for matched API responses, and this data will be overridden on API responses.
Setup a browsermob proxy in Appium
Create a public instance.
companion object{
…
val bmpHelper: BrowserMobHelper = BrowserMobHelper.instance
}Initialize the browsermob proxy once AppiumDriverInstance is initiated.
driver = AndroidDriver(serverAddress, capabilities)
driver?.let {
it.manage()?.timeouts()?.implicitlyWait(30, TimeUnit.SECONDS)
bmpHelper.overwriteApiResponse().startRecording()
}Don't forget to stop recording.
@AfterSuite
open fun tearDown() {
println("AfterSuite tearDown...")
bmpHelper.stopRecording()!!.writeTo(File("sample.har"))
bmpHelper.stopProxy()
}Our MITM proxy is set up, and we are ready to stub API responses.
Setup dummy response to stub
As we already know what kind of JSON data, we usually receive for an API call, let's create a similar JSON file with some dummy responses.
{
"activity": "Some dummy social activity!!!",
"type": "Social",
"participants": 2,
"price": 0,
"link": "",
"key": "1718657",
"accessibility": 0.5
}We need to process the above JSON file and set this response for API.
Create a util function and call this to override getActivityResponse in the test class.
fun readJSONFromFile(filePath: String): String? {
val bufferedReader: BufferedReader = File(filePath).bufferedReader()
return bufferedReader.use { it.readText() }
}
fun getStubActivityInfoResponse(): String {
return readJSONFromFile("${getProjectSrcPath()}/home/activityInfo.json")!!
}And our final test looks like…
package com.example.gotbored.home
import com.example.gotbored.base.Capabilities
import com.example.gotbored.utils.getStubActivityInfoResponse
import org.testng.annotations.BeforeSuite
import org.testng.annotations.Test
class HomeTest : Capabilities() {
@BeforeSuite
override fun setup() {
super.setup()
bmpHelper.getActivityResponse = Pair(false, getStubActivityInfoResponse())
}
@Test
fun `validate activity details`() {
val page = HomePage(driver!!)
page.run {
validateActivityDetails()
}
}
}Error test:
package com.example.gotbored.home
import com.example.gotbored.base.Capabilities
import com.example.gotbored.utils.getStubActivityInfoResponse
import org.testng.annotations.BeforeSuite
import org.testng.annotations.Test
class HomeErrorTest : Capabilities() {
@BeforeSuite
override fun setup() {
super.setup()
bmpHelper.getActivityResponse = Pair(true, "")
}
@Test
fun `validate error page when API fails`() {
val page = HomePage(driver!!)
page.run {
validateErrorPage()
}
}
}Before we jump into a running test, we need to setup the device to use a proxy.
Device configuration
Install CA certificate
- Download below CA certificate from repo.
- Copy both certificate files to the root of the /sdcard folder inside your Android device.
- Inside the device, go to Settings -> Security & Privacy -> More Security & Privacy -> Encryption & Credentials -> Tap on Install a Certificate -> Select CA certificate -> Select Install anyway -> Select ca-certificate-rsa.cer file and install.
- Follow the above process for ca-certificate-ec.cer to install on the device.
- Note: Before you install the certificate, a PIN might be prompted. You'll have to create one or use the existing one.
Configure proxy
- Verify that the proxy host and device are on the same network.
- Get a proxy host IP.
- Go to the device's settings and wifi configuration.
- Press and hold on the network you're going to use, and select Modify network from the alert modal.
- Then check Advanced Options and scroll until you see Proxy.
- Tap the Proxy dropdown and select Manual.
- For the proxy hostname, input your proxy's IP address.
- Then, for the proxy port use 9999.
- Tap Save.
Configure the test app
- Specify the res/xml/network_security_config.xml file in the Android app.
- Update AndroidManifest.xml to use network_security_config.
- Another available option is to setup a custom trust manager.
- Rebuild the app and copy the updated APK to the required place.
Run the test
The test now passes every time we run with stub data.
Few more automation tips
While developing this I ended up automating few more things like…
Android platform version
Capabilities "PLATFORM_VERSION" will always change based on which device we use for testing.
To dynamically determine the version, we are using the adb tool with ProcessBuilder. ProcessBuilder helps run the adb command, and the output we get will be the device platform version.
fun getAndroidDeviceVersion(): String {
var version = 12
try {
val processBuilder = ProcessBuilder("adb", "shell", "getprop", "ro.build.version.release")
val process: Process = processBuilder.start()
process.waitFor()
val stdOut = IOUtils.toString(process.inputStream, Charsets.UTF_8)
version = stdOut.trim().toInt()
} catch (_: Exception) {
}
return version.toString()
}Extent report generation
Once the "Extent Report" tool is integrated, we can use "onFinish" of "TestListener" to save reports at a designated place with proper date and time information.
Note: please refer to this article to follow the steps for integrating the extent report.
override fun onFinish(testContext: ITestContext?) {
println("onFinish...")
try {
Capabilities.extentReport.flush()
val dateFormat = SimpleDateFormat(" dd-MMM-yyyy_HH-mm-ss")
val date = Date()
val filePathDate: String = dateFormat.format(date).toString()
val actualReportPath = "${getProjectCurrentPath()}/AutomationReport/index.html"
File(actualReportPath).renameTo(
File(
"${getProjectCurrentPath()}/AutomationReport/Automation_Report_$filePathDate.html"
)
)
} catch (e: Exception) {
e.printStackTrace()
}
}
Conclusion
We have successfully integrated BMP into the Appium testing framework and hope this will help automate testing using Appium. BMP is not the only proxy tool available; there are others that can be used.
More on Appium
It's an open-source mobile User Interface (UI) automation tool for testing native apps (Android and iOS) and mobile browsers. To know more about this and also to understand and correlate with this article, I would recommend everyone read Appium Mobile Automation Testing - WWT. This document covers everything we need to setup Appium and how to use it.