Reproxy: A Simple Yet Powerful Mock Server for Mobile App Development

Photo by: Christina Morillo

Editor’s Note: Today we hear from Ahmad Fadli, one of our Android Engineers with the Experience team on how he and his front end teammates attempted various approaches to decouple themselves from back end APIs dependency, from mocking those APIs responses in-app to creating a dedicated external mock server for the app to connect to, in order to maintain their application testing workflow uninterrupted and independent with minimal to no change in the app’s code.

The Experience team at Traveloka is responsible for various lifestyle products in the Traveloka ecosystem such as Traveloka Xperience and Movies and one of the cornerstone groups driving Traveloka forward.

Consuming API provided by Back End (BE) is a common Front End (FE) task for both web and mobile app engineers. Here in Traveloka is no exception. When working on such a task, we would prefer an ideal situation, where all the APIs are constantly accessible. Unfortunately, as much as we love such a favorable situation, it just does not happen every time.

In this article, we will walk you through the journey of how we minimized our API dependency with BE and ended up creating a tool that gives us flexibility and independence during testing.

Defining the Contract

First things first, all the approaches below require us to establish an API contract before any development can start, which includes the API endpoint, request, and response specifications. Both FE and BE teams sit down together for an API Contract Discussion to hammer down the draft of the specification in order to reach a mutually agreeable contract.

Once the API contract is locked, both BE and FE teams can start implementing the specifications on their respective environment in parallel. When both implementations are completed on both sides, the integration process can then begin.

But, there is a problem.

Imagine being a mobile app engineer working on a feature inside Traveloka Xperience product with the following development workflow:

1. UI rendering ✅
2. The mapping between API model to View Model ✅
3. Feature logic implementation ✅
4. Testing 🤔

After completing the first three tasks, we then need to test the mobile app before handing it over to the QA team for quality check. But unfortunately in this instance, the API implementation by the BE team is not yet ready or the staging machine is down. Either case, we’re blocked.

If we don’t do anything, our daily stand-up will have reports of our progress being blocked until BE delivers. So, like all great engineers, we need to unblock ourselves. But how?

Automated Testing

One way is to create tests for our code and mock the API response. But, such testing only works to some extent because despite verifying our logic, unit testing will not verify UI correctness. Moreover, we can’t give the mobile app yet to the QA team because they will need to test the UI and functionality manually. We need something else to complement the tests.

Changing the Value in Runtime

The idea is simple. We change the returned value of the API in runtime. We can do this by setting a breakpoint just before consuming the API response and modify the value of the variable to our needs.

Pros ✅

1. Quick to do for simple value changes.
2. Needless app rebuilds upon change of value.
3. Extremely flexible. We can choose which field or endpoint that we want to change.

Cons 🔴

1. Unusable when the API is unreachable.
2. Hassle to do for big modification.
3. Requires additional context for the QA team to fully understand where to set the breakpoints.
4. We need to make the fields mutable in order to change it in the runtime.

Although we will still be able to do our testing given the availability and accessibility of the required APIs, there are some drawbacks to this approach. One of which is that we are not able to edit the response of a non-accessible API, which brings us to the next approach.

Abstracting the API Layer

Abstracting the API layer is something that we should have done from the start. We create an abstraction layer so that we can change the implementation when needed such as in the event of running Unit Tests.

Abstracting API caller implementation

P.S: Here we instantiate ExperienceApi directly in the presenter which is violating IoC concept. This is done for simplicity. In production-level code, it should be injected via constructor (i.e. Dependency Injection)

P.P.S: Also depending on your architecture, you might want to inject the ExperienceApi through another layer (e.g. Repository) before accessing it directly in the Presenter

We can easily switch between mock (ExperienceApiMock) and real implementation (ExperienceApiImpl) without changing the rest of the code

Pros ✅

1. Usable even when the API is unreachable.
2. Hassle-free for either small or large modification.
3. We don’t have to make the fields mutable

Cons 🔴

1. Additional context is still required by QA to know where to change the code. Moreover, QA will have to code both in Swift and Kotlin when they want to change the mock in both iOS and Android.
2. Partial mocking is not possible. For example, we want getProduct to be mocked but getProductList to call the real API instead.
3. For every change in mock response requires app rebuild. Build time can be significant for a big project.

Let’s address each of those three cons in the same sequence:

Solutions
1. Instead of creating the response from native code, we read from a local JSON. QA then can just edit this JSON file, which could also be reused for Android and iOS.

Read mock from local JSON

2. Introduce a simple flag for each API endpoint.

Provide flag for each endpoint

ExperienceApiMock serves as a router that directs requests to either BE API or reads from a local JSON file, depending on the configuration.

Here is the diagram of how this approach works:

Mock API act as a router

3. This one is tricky. For every change in ExperienceApiMock such as changing the JSON content or turning on/off specific endpoints, we need to rebuild the app, which takes approximately three minutes of build time in the case of our Android app.

Read “Dagger and Multi-module Traveloka Android App” on how we were able to reduce the build time to just one minute.

We figure out that if we want to avoid the app rebuild, we need to build the mock outside of the app’s code by creating a mock server. This is how Reproxy was born.

Creating Reproxy — A Mock Server

The idea is pretty simple. We just move ExperienceApiMock outside of the app and voila, we solve cons #3 above.

Before we create Reproxy, we ensure that all the benefits that we have in the previous approaches are still intact, which includes:

• Easy to change by engineer and QA.
• Usable mock response for both Android and iOS.
• Support for partial mocking.

On top of that, we also set these requirements for Reproxy:

• Minimum or no change at all in the app’s code to switch between mock and non-mock.
• Easy to set up and run.

The Engine: Express JS

The primary goal of Reproxy is to help engineers and QA team in performing FE-testing tasks without BE dependency. So, we made Reproxy run on our local machine powered by Express JS as the web-server.

The Mechanism: Rule-based Response

Although we use Express JS, we don’t want to create the router logic in JavaScript (JS) for several reasons:

• Not everyone knows JS; creating an entry-barrier for using the tools.
• The configuration is not easily shareable among engineers.
• The server needs to be restarted every time we make any change to the configuration.

As a solution, we decided to create the router logic in JSON. We have one file called rules.json, which has a list of rules and each rule, in turn, has individual condition and response. When the condition is met, the response will be returned.

JSON Example of rule-based response

The configuration above will first check whether the URL is experience/product. If yes, then it will return the experience/product.json file. Else, it will redirect the request to traveloka.com/real-api.

And that’s it. The entire concept is basically a series of if-else statements typed in JSON format.

We defined several condition types for common usage as follows:

• ANY: Match anything.
• ANY_OF: Match at least one.
• COMPOSITE_AND: Match all the conditions.
• URL_MATCH: Match the URL exactly.
• URL_PREFIX: Match the URL prefix (useful for URL with query strings).

And likewise for response types, as such:

• JSON_FILE: Read local JSON file.
• TEXT: Return anything inside the value.
• REDIRECT: Redirect the request to the given URL.
• DELAY: Delay the response (best used with SEQUENCE).
• SEQUENCE: Run multiple responses.

One good example of using SEQUENCE and DELAY is to give some delay before returning a particular value so that we can test the loading state in the app’s page:

Combining DELAY and SEQUENCE to test app loading state

Now what we need to do is to make Reproxy understand how to read these rules. There is nothing fancy here; just a simple loop that looks for first matching-rule and then creates the response based on that matching rule:

The core logic of Reproxy

With all the pieces set in place, all we need to do next is to redirect the app to call Reproxy instead of the real API server.

Changes Needed in the App’s Code

With the preference for minimal change in the app’s code as mentioned before, with Reproxy, we only need to change the base URL.

Instead of calling traveloka.com/real-api (not a real API endpoint), we change the base URL to IP_ADDRESS:3000/run and that’s it. No need to add anything to the header nor request’s payload.

This IP address is the public address of the machine running Reproxy. If we run it on our local machine instead, so that we can connect from an emulator/simulator, then we can use the following default IP addresses: 10.0.2.2 for android emulator and 127.0.0.1 for iOS simulator.

If we want to use a real device, then we need to connect both the server and the device in the same network and use the local IP address of the server.

Pro Tips: In Traveloka debug app, in order to negate rebuilding the app, we have built the functionality to change the base URL in runtime when switching between Reproxy and any other server.

That’s it! We’ve achieved all the requirements for utilizing Reproxy:

• Easy JSON configuration for the various response types.
• Ability to do partial mocking by utilizing REDIRECT.
• Only one change required in-app: changing the BASE URL.
• Just one command line in Terminal to run Reproxy (courtesy of Express JS).

Now, we can develop and test our features confidently even when BE has not finished their API implementation.

Evolving Reproxy: More Than Just a Simple Stub Server

Although we have achieved all the original goals of creating Reproxy, other use cases started to surface as more people try out the tools with questions or feedback such as:

“Can I differentiate responses based on the value in the query strings?”
“I need to have a different response depending on the request payload.”
“I wanted to call real API and inject a number of new fields to the response.”

It might be obvious that we need to add several new condition types like QUERY_STRING_MATCH & REQUEST_MATCH. But how do we make this scalable? Writing new types for every new use case in one JS file will surely bloat the file up in no time.

Introducing JS_SCRIPT Type

Instead of writing all logic in the responseCreator.js file, we introduce a new type that will execute a JS file based on the declared arguments (i.e. delegation pattern):

When this particular rule is checked by the router, it will call queryStringMatch.js and pass anything inside the arguments field. It is up to the creator of the script on how to use the values of those arguments.

Next, we create the queryStringMatch.js

And the final step, we add new type handler in our responseCreator.js

Now anyone can extend the functionality of the tools without modifying Reproxy’s core files.

Better Grouping: RuleSet Response

We are now able to prevent responseCreator.js from getting bloated by future modification of the tools. But, we still define all the rules in one file; the rules.json file.

As we add new configurations for several products such as Flight, Accommodation, or Experience, we need a sustainable way to make rules.json file manageable.

Enter RULE_SET Response Type.

RuleSet is a way to define a collection of rules. rules.json is one example of a RuleSet. With RULE_SET type, we can set a RuleSet as a response:

And in experienceRules.json:

Finally, in our responseCreator.js file, when we hit a RULE_SET response type, it will just call the same function again, but with a new RuleSet.

Other Use Cases

We now have managed to create a simple yet powerful mocking tool. It provides basic functionality like a regular stub server. But it can be as complex as needed by utilizing the JS_SCRIPT type. So what are the other use cases?

Mock for Automation Test

Our team of Software Development Engineer in Test (SDET) is running their automation tests on our staging server. They have several pain points:

• The staging server can go down unexpectedly resulting in flaky results for some tests.
• Some corner cases are hard to reproduce. For example, search results that return only one item or a product detail which only have one image in the photo gallery.

Reproxy sounds like a good solution to address those pain points mentioned above. We can just use it as a backup plan when the staging server is down. And for those corner cases, we can just create a JSON file that satisfies the requirements.

We are now piloting this project and will share the result in the future installment of this topic on what would have actually worked and not worked.

Mock for Stakeholders to Test Unreleased Feature

When our Product Managers or Design Team wants to test the new feature we are working on, we can showcase the feature to them even when BE is not yet ready by deploying Reproxy to a machine that is accessible from Traveloka’s internal network.

But that means we can’t use hard-coded rules.json file anymore as the entry point, since many features are being developed at the same time and each of these rules requires different RuleSet configurations.

Our solution is to make the RuleSet entry point as a part of the Base URL. That means, instead of calling IP_ADDRESS:3000/run, we change it to IP_ADDRESS:3000/run/{ruleSetName}

With this new setup, when someone wants to get early access to our feature, we can just say:
“Oh just change the Base URL to IP_ADDRESS:3000/run/experienceShinyNewFeature“

Thanks for reading until the end! Although we are mobile app engineers, it doesn’t mean that we should limit our scope of work to the domain of just mobile app development.

Great software engineers need to be creative in how to unblock themselves from their dependencies. If that sentence resonates very well with you, maybe you will have lots of fun as we are in creating a world-class mobile app with us! Visit our careers page or drop me a message if you are interested.