API Recorder

Overview

API Recorder is a tool that captures the traffic between API clients and API providers. For APIs exposed over HTTP it records each raw HTTP request and response into a pair of files.

Why is this useful?

definition

The recordings make it easy to create and configure API simulation simlets!

How It Works

API Recorder acts as a man-in-the-middle between API clients (like your application) and API providers (API dependencies). All requests and responses go through API Recorder, which relays them to the API providers and clients, respectively, after saving the traffic to files. What is recorded is the raw HTTP traffic including HTTP request and status lines, HTTP headers, and the original carriage return and line feed characters.

api-recorder-diagram

Modes of Operation

API Recorder supports two modes of operation:

  • Proxy, and

  • Router.

Proxy Mode

When API Recorder is running in proxy mode, API clients do not have to change the URLs where they submit their requests. The HTTP client library they use has to be configured to use the API Recorder as an HTTP proxy. The API client is unaware that requests are sent to the API Recorder first which then relays them to the API Provider’s endpoint.

Consult the documentation of your HTTP client for how to configure the use of HTTP proxy.

Router Mode

API Recorder in router mode routes the incoming requests to pre-configured host and port number for the API provider. An API client has to be configured to submit its requests to a running API Recorder and not to the API provider.

We have plans to support routing tables in the future to allow the routing of HTTP traffic to more than one host and port number with a single instance of API Recorder.

Support for TLS/SSL

Often times an API client communicates with API provider over HTTPS. That is - TLS/SSL-wrapped HTTP connection to secure the communications.

In either mode of API Recorder operation, an API client connects to API Recorder, which in turn establishes a connection with the API provider. When the connection has to be secured with TLS/SSL, API Recorder terminates the TLS/SSL from the API client and establishes a new TLS/SSL session with the API provider. The reason is that API Recorder must be able to get to the plain text HTTP traffic in order to record it.

This requires an X.509 certificate and the Private Key to be configured in API Recorder.

API Recorder supports key stores in the Java KeyStore format (.jks). The cert and its corresponding Private Key have to be imported into a Java KeyStore and its location passed in as command line argument (see below).

Support for other formats, like .pem, and mutual authentication is planned for the future.

How to Use API Recorder

Go to the API Simulator’s bin directory (replace the version, 1.1, and the actual location of the install as appropriate):

cd "/apisimulator/apisimulator-http-1.1/bin"

Start an API Recorder instance by executing the following:

Linux:

./apirecorder start <arguments> &

Windows:

apirecorder start <arguments>

To stop the API Recorder, execute this:

Linux:

./apirecorder stop <arguments>

Windows:

apirecorder stop <arguments>

Command-line Arguments

Argument Description

-admin_host <arg>

Admin server host. Useful when there are multiple network interfaces.

-admin_key <arg>

Value of the key/passphrase, if the admin server is secured.

-admin_port <arg>

Admin server port number

-d <arg>

Directory destination for the recordings.

-h <arg>

Recorder host. Useful when there are multiple network interfaces.

-H <arg>

Remote host name or IP address when in router mode.

-jks_file <arg>

Java KeyStore (JKS) file containing the TLS/SSL cert(s) to use. classpath:<KeyStoreFileName> will cause the file to be loaded from the classpath.

-jks_password <arg>

Password to access the Java KeyStore (JKS).

-key_password <arg>

Password for the key.

-m <arg>

Mode of capturing traffic. Expected value is proxy or router. Defaults to proxy if not specified.

-p <arg>

Recorder local port number.

-P <arg>

Port number for the remote server when in router mode.

-tls <arg>

TLS/SSL will be used if specified. The value denotes the format for the certificate and key - pem or jks (Java KeyStore). Currently pem is not supported.

Examples

Proxy Mode

Go to the API Simulator’s bin directory (replace the version, 1.1, and the actual location of the install as appropriate):

cd "/apisimulator/apisimulator-http-1.1/bin"

Start an API Recorder instance on port 8880 and configure the directory where the traffic files will be saved. This example reuses the logs directory but you should specify your own location:

Linux:

./apirecorder start -p 8880 -d "/apisimulator/apisimulator-http-1.1/logs" &

Windows:

apirecorder start -p 8880 -d "/apisimulator/apisimulator-http-1.1/logs"

Notice that we did not specify -m argument so the recorder by default will run in proxy mode. The internal admin server by default will listen on port 8881.

Let’s submit a request using cURL to get a jQuery UI CSS file that is hosted by Google. The --proxy option tells cURL to use our recorder as a proxy:

curl --proxy localhost:8880 -X GET "http://ajax.googleapis.com/ajax/libs/jqueryui/1.12.0/themes/smoothness/jquery-ui.css"

Look in the directory with the recordings - there should be two new files created for the raw request (input) and response (output).

Stop the API Recorder before continuing with the examples.

Proxy Mode and TLS/SSL

To demonstrate the use of TLS/SSL, let’s start up an API Recorder instance in the default HTTP proxy mode to listen on port 8880 for TLS/SSL connections and use the built-in Java KeyStore (JKS) that contains a self-signed certificate:

Linux:

./apirecorder start -p 8880 -d "/apisimulator/apisimulator-http-1.1/logs" -tls jks &

Windows:

apirecorder start -p 8880 -d "/apisimulator/apisimulator-http-1.1/logs" -tls jks

Let’s now submit a request using cURL to get the same jQuery UI CSS file as above but this time we’ll use https instead of http:

curl --proxy localhost:8880 -X GET "https://ajax.googleapis.com/ajax/libs/jqueryui/1.12.0/themes/smoothness/jquery-ui.css" -k

Notice the -k option - it tells cURL not to validate the certificate.

The directory with the recordings should contain four new files. The first two files (chronologically) are for the HTTP CONNECT request to the proxy (our API Recorder) to establish HTTPS tunnel and the 200 OK response; the next two files are for the actual raw request (input) to ajax.googleapis.com and its response (output).

Without restarting API Recorder, submit the same request but using http:

curl --proxy localhost:8880 -X GET "http://ajax.googleapis.com/ajax/libs/jqueryui/1.12.0/themes/smoothness/jquery-ui.css"

There should be two new recording files. This demonstrates how API Recorder correctly handles plain http even when it is running with support for TLS/SSL.

Stop the API Recorder before continuing with the examples.

Router Mode

Use the -m argument to start API Recorder up as a router:

Linux:

./apirecorder start -p 8880 -d "/apisimulator/apisimulator-http-1.1/logs" -m router -H ajax.googleapis.com -P 80 &

Windows:

apirecorder start -p 8880 -d "/apisimulator/apisimulator-http-1.1/logs" -m router -H ajax.googleapis.com -P 80

The command tells API Recorder to route to ajax.googleapis.com, port 80, any requests it receives.

Let’s now submit a request using cURL to fetch the jQuery UI CSS file as above but this time send the request to the API Recorder running on localhost, port 8880:

curl -X GET "http://localhost:8880/ajax/libs/jqueryui/1.12.0/themes/smoothness/jquery-ui.css"

The directory with the recordings should have two new files - one for the request (input) and one for the response.

Router Mode and TLS/SSL

TLS/SSL is configured similarly when API Recorder acts as a router.

We would love to hear your feedback! Shoot us a quick email to [feedback at APISimulator.com] to let us know what you think.

Happy API Simulating!