Tests

We’ve created a JSON Editor that has full semantic and syntax check so that it’s as easy as possible to define a test case.

You have the possibility of:

  • saving your work locally to a JSON file for later use

  • loading your work from a JSON file

  • copying a test case that on your colleague’s made and reuse it’s logic

Permissions

Member role vs API call

List tests

Start test

Cancel test

Get test details

Set test sharing

Reader

True

False

False

True

False

Editor

True

True

True

True

False

Admin

True

True

True

True

True

Owner

True

True

True

True

True

Test properties

The properties that can set up for yout tests are:

  1. num_clients:

    • Property type: MANDATORY

    • Value type: Integer

    • Explanation: Number of clients to spawn per EACH thread

    • Restrictions: Minimum is 1 and Maximum is 250

  2. hatch_rate:

    • Property type: MANDATORY

    • Value type: Integer

    • Explanation: Rate of clients to spawn per EACH thread. It helps you define if all virtual users should start for each thread should start at once, or if they should progressively be launched at 1-second intervals

    • Restrictions: Should be equal or less to “num_clients” value, but higher than 1

  3. run_time:

    • Property type: MANDATORY

    • Value type: Integer

    • Explanation: How long should the test run for in seconds

    • Restrictions: Minimum is 1 (second) and Maximum is 3600 (seconds), depending on your subscription type

  4. threads_per_region:

    • Property type: MANDATORY

    • Value type: Integer

    • Explanation: The number of containers to spawn in paralel in each region

    • Restrictions: Minimum is 1 and Maximum is 25, depending on your subscription type

  5. domain_name:

    • Property type: MANDATORY

    • Value type: String

    • Explanation: Domain name to run the test against

    • Restrictions: Must be a domain that is already validated or sub-domain of a domain that is already validated

  6. protocol:

    • Property type: MANDATORY

    • Value type: String

    • Explanation: Protocol to run the test against

    • Restrictions: Can be either “http” or “https”

  7. min_wait:

    • Property type: OPTIONAL (default value is 1000)

    • Value type: Integer

    • Explanation: Minimum time for clients to wait before doing another request in milliseconds

    • Restrictions: Minimum value is 0 (milliseconds) and Maximum value is 10.000 (milliseconds) (min = 0 and max = 10000)

  8. max_wait:

    • Property type: OPTIONAL (default value is 1000)

    • Value type: Integer

    • Explanation: Maximum time for clients to wait before doing another request in milliseconds

    • Restrictions: Minimum value is 0 (milliseconds) and Maximum value is 10.000 (milliseconds), but definitely higher than min_wait

  9. test_region:

    • Property type: MANDATORY

    • Value type: Array of Strings

    • Explanation: Array of one or more regions to run the test in

    • Restrictions: Values must include one or more of the following

["ap-northeast-1", "ap-northeast-2", "ap-southeast-1", "ap-southeast-2", "eu-central-1", "eu-west-1", "eu-west-2", "eu-west-3", "us-east-1", "us-east-2", "us-west-1", "us-west-2", "ca-central-1", "ap-south-1", "sa-east-1"]
These values emulate the following regions:
ap-northeast-1 = Tokyo
ap-northeast-2 = Seoul
ap-southeast-1 = Singapore
ap-southeast-2 = Sydney
eu-central-1 = Frankfurt
eu-west-1 = Ireland
eu-west-2 = London
eu-west-3 = Paris
us-east-1 = North Virginia
us-east-2 = Ohio
us-west-1 = North California
us-west-2 = Oregon
ca-central-1 = Canada (Central)
ap-south-1 = Mumbai
sa-east-1 = Sao Paulo
  1. workflow:

    • Property type: MANDATORY

    • Value type: Array of Objects

    • Explanation: Array of object-based steps to execute sequentially

    • Object properties:

      10.1. path:

      • Property type: MANDATORY

      • Value type: String

      • Explanation: Path to test against

      • Restrictions: Must start with a “/”

      10.2. method:

      • Property type: MANDATORY

      • Value type: String

      • Explanation: Method to test against

      • Restrictions: Must be one of [“GET”, “POST”, “PATCH”, “HEAD”, “PUT”, “DELETE”, “OPTIONS”]

      10.3. data:

      • Property type: OPTIONAL (default is a blank string)

      • Value type: String

      • Explanation: Data to include as parameters in the body of the request (aka payload)

      10.4. headers:

      • Property type: OPTIONAL (default is a blank object)

      • Value type: Key-Value object

      • Explanation: Any key-value pair to include as headers in the request

      10.5. extract:

      • Property type: OPTIONAL (default is a blank object)

      • Value type: List of key-value objects

      • Explanation: A list of values to extract from either BODY or HEADERS to reuse in future steps of workflow

      10.5.1. parameter_name:

      • Property type: REQUIRED

      • Value type: String

      • Explanation: The name under which to store this value

      10.5.2. location:

      • Property type: REQUIRED

      • Value type: String

      • Explanation: The location from which to fetch this value (can be either “body” or “headers”)

      10.5.3. key:
      • Property type: OPTIONAL

      • Value type: String

      • Explanation: If this property is specified, the platform will check for that specific “KEY” in either the body (if it’s a JSON) or the headers, depending on the value set up for “location”

      • Note: You can either specify “key” or “element_with_regex”, but not both!

      10.5.4. element_with_regex:
      • Property type: OPTIONAL

      • Value type: String

      • Explanation: If this property is specified, the platform will check for that specific “KEY” in either the body (if it’s a JSON) or the headers, depending on the value set up for “location”

      • Note: You can either specify “key” or “element_with_regex”, but not both!

Good to know!

Reusing responses in workflow steps

Yes, it is possible!

ALl you have to do is properly define the _extract_ field in the workflow step and afterwards you can reuse it in any future step, in either the payload or the headers through a bash-like notation -> ${parameter-name-here}.

Here’s a sample:

"workflow": [
    {
        "path": "/login_path",
        "method": "POST",
        "headers": {
            "Content-Type": "application/x-www-form-urlencoded"
        },
        "data": "user=user&password=pass",
        "extract": [
            {
                "parameter_name": "authtoken",
                "location": "body",
                "key": "access_token"
            }
        ]
    },
    {
        "path": "/my/profile",
        "method": "GET",
        "data": "",
        "headers": {
            "Authorization": "Bearer ${authtoken}"
        }
    }
]

The code above extracts the value of access_token from the respone body, stores it in the parameter named authtoken, and then reuses it in the next step inside the headers block by using a bash-like syntax for ${authtoken}.

Makes sense now ? :)

List tests

Web

In order to view the tests on our platform, all you have to do is browse to the Dashboard page.

API call

The API call is:

curl -s \
    -d '{"team_id":"my-team"}' \
    -H "X-Api-Key: my_api_key" \
    -X POST \
    https://api.rungutan.com/v1/api/tests/list | jq . -r

{
  "Tests": [
    {
        "test_id": "my_test_id",
        "domain_name": "my-team.com",
        "test_status": "FINISHED",
        "threads_per_region": 2,
        "num_clients": 30,
        "hatch_rate": 30,
        "created_date": "2020-03-26T09:08:15Z",
        "run_time": "30s",
        "member_email": "[email protected]",
        "sharing_type": "PRIVATE",
        "test_region": "ap-southeast-1,ap-southeast-2",
        "test_type": "MANUAL"
    },
    {
        "test_id": "my_test_id_2",
        "domain_name": "my-team.com",
        "test_status": "FINISHED",
        "threads_per_region": 5,
        "num_clients": 60,
        "hatch_rate": 20,
        "created_date": "2020-03-29T10:12:34Z",
        "run_time": "60s",
        "member_email": "[email protected]",
        "sharing_type": "PRIVATE",
        "test_region": "us-east-1,us-east-2",
        "test_type": "MANUAL"
    }
}

You can also filter based on test_id to generate the summary for only that specific test_id:

curl -s \
    -d '{"team_id":"my-team", "test_id": "my_test_id"}' \
    -H "X-Api-Key: my_api_key" \
    -X POST \
    https://api.rungutan.com/v1/api/tests/list | jq . -r

{
  "Tests": [
    {
        "test_id": "my_test_id",
        "domain_name": "my-team.com",
        "test_status": "FINISHED",
        "threads_per_region": 2,
        "num_clients": 30,
        "hatch_rate": 30,
        "created_date": "2020-03-26T09:08:15Z",
        "run_time": "30s",
        "member_email": "[email protected]",
        "sharing_type": "PRIVATE",
        "test_region": "ap-southeast-1,ap-southeast-2",
        "test_type": "MANUAL"
    }
}

CLI

List all tests:

$ rungutan tests list
{
    "Tests": [
        {
            "test_id": "ad837abd-15d4-47ce-a344-01a3323bdd09",
            "test_name": "wtf",
            "domain_name": "rungutan.com",
            "test_status": "CANCELLED",
            "threads_per_region": 1,
            "num_clients": 30,
            "hatch_rate": 30,
            "created_date": "2020-05-19T12:04:42Z",
            "run_time": 30,
            "member_email": "[email protected]",
            "sharing_type": "PRIVATE",
            "test_region": "us-east-1",
            "test_type": "MANUAL"
        },
        {
            "test_id": "15fa78b9-097b-401a-bedb-a70ddc4726ed",
            "test_name": "Sample worklfow test",
            "domain_name": "rungutan.com",
            "test_status": "FINISHED",
            "threads_per_region": 1,
            "num_clients": 30,
            "hatch_rate": 30,
            "created_date": "2020-05-19T11:56:19Z",
            "run_time": 30,
            "member_email": "[email protected]",
            "sharing_type": "PRIVATE",
            "test_region": "us-east-1",
            "test_type": "MANUAL"
        }
    ]
}

Filter based on test_id:

$ rungutan tests list --test_id ad837abd-15d4-47ce-a344-01a3323bdd09
{
    "Tests": [
        {
            "test_id": "ad837abd-15d4-47ce-a344-01a3323bdd09",
            "test_name": "wtf",
            "domain_name": "rungutan.com",
            "test_status": "CANCELLED",
            "threads_per_region": 1,
            "num_clients": 30,
            "hatch_rate": 30,
            "created_date": "2020-05-19T12:04:42Z",
            "run_time": 30,
            "member_email": "[email protected]",
            "sharing_type": "PRIVATE",
            "test_region": "us-east-1",
            "test_type": "MANUAL"
        }
    ]
}

Start test

Web

In order to start a new test, all you have to do is browse to the Dashboard page and click on the Start test button.

API call

The API call is:

curl -s \
    -d '{
        "team_id": "my-team",
        "num_clients": 30,
        "hatch_rate": 30,
        "run_time": 30,
        "threads_per_region": 1,
        "domain_name": "my-team.com",
        "protocol": "https",
        "min_wait": 1000,
        "max_wait": 1000,
        "test_region": [
            "us-east-1"
        ],
        "workflow": [
            {
                "path": "/",
                "method": "GET",
                "data": "",
                "headers": {}
            }
        ]}' \
    -H "X-Api-Key: my_api_key" \
    -X POST \
    https://api.rungutan.com/v1/api/tests/add | jq . -r

{
    "test_id": "some_test_id",
    "message": "Successfully created new test"
}

CLI

Start the test in the background:

$ rungutan tests add --test_file test_file.json
{
    "test_id": "fff6580d-999b-4ce5-8e1b-b14eb66f42be",
    "test_name": "test from cli",
    "message": "Successfully created new test"
}

Or, start it and wait for it to finish as well:

$ rungutan tests add --test_file test_file.json --wait_to_finish
{
    "test_id": "61c8bc4b-e1b1-4fb5-bb24-d860bf79e460",
    "test_name": "test from cli",
    "message": "Successfully created new test"
}
Waiting for test to finish...
Test with ID 61c8bc4b-e1b1-4fb5-bb24-d860bf79e460 has finished running with status FINISHED
{
    "Results": {
        "region": "overall",
        "failures": [],
        "public_ips": [
            "107.20.40.171"
        ],
        "requests": [
            {
                "Method": "GET",
                "Path": "/",
                "num_requests": 4,
                "min_response_time": 21,
                "median_response_time": 22,
                "avg_response_time": 56,
                "max_response_time": 149,
                "response_time_percentiles": {
                    "55": 31,
                    "65": 31,
                    "75": 150,
                    "85": 150,
                    "95": 150
                },
                "total_rps": 1,
                "total_rpm": 60
            }
        ]
    },
    "ResultsMetadata": {
        "team_id": "rungutan",
        "test_id": "61c8bc4b-e1b1-4fb5-bb24-d860bf79e460"
    }
}

Cancel test

Web

In order to cancel a running test, all you have to do is browse to the Dashboard page and click on the Cancel test button for the respective test.

You can only cancel a test if it’s not already finished running.

API call

The API call is:

curl -s \
    -d '{
        "test_id": "some_test_id",
        "team_id": "my-team"
        }' \
    -H "X-Api-Key: my_api_key" \
    -X POST \
    https://api.rungutan.com/v1/api/tests/cancel | jq . -r

{
    "test_id": "some_test_id",
    "message": "Successfully cancelled test.",
}

CLI

It’s as simple as:

$ rungutan tests cancel --test_id 61c8bc4b-e1b1-4fb5-bb24-d860bf79e460
{
    "message": "The test cannot be cancelled since it's currently in FINISHED status"
}

Get test details

Web

In order to get the details of a test, all you have to do is browse to the Dashboard page and click on the Copy test details button for the respective test.

API call

The API call is:

curl -s \
    -d '{
        "test_id": "some_test_id",
        "team_id": "my-team"
        }' \
    -H "X-Api-Key: my_api_key" \
    -X POST \
    https://api.rungutan.com/v1/api/tests/get | jq . -r

{
    "TestData": {
        "num_clients": 30,
        "hatch_rate": 30,
        "run_time": 30,
        "threads_per_region": 1,
        "workflow": [
            {
                "path": "/",
                "method": "GET",
                "data": "",
                "headers": {}
            }
        ],
        "domain_name": "my-team.com",
        "protocol": "https",
        "test_region": [
            "us-east-1"
        ],
        "min_wait": 1000,
        "max_wait": 1000
      }
}

CLI

This is how you run the CLI:

$ rungutan tests get --test_id 61c8bc4b-e1b1-4fb5-bb24-d860bf79e460
{
    "TestData": {
        "num_clients": 1,
        "hatch_rate": 1,
        "run_time": 5,
        "threads_per_region": 1,
        "workflow": [
            {
                "path": "/",
                "method": "GET",
                "data": "",
                "headers": {}
            }
        ],
        "domain_name": "rungutan.com",
        "protocol": "https",
        "test_region": [
            "us-east-1"
        ],
        "min_wait": 1000,
        "max_wait": 1000
    }
}

And this is how you get the TestData and save it to a file for later use:

$ rungutan tests get --test_id 61c8bc4b-e1b1-4fb5-bb24-d860bf79e460 | jq -r .TestData > file.json
$ cat file.json
{
  "num_clients": 1,
  "hatch_rate": 1,
  "run_time": 5,
  "threads_per_region": 1,
  "workflow": [
    {
      "path": "/",
      "method": "GET",
      "data": "",
      "headers": {}
    }
  ],
  "domain_name": "rungutan.com",
  "protocol": "https",
  "test_region": [
    "us-east-1"
  ],
  "min_wait": 1000,
  "max_wait": 1000
}

Set test sharing

Web

All tests are by default PRIVATE.

However, should you want to brag with your results or share them to a person that is not a member of your team, we thought you should be able to do so.

Keep in mind though, that once the tests has been switched to PUBLIC view, any non-authenticated person can see it.

Don’t worry though, you can easily switch it back to PRIVATE mode any time you want.

In order to get the set the sharing type of a test, all you have to do is browse to the Dashboard page and click on the Enable public results button for the respective test.

API call

The API call is:

curl -s \
    -d '{
        "test_id": "some_test_id",
        "team_id": "my-team",
        "test_public": "PUBLIC"                 // or "PRIVATE", depending on your scenario
        }' \
    -H "X-Api-Key: my_api_key" \
    -X POST \
    https://api.rungutan.com/v1/api/tests/set-sharing | jq . -r

{
    "message": "Successfully updated test sharing",
    "test_id": "some_test_id"
}

CLI

Setting it as public:

$ rungutan tests set-sharing --test_id 61c8bc4b-e1b1-4fb5-bb24-d860bf79e460 --test_public public
{
    "message": "Successfully updated test sharing",
    "test_id": "61c8bc4b-e1b1-4fb5-bb24-d860bf79e460"
}

Or as private:

$ rungutan tests set-sharing --test_id 61c8bc4b-e1b1-4fb5-bb24-d860bf79e460 --test_public private
{
    "message": "Successfully updated test sharing",
    "test_id": "61c8bc4b-e1b1-4fb5-bb24-d860bf79e460"
}