Automated client requests

Downloads in past


142342.3.05 years ago6 years agoMinified + gzip package size for test-openapi in KB


Travis Node Gitter
Automatic API integration testing.
  • Declarative. Tasks are specified in simple YAML files.
  • Easy. Each task is a single HTTP request/response. You only need to
specify the request parameters and the response validation. More complex requests flows are also supported.
  • Integrated to OpenAPI. Tasks re-use your OpenAPI
specification by default, making them less verbose and ensuring they match your documentation.
  • Fast. Tasks have minimum overhead and run in parallel.
  • Nice developer experience. Reporting is pretty,
informative and usable. templating system
  • Flexible. Core functionalities can be extended with plugins.
$ npm install -D test-openapi
Usage (shell)
$ test-openapi

If a task failed, exit code will be 1.
Options are passed as CLI flags.
$ test-openapi --merge.spec.definition openapi_schema.yml

Tasks are passed as positional argument.
$ test-openapi **/*.tasks.yml
Usage (Node.js)
const { run } = require('test-openapi')

const promise = run(options)

If a task failed, run() will reject the promise with a TestOpenApiError.
Options are passed as arguments. Tasks are passed as a tasks argument.
const promise = run({
  tasks: ['**/*.tasks.yml'],
  merge: { spec: { definition: 'openapi_schema.yml' } },
Tasks are specified in YAML or JSON files.
By default tasks at **/*.tasks.yml|json will be used.
Each task file contains an array of tasks definitions.
A single task performs the following:
  • sends an HTTP request to the API. The request parameters are specified using
the call property.
  • validates the HTTP response according to the
validate property.
Each task must specify a name unique within its file.
Example input
- name: exampleTask
    method: GET
    server: http://localhost:8081
    path: /tags
    query.onlyPublic: false
    status: 200
      type: array
      # Each tag object
        type: object
        required: [tag, isPublic]
            type: string
            maxLength: 32
            type: boolean

# And so on
- name: anotherTask

This task calls:
GET http://localhost:8081/icoTagNames?onlyPublic=false

It then validates that:
  • the response status is 200
  • the response body is an array of { tag: string, isPublic: boolean }
Example output
This screenshot shows a typical task run with few task failures.
The failed task is called getLists.success and performs the following HTTP request:

It expects a status code of 200 but receives 500 instead.
Other tasks are shown failing at the end. A final summary is also present.
HTTP requests
HTTP requests are specified with the call task property.
- name: taskName
    method: PUT
    server: https://localhost:8081
    path: /tags/:tagName
    url.tagName: exampleTagName
    query.accessToken: 1e42f0e1
    headers.content-type: application/json
      _id: 1
      name: exampleTagName
      color: red
      rejectUnauthorized: false

  • method {string} (default: GET): HTTP method
  • server {string}:
- server's origin (protocol + hostname + port) - default values:
- protocol: `http://`
- hostname: environment variable `HOST` or (if absent) `localhost`
- port: environment variable `PORT` (if present)
  • path {string}: URL's path
  • url.NAME {any}:
- variable inside server or path using the :NAME notation - for example if the path is /tags/:tagName it can be url.tagName - the syntax is the same as
[Express route parameters](https://expressjs.com/en/guide/routing.html#route-parameters):
- `:NAME`: required parameter
- `:NAME?`: optional parameter
- `:NAME*`: several optional parameters
- `:NAME+`: several required parameters
  • query.NAME {any}:
- URL query variable - specify a list delimited by &= to use NAME several times
- e.g. `query.name: "a&=b&=c"` becomes the query variables
  • headers.NAME {any}:
- HTTP request header - case insensitive - headers.content-type defaults to:
- `application/json` if `body` is an object or an array
- `application/octet-stream` otherwise
  • body {any}: request body
  • https {object}:
- HTTPS/TLS options - Same as the ones allowed by https.request(), i.e. ca, cert, ciphers, clientCertEngine, crl, dhparam, ecdhCurve, honorCipherOrder, key, passphrase, pfx, rejectUnauthorized, secureOptions, secureProtocol, servername, sessionIdContext.
url.NAME, query.NAME, headers.NAME and body can be either a string or any other JSON type:
  • they will be serialized according to the HTTP request header Content-Type
  • however at the moment only JSON is supported. Notably multipart/form-data
and x-www-form-urlencoded are not supported yet.
  • same goes for the response headers and body
Response validation
The HTTP response is validated against the validate task property.
- name: taskName
    status: 201
    headers.content-type: application/json
      type: array

  • status {string|integer}:
- expected HTTP status code - can be:
- a specific status code like `201`
- a range like `1xx`, `2xx`, `3xx`, `4xx` or `5xx`
- a space-delimited list of these like `201 202 3xx`
- default: 2xx
  • headers.NAME {any|jsonSchema}:
- expected value for this HTTP response header - NAME is case-insensitive - this can be either:
- any value checked for equality
- a
  [JSON schema version 4](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#schemaObject) with the additional following properties:
  - `x-optional` `{boolean}` (default: `true`): if `false`, validate that
    the HTTP header is present in the response
  - `x-forbidden` `{boolean}` (default: `false`): if `true`, validate
    that the HTTP header is not present in the response
  • body {any|jsonSchema}:
- expected value for the response body - this can be either a non-object checked for equality or a
[JSON schema version 4](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#schemaObject)
(like `headers.NAME`)
Validation can also vary according to the response's status code by using the following notation.
- name: taskName
        type: array
        type: object
The call and validate tasks properties can be pre-filled if you have described your API endpoints with OpenAPI.
- name: taskName
    operation: getTags
    definition: ../openapi_document.yml

  • operation {string}: OpenAPI's
  • definition {string}:
- path to the OpenAPI document - it is likely that the same OpenAPI document is re-used across tasks, so
the [`merge` task property](#shared-properties) can be used
- the OpenAPI document syntax is validated - only OpenAPI 2.0 is currently supported but we plan to add OpenAPI 3.0
The following OpenAPI properties are currently used:
  • the
consumes OpenAPI property sets the request Content-Type header (call['headers.content-type'])
  • the
produces OpenAPI property sets the request Accept header (call['headers.accept']) and validate the response's Content-Type header (validate['headers.content-type'])
  • the
host and basePath OpenAPI properties set the call.server task property. At the moment the protocol is always http://.
  • the call.method and call.path is taken from the
OpenAPI definition
  • the request parameters are randomly generated from the
parameters OpenAPI property: - the random generation is based on
[JSON schema faker](https://github.com/json-schema-faker/json-schema-faker)
- OpenAPI parameters not marked as required will only be used (and merged)
if they are explicitly present in the `call` task property
- the following special values can used in the call task property:
- `valid`: re-use the OpenAPI parameter definition. Useful if the OpenAPI
  parameter is not marked as `required`. Redundant otherwise.
- `undefined`: do not use the OpenAPI parameter definition
  • the
response's schema and headers OpenAPI properties are used to validate the HTTP response (validate.status|body|headers)
OpenAPI schemas can use the following extensions:
  • schema.x-nullable|oneOf|anyOf|not: behaves like OpenAPI 3.0
  • schema.x-additionalItems|dependencies: behaves like JSON schemas
Shared properties
To specify properties shared by all tasks, use the merge option:
$ test-openapi --merge.spec.definition ../openapi_document.yml

To specify properties shared by a few tasks, create a task with a merge property.
- name: sharedTask
  merge: invalidCheck/.*
    status: 400

The merge property should be a regular expression (or an array of them) targeting other tasks by name. The shared task will not be run. Instead it will be deeply merged to the target tasks.
The target tasks can override the shared task by using undefined inside task properties.
Template variables
Template variables can be used using the $$name notation.
Template variables are specified using the template task property.
- name: exampleTask
    $$exampleVariable: true
    query.isPublic: $$exampleVariable

The example above will be compiled to:
- name: exampleTask
    query.isPublic: true

Template variables can:
  • be concatenated within a string like $$exampleVariable --- $$anotherVariable
  • use brackets and dots to access object properties and array indexes like
  • be functions:
- by default they are triggered with no arguments - to specify arguments one can use the following notation:
`{ $$exampleFunction: [firstArg, secondArg] }`
The following template variables are always available:
  • $$env: use environment variables (case-sensitive)
  • $$random: generate fake data using a
JSON schema version 4
  • $$faker: generate fake data using
- name: exampleTask
    server: $$env.SERVER
        type: string
        minLength: 12
        pattern: '[a-zA-Z0-9]'
      name: $$faker.name.firstName
Sequences of requests
A request can save its response using variables. Other requests will be able to re-use it as template variables. This creates sequences of requests.
- name: createAccessToken
    $$accessToken: call.response.body.accessToken

- name: taskName
    query.accessToken: $$accessToken

The call.request and call.response are available to re-use the HTTP request and response.
The task will fail if the variable is undefined unless you append the word optional to its value.
- name: createAccessToken
    $$accessToken: call.response.body.accessToken optional
Tasks selection
By default all tasks are run in parallel at the same time.
To only run a few tasks use the only option.
$ test-openapi --only 'taskNameRegularExpression/.*'

Or the only task property.
- name: taskName
  only: true

The skip option and task property can be used to do the opposite.
The following reporters are available:

Specify the --report.REPORTER option to select which reporter to use
$ test-openapi --report.notify --report.pretty

Use the --report.REPORTER.output to redirect the output of a reporter to a file:
$ test-openapi --report.pretty.output path/to/file.txt

Use the --report.REPORTER.level to modify the verbosity:
$ test-openapi --report.pretty.level info

The available levels are:
  • silent
  • error
  • warn (default for pretty)
  • info (default for tap and notify)
  • debug (default for data)
Data-driven testing
With the repeat.data task property, tasks are repeated by iterating over an array of inputs.
- name: exampleTask
      - name: apples
        quantity: 1
      - name: oranges
        quantity: 10
      - name: plums
        quantity: 100
    method: GET
    path: /fruits/:fruitName
    url.fruitName: $$data.name
      quantity: $$data.quantity

The task above will be run three times: GET /fruits/apples?quantity=1, GET /fruits/oranges?quantity=10 and GET /fruits/plums?quantity=100.
With the repeat.times task property, tasks are simply repeated as is. This can be useful when used with the $$random template function.
- name: exampleTask
    times: 10

repeat.data and repeat.times can be combined.