async-qps-throttle

A promise-based throttling mechanism for Javascript.

Downloads in past

Stats

StarsIssuesVersionUpdatedCreatedSize
async-qps-throttle
001.0.15 years ago5 years agoMinified + gzip package size for async-qps-throttle in KB

Readme

async-qps-throttle
This package provides throttling of promise-based work, based on one or both of the amount of concurrent active work and QPS (queries per second). ES6 or higher is required.

Use

Creation

Simply import AsyncThrottle from this package:
import {AsyncThrottle} from 'async-qps-throttle';

const throttle = new AsyncThrottle({maxOutstanding: 10, maxQps: 10});

The constructor takes an options object hash. The available options are: maxOutstanding: The maximum number of work items that may be actively running at any given time maxQps: The maximum number of work items that will be started in any given second using a rolling
window (note that work may continue to run and won't count against QPS)
The example above creates a throttle that will ensure that no more than 10 work items are ever concurrently running, and no more than 10 work items are executed in any given period of one second.

Providing Work

Work is provided in the form of a function which takes no parameters and returns a promise (the implementation uses the native ES6 Promise, see below for more information). The use of a generator function allows the throttle to determine the appropriate time to actually start the work. Work is provided to the single method callThrottled, like so:
...

// Pre-defined function.
function postToSite() {
  return xhrPromise.send({...});
}
throttle.callThrottled(postToSite);

// Arrow function.
throttle.callThrottled(() => xhrPromise.send({...}));

...

Where's My Promise?

callThrottled returns a promise that is immediately available and will eventually settle in the same way as the promise generated by the work function. If the work function throws, this promise will also be rejected with that error.
...

// Unthrottled call.
const unthrottledPromise = xhrPromise.send({...});

// Throttled call.
const throttledPromise = throttle.callThrottled(() => xhrPromise.send({...}));

// Inline then-ing with throttle.
throttle
  .callThrottled(() => xhrPromise.send({...}))
  .then(response => {...});
  
...

When Is Everything Done?

AsyncThrottle provides two additional methods for figuring out when all work is complete (aside from just keeping and waiting on the returned promises). The first is whenDrained, a method that provides a promise that will resolve at the next moment when all tracked work is complete. This promise does not provide any results, but simply acts as a way to ensure work has completed. This promise is never rejected.
...

throttle.callThrottled(() => xhrPromise.send({...}));
throttle.callThrottled(() => xhrPromise.send({...}));
throttle.callThrottled(() => xhrPromise.send({...}));
throttle.callThrottled(() => xhrPromise.send({...}));
throttle
  .whenDrained()
  .then(() => {...});

...

The second is a static method called callAllThrottled, which works similarly to Promise.all but will throttle the array of work items according to throttling options provided with the call.
...

const workFn1 = () => return Promise.resolve(1);
const workFn2 = () => return Promise.resolve(2);
const workFn3 = () => return Promise.resolve(3);

AsyncThrottle
  .callAllThrottled([workFn1, workFn2, workFn3], {maxQps: 1})
  .then(result => {... result will be [1, 2, 3] ...});

...

The promise returned by callAllThrottled will be resolved with an array of results that correspond to the input work, or will be rejected with the first error that any executed work returned.

Details

Outstanding Work Measurement

Outstanding work is measured based on how many concurrent work items are currently running. As an example, suppose there are work items that takes varying lengths of time to run and the throttle is set up with {maxOutstanding: 2}. Execution might proceed as follows:
. = throttled
* = running
! = complete

    0ms       1000ms   2000ms    3000ms    4000ms    5000ms    6000ms
    +---------+--------+---------+---------+---------+---------+
W1  ********************************************!
W2  **********!
W3  ..........******************************!
W4  ........................................*********!
W5 .............................................********!

QPS Measurement

QPS is strictly measured by the invocation of work, never how long the work takes to complete. As an example, suppose there are work items that takes 3000ms to run and the throttle is set up with {maxQps: 1}. Execution would proceed as follows:
. = throttled
* = running
! = complete

    0ms       1000ms   2000ms    3000ms    4000ms    5000ms    6000ms
    +---------+--------+---------+---------+---------+---------+
W1  *****************************!
W2  ..........*****************************!
W3  ...................******************************!
W4  .............................******************************!

Note that during execution, at times there are up to 3 concurrent work items.

Promises

This package consumes and returns native ES6 promises (Promise). If your project uses native promises, read no further. However, if your project uses an external promise library (e.g., Bluebird), read on.
If running in a strongly-typed environment (e.g., TypeScript), simply wrap promises on the way in and/or the way out:
...
function fancyPromiseGenerator() {return fancyPromise();}
function fancyPromiseConsumer(fancyPromise) {...}

const rawPromise = throttle.callThrottled(() => Promise.resolve(fancyPromiseGenerator()));
fancyPromiseConsumer(FancyPromise.resolve(rawPromise));

On the other hand, if you are running in a weakly-typed environment (e.g., ES6), it may not be necessary to wrap the promises at all. Just note, the promises returned by this module will be fairly basic (no tap, finally, etc.).