bullmq

Queue for messages and jobs based on Redis

  • bullmq

Downloads in past

Stats

StarsIssuesVersionUpdatedCreatedSize
bullmq
5,0712805.4.63 days ago9 years agoMinified + gzip package size for bullmq in KB

Readme




The fastest, most reliable, Redis-based distributed queue for Node. <br/>
Carefully written for rock solid stability and atomicity.

Read the documentation

<a href="https://openbase.com/js/bullmq?utm_source=embedded&utm_medium=badge&utm_campaign=rate-badge">
  <img src="https://badges.openbase.com/js/rating/bullmq.svg"/>
</a>
<a href="https://gitter.im/OptimalBits/bull">
  <img src="https://badges.gitter.im/Join%20Chat.svg"/>
</a>    
<a href="https://badge.fury.io/js/bullmq">
  <img src="https://badge.fury.io/js/bullmq.svg"/>
</a>
<a href="https://coveralls.io/github/taskforcesh/bullmq?branch=master">
  <img src="https://coveralls.io/repos/github/taskforcesh/bullmq/badge.svg?branch=master"/>
</a>
<a href="https://github.com/semantic-release/semantic-release">
  <img src="https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg"/>
</a>

<em>Follow <a href="https://twitter.com/manast">@manast</a> for *important* Bull/BullMQ/BullMQ-Pro news and updates!</em>

πŸ›  Tutorials
You can find tutorials and news in this blog: https://blog.taskforce.sh/
News πŸš€
Do you need to work with BullMQ on platforms other than Node.js? If so, check out the BullMQ Proxy
Official FrontEnd
Taskforce.sh, Inc
Supercharge your queues with a professional front end:
  • Get a complete overview of all your queues.
  • Inspect jobs, search, retry, or promote delayed jobs.
  • Metrics and statistics.
  • and many more features.

Sign up at Taskforce.sh
πŸš€ Sponsors πŸš€
<td>
  <a href="https://www.dragonflydb.io/">
    <img src="https://raw.githubusercontent.com/dragonflydb/dragonfly/main/.github/images/logo-full.svg" width=550 alt="Dragonfly" />
  </a>
</td>
<td>
  Dragonfly is a new Redisβ„’ drop-in replacement that is fully compatible with BullMQ and brings some important advantages over Redisβ„’ such as massive
  better performance by utilizing all CPU cores available and faster and more memory efficient data structures. Read more <a href="https://www.dragonflydb.io/docs/integrations/bullmq">here</a> on how to use it with BullMQ.
</td>
<td>
  <a href="https://dashboard.memetria.com/new?utm_campaign=BULLMQ">
    <img src="https://www.memetria.com/images/logo/memetria-logo.svg" width=350 alt="Memetria for Redis" />
  </a>
</td>
<td>
If you need high quality production Redis instances for your BullMQ project, please consider subscribing to <a href="https://dashboard.memetria.com/new?utm_campaign=BULLMQ">Memetria for Redis</a>, leaders in Redis hosting that works perfectly with BullMQ. Use the promo code "BULLMQ" when signing up to help us sponsor the development of BullMQ!    
</td>
Used by
Some notable organizations using BullMQ:
<td valign="center">
   <a href="https://github.com/microsoft/lage">
    <img
      src="https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LUuDmt_xXMfG66Rn1GA%2Fuploads%2FUvwInTAmk7hxAViDwJzU%2Fclipart1565701.png?alt=media"
      width="150"
      alt="Microsoft"
    />
   </a>
</td>
<td valign="center">
   <a href="https://github.com/vendure-ecommerce/vendure">
    <img
      src="https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LUuDmt_xXMfG66Rn1GA%2Fuploads%2FvT30DUqsi61gL8edn3R2%2Fwordmark-logo.png?alt=media"
      width="150"
      alt="Vendure"
    />
   </a>
</td>
<td valign="center">
   <a href="https://github.com/datawrapper/datawrapper">
    <img
      src="https://files.gitbook.com/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LUuDmt_xXMfG66Rn1GA%2Fuploads%2FCJ5XmotpBBsuSgD8CilC%2Fdatawrapper-logo.png?alt=media"
      width="150"
      alt="Datawrapper"
    />
   </a>
</td>
<td valign="center">
  <a href="https://github.com/nestjs/bull/tree/master/packages/bullmq">
    <img
      src="https://876297641-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LUuDmt_xXMfG66Rn1GA%2Fuploads%2FfAcGye182utFUtPKdLqJ%2FScreenshot%202022-02-15%20at%2011.32.39.png?alt=media&token=29feb550-f0bc-467d-a290-f700701d7d15"
      width="150"
      alt="Nest"
    />
    </a>
</td>
</tr>
<tr>
  <td valign="center">
   <a href="https://github.com/teamcurri">
    <img
      src="https://user-images.githubusercontent.com/659829/161662129-ae645bc4-c1e9-48ff-997e-4cee281a964a.png"
      width="150"
      alt="Curri"
    />
  </a>
</td>
  <td valign="center">
   <a href="https://novu.co">
    <img
      src="https://assets.super.so/1e9f5a51-c4c6-4fca-b6e8-25fa0186f139/images/0f550019-16db-4a65-90d1-1bdb7d3c5f20/novu-logo-gradient-light-background2x.png"
      width="150"
      alt="Novu"
    />
  </a>
</td>
</td>
  <td valign="center">
   <a href="https://www.nocodb.com">
    <img
      src="https://github.com/nocodb/nocodb/raw/develop/packages/nc-gui/assets/img/icons/512x512.png"
      width="50"
      alt="NoCodeDB"
    />
  </a>
</td>
</td>
  <td valign="center">
   <a href="https://infisical.com/">
     <picture>
      <source media="(prefers-color-scheme: dark)" srcset="https://mintlify.s3-us-west-1.amazonaws.com/infisical/logo/dark.svg">
      <img
      src="https://mintlify.s3-us-west-1.amazonaws.com/infisical/logo/light.svg"
      width="150"
      alt="Infisical"
    />
      </picture>
  </a>
</td>
</tr>
The gist
Install:
$ yarn add bullmq

Add jobs to the queue:
import { Queue } from 'bullmq';

const queue = new Queue('Paint');

queue.add('cars', { color: 'blue' });

Process the jobs in your workers:
import { Worker } from 'bullmq';

const worker = new Worker('Paint', async job => {
  if (job.name === 'cars') {
    await paintCar(job.data.color);
  }
});

Listen to jobs for completion:
import { QueueEvents } from 'bullmq';

const queueEvents = new QueueEvents('Paint');

queueEvents.on('completed', ({ jobId }) => {
  console.log('done painting');
});

queueEvents.on(
  'failed',
  ({ jobId, failedReason }: { jobId: string; failedReason: string }) => {
    console.error('error painting', failedReason);
  },
);

This is just scratching the surface, check all the features and more in the official documentation
Feature Comparison
Since there are a few job queue solutions, here is a table comparing them:
| Feature | BullMQ-Pro | BullMQ | Bull | Kue | Bee | Agenda | | :------------------------ | :-------------: | :-------------: | :-------------: | :---: | -------- | ------ | | Backend | redis | redis | redis | redis | redis | mongo | | Observables | βœ“ | | | | | | | Group Rate Limit | βœ“ | | | | | | | Group Support | βœ“ | | | | | | | Batches Support | βœ“ | | | | | | | Parent/Child Dependencies | βœ“ | βœ“ | | | | | | Priorities | βœ“ | βœ“ | βœ“ | βœ“ | | βœ“ | | Concurrency | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | | Delayed jobs | βœ“ | βœ“ | βœ“ | βœ“ | | βœ“ | | Global events | βœ“ | βœ“ | βœ“ | βœ“ | | | | Rate Limiter | βœ“ | βœ“ | βœ“ | | | | | Pause/Resume | βœ“ | βœ“ | βœ“ | βœ“ | | | | Sandboxed worker | βœ“ | βœ“ | βœ“ | | | | | Repeatable jobs | βœ“ | βœ“ | βœ“ | | | βœ“ | | Atomic ops | βœ“ | βœ“ | βœ“ | | βœ“ | | | Persistence | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | βœ“ | | UI | βœ“ | βœ“ | βœ“ | βœ“ | | βœ“ | | Optimized for | Jobs / Messages | Jobs / Messages | Jobs / Messages | Jobs | Messages | Jobs |

Contributing

Fork the repo, make some changes, submit a pull-request! Here is the contributing doc that has more details.
Thanks
Thanks for all the contributors that made this library possible, also a special mention to Leon van Kammen that kindly donated his npm bullmq repo.