Skip to main content

Queue workers

Queue workers enables async and persistent queues (MQ). Queue workers are a brilliant way of adding scalability and resilience to your application backend.

Objects (messages) are added to queues using the built-in JavaScript Queue API. Serverless queue workers process each object in a queue asynchronously. The illustration below shows how three independent queues (named topic-1, topic-2, topic-3) with their message objects are being processed by independent workers that eventually execute your function.

Parallel message queues invoking your serverless functions
            / "topic-1" : x x x x x x -> (WORKERS) -> yourFunc1(x)
Enqueue(x) - "topic-2" : x x -> (WORKERS) -> yourFunc2(x)
\ "topic-3" : x x x x x -> (WORKERS) -> yourFunc3(x)

Example queue worker

The following example queue worker function prints to the log each time a request.body is added to the queue as a POST to the example route /addQueue.

import {coho as app, Datastore} from 'codehooks-js'

// Serverless queue worker function for topic 'myqueue'
app.worker('myqueue', (req, res) => {
console.log("The queue item is", req.body.payload);
res.end(); // done with one item

// API route that adds to the queue'/addQueue', async (req, res) => {
const conn = await;
const jobId = await conn.enqueue("myqueue", {"data": req.body});

export default app.init();
  • Topic names are unique strings. E.g. "sendEmails"
  • Queue data are accessed through the request.body.payload object
  • Each topic is processed as a separate queue
  • QUEUE_WORKERS environment variable manage number of parallell workers. E.g. QUEUE_WORKERS=10
  • Each queue are stored in separate system collections prefixed with _queue_{TOPIC}. E.g. if the topic is sendEmail the queue would be named _queue_sendEmails
  • Read more about the JavaScript Queue API