@kafkajs/zstd

A KafkaJS codec for ZStandard compression

Downloads in past

Stats

StarsIssuesVersionUpdatedCreatedSize
@kafkajs/zstd
620.1.1a year ago2 years agoMinified + gzip package size for @kafkajs/zstd in KB

Readme

@kafkajs/zstd
ZStandard codec for KafkaJS.
NOTE: This codec relies on cppzst for compression and decompression. Please divert any issues with cppzst there, and only open issues on this repo in case the version being used here needs to be updated.
This package is provided on a best-effort basis. See #2

Installation

npm install --save @kafkajs/zstd

Configuration

const {  CompressionTypes, CompressionCodecs } = require('kafkajs')
const ZstdCodec = require('@kafkajs/zstd')

// Both compressionParams and decompressionParams are optional
const compressionParams = { level: 1 }
const decompressionParams = {}
CompressionCodecs[CompressionTypes.ZSTD] = ZstdCodec(compressionParams, decompressionParams)

compressionParams

Optional Allows you to configure the compression level and training data.
{
    /*  
     * Higher compression level means faster at the cost of compression ratio or memory usage.
     * See https://facebook.github.io/zstd/
     */
    level: 3,

    /*
     * Training data for improving performance on small payloads.
     * See https://facebook.github.io/zstd/#small-data
     */
    dict: trainingData,
    dictSize: Buffer.byteLength(trainingData)
}

decompressionParams

Optional Allows you to configure the training data.
{
    /*
     * Training data for improving performance on small payloads.
     * See https://facebook.github.io/zstd/#small-data
     */
    dict: trainingData,
    dictSize: Buffer.byteLength(trainingData)
}

Testing

$ docker-compose up -d && node scripts/waitForKafka.js
$ npm test

# Interactive mode
$ npm run test:watch

# Verify type definitions
$ npm run test:types

# Lint
$ npm run lint

License

See LICENSE for more details.