cache-manager-fs-binary

file system store for node cache manager with binary data as files

Downloads in past

Stats

StarsIssuesVersionUpdatedCreatedSize
cache-manager-fs-binary
1.0.45 years ago6 years agoMinified + gzip package size for cache-manager-fs-binary in KB

Readme

Build Status
node-cache-manager-fs-binary
Node Cache Manager store for Filesystem with binary data
The Filesystem store for the node-cache-manager module, storing binary data as separate files, returning them as readable streams or buffers. This should be convenient for caching binary data and sending them as streams to a consumer, e.g. res.send(). The library caches on disk arbitrary data, but values of an object under the special key binary is stored as separate files.

Node.js versions

Works with versions 4, 5 and iojs.

Installation

npm install cache-manager-fs-binary --save

Usage examples

Here are examples that demonstrate how to implement the Filesystem cache store.

Features

  • limit maximum size on disk
  • refill cache on startup (in case of application restart)
  • returns binary data as buffers or readable streams (keys of the binary key)
  • can store buffers inside the single cache file (not keys of the binary key)

Single store

// node cachemanager
var cacheManager = require('cache-manager');
// storage for the cachemanager
var fsStore = require('cache-manager-fs-binary');
// initialize caching on disk
var diskCache = cacheManager.caching({
    store: fsStore,
    options: {
        reviveBuffers: true,
        binaryAsStream: true,
        ttl: 60 * 60 /* seconds */,
        maxsize: 1000 * 1000 * 1000 /* max size in bytes on disk */,
        path: 'diskcache',
        preventfill: true
    }
});

// ...
var cacheKey = 'userImageWatermarked:' + user.id + ':' + image.id;
var ttl = 60 * 60 * 24 * 7; // in seconds

// wrapper function, see more examples at node-cache-manager
diskCache.wrap(cacheKey,
    // called if the cache misses in order to generate the value to cache
    function (cacheCallback) {
        var image; // buffer that will be saved to separate file
        var moreData; // string that will be saved to a separate file
        var userLastVisit; // Date
        var signature; // small binary data to store inside as buffer

        // ... generating the image

        // now returning value to cache and process further
        cacheCallback(err,
// Some JSONable object. Note that null and undefined values not stored.
// One can redefine isCacheableValue method to tweak the behavior.
            {
                binary: {
// These will be saved to separate files and returned as buffers or
// readable streams depending on the cache settings.
// **NB!** The initial values will be changed to buffers or readable streams
// for the sake of simplicity, usability and lowering the memory footprint.
// Check that the keys are suitable as parts of filenames.
                    image: image,
                    someOtherBinaryData: moreData
                },
// Other data are saved into the main cache file
                someArbitraryValues: {
                    eg: userLastVisit
                },
                someSmallBinaryValues: {
// While buffer data could be saved to the main file, it is strongly
// discouraged to do so for large buffers, since they are stored in JSON
// as Array of bytes. Use wisely, do the benchmarks, mind inodes, disk
// space and performance balance.
                    pgpSignatureBufferForm: signature
                }
            });
    },
// Options, see node-cache-manager for more examples
    {ttl: ttl},
// Do your work on the cached or freshly generated and cached value.
// Note, that result.binary.image will come in readable stream form
// in the result, if binaryAsStream is true
    function (err, result) {

        res.writeHead(200, {'Content-Type': 'image/jpeg'});
        var image = result.binary.image;

        image.pipe(res);

        var usedStreams = ['image'];
        // you have to do the work to close the unused files
        // to prevent file descriptors leak
        for (var key in result.binary) {
            if (!result.binary.hasOwnProperty(key))continue;
            if (usedStreams.indexOf(key) < 0
                && result.binary[key] instanceof Stream.Readable) {
                if(typeof result.binary[key].close === 'function') {
                    result.binary[key].close(); // close the stream (fs has it)
                }else{
                    result.binary[key].resume(); // resume to the end and close
                }
            }
        }
    }
)

Options

options for store initialization
// default values

// time to live in seconds
options.ttl = 60;
// path for cached files
options.path = 'cache/';
// prevent filling of the cache with the files from the cache-directory
options.preventfill = false;
// callback fired after the initial cache filling is completed
options.fillcallback = null;
// if true the main cache files will be zipped (not the binary ones)
options.zip = false;
// if true buffers not from binary key are returned from cache as buffers,
// not objects
options.reviveBuffers = false;
// if true, data in the binary key are returned as StreamReadable and 
// (**NB!**) the source object will also be changed. 
// You have to do the work for closing the files if you do not read them,
// see example.
options.binaryAsStream = false;

Tests

To run tests:
npm test

Code Coverage

To run Coverage:
npm run coverage

License

cache-manager-fs-binary is licensed under the MIT license.

Credits

Based on https://github.com/hotelde/node-cache-manager-fs