Amazon S3 tokenizer

Downloads in past


840.2.34 months ago3 years agoMinified + gzip package size for @tokenizer/s3 in KB


Node.js CI NPM version npm downloads Known Vulnerabilities Total alerts Language grade: JavaScript
Specialized tokenizer to access files stored on the Amazon Web Services (AWS) S3 cloud storage.


Install using npm: ```shell script npm install @tokenizer/s3 @aws-sdk/client-s3
or using [yarn](https://yarnpkg.com/):
```shell script
yarn add install @tokenizer/s3 @aws-sdk/client-s3

To configure AWS client authentication see Configuration and credential file settings.


Determine S3 file type

Determine file type (based on it's content) from a file stored Amazon S3 cloud:
const FileType = require('file-type');
const { S3Client } = require('@aws-sdk/client-s3');
const { makeTokenizer } = require('@tokenizer/s3');

(async () => {

  // Initialize S3 client
  const s3 = new S3Client({});

  // Initialize  S3 tokenizer
  const s3Tokenizer = await makeTokenizer(s3, {
    Bucket: 'affectlab',
    Key: '1min_35sec.mp4'

  // Figure out what kind of file it is
  const fileType = await FileType.fromTokenizer(s3Tokenizer);

See also example at file-type.

Reading audio metadata from Amazon S3

Retrieve music-metadata
const s3tokenizer = require("@tokenizer/s3");
const { S3Client } = require('@aws-sdk/client-s3');
const mm = require("music-metadata/lib/core");

 * Retrieve metadata from Amazon S3 object
 * @param objRequest S3 object request
 * @param options music-metadata options
 * @return Metadata
async function parseS3Object(s3, objRequest, options) {
  const s3Tokenizer = await  s3tokenizer.makeTokenizer(s3, objRequest, options);
  return mm.parseFromTokenizer(s3Tokenizer, options);

(async () => {
  const s3 = new S3Client({});

  const metadata = await parseS3Object(s3, {
    Bucket: 'standing0media',
    Key: '01 Where The Highway Takes Me.mp3'


An module implementation of this example can be found in @music-metadata/s3.

Dependency graph

dependency graph