term-vector

A node.js module that creates a term vector from tokenized text. Supports customisable stopwords and separators.

Downloads in past

Stats

StarsIssuesVersionUpdatedCreatedSize
term-vector
1811.0.04 years ago8 years agoMinified + gzip package size for term-vector in KB

Readme

!NPM versionnpm-version-imagenpm-url !NPM downloadsnpm-downloads-imagenpm-url !MIT Licenselicense-imagelicense-url !Build Statustravis-imagetravis-url
term-vector
A node.js module that creates a term vector from tokenized text. Use term-vector when implementing a vector space model
Works with Unicode!
Does ngrams!
const tokens = 'this is really really really cool'.split(' ')

// just make a simple term vector
tv(tokens)
// [
//   { term: [ 'cool' ], positions: [ 5 ] },
//   { term: [ 'is' ], positions: [ 1 ] },
//   { term: [ 'really' ], positions: [ 2, 3, 4 ] },
//   { term: [ 'this' ], positions: [ 0 ] }
// ]

// make a term vector with ngrams of length 1 and 2
tv(tokens, { ngramLengths: [ 1, 2 ] })
// [
//   { term: [ 'cool' ], positions: [ 5 ] },
//   { term: [ 'is' ], positions: [ 1 ] },
//   { term: [ 'is', 'really' ], positions: [ 1 ] },
//   { term: [ 'really' ], positions: [ 2, 3, 4 ] },
//   { term: [ 'really', 'really' ], positions: [ 2, 3 ] },
//   { term: [ 'really', 'cool' ], positions: [ 4 ] },
//   { term: [ 'this' ], positions: [ 0 ] },
//   { term: [ 'this', 'is' ], positions: [ 0 ] }
// ]