watchbot-progressa series of tracking tools available on reduce-enabled Watchbot stacks to track the progress of distributed map-reduce operations. See the included
examples.mdfor some basic worker recipes.
npm install watchbot-progress
Using the CLI commandA CLI command is available to report progress to Watchbot, and is accessible by default on reduce-enabled Watchbot stacks.
$ watchbot-progress --help Tracks the progress of a distributed map-reduce operation USAGE: watchbot-progress <command> <job-id> [options] Environment variable $ProgressTable must be set as the ARN for the DynamoDB table that is used to track progress. COMMANDS: - status: check the status of a job - set-total: set the total number of parts in a job - set-metadata: set arbitrary metadata on the job record - complete-part: complete a single part - fail-job: mark a job as a failure OPTIONS: -t, --total (for set-total) the total number of parts in a job -p, --part (for complete-part) the part number to mark as complete -m, --metadata (for set-metadata) the JSON metadata object to store -r, --reason (for fail-job) a description of why the job failed
Note that by default, workers in reduce-enabled Watchbot stacks will have the
$ProgressTableenvironment variable set automatically.
var table = require('watchbot-progress').table;
This function requires 2 parameters:
- name: DynamoDB table name
- throughout: Object that contains user specification for progress table read and write capacity units:
var progress = require('watchbot-progress').progress;
- progress.setTotal(jobId, total, callback): Set the total number of parts for a particular map-reduce operation.
- progress.completePart(jobId, part, callback): Mark a single part as complete. The response will be a boolean
- progress.setMetadata(jobId, metadata, callback): Associate arbitrary metadata with a particular map-reduce