View on GitHub

Asset-smasher

Asset pre-processor, merger, and compressor for Node.js

Download this project as a .zip file Download this project as a tar.gz file

Asset Smasher

Asset pre-processor, merger, and compressor for Node.js

Overview

Asset Smasher is a command-line tool, express middleware, and programmatic interface for:

It's released under the MIT license.

Structuring Your Assets

Asset Smasher has the concept of "asset paths". These are locations in which your asset files will be located, and from which any relative asset paths will be rooted to.

The simplest structure has one asset path.

E.g.

Asset Paths
-----------
 - app

File Structure
--------------
app/
  js/
  css/
  images/

A more complicated structure might be

Asset Paths
-----------
 - app
 - lib
 - vendor

File Structure
--------------
app/
  js/
  css/
  images/
lib/
  js/
  css/
  images/
vendor/
  js/
  css/
  images/

Both of these examples will result in a compiled structure of

js/
css/
images/

Manifest Files

Manifest (.mf) files are used to merge many assets into a single resulting file. The file should be named with the resulting file type before the .mf extension (e.g. manifest.css.mf or manifest.js.mf. Manifest files can require other manifest files

Only files that will be transformed down to a file of a manifest's "type" (e.g. manifest.css.mf => .css, manifest.js.mf => .js) will be included. This means that, for example, if you require_dir a directory in a JavaScript manifest that happens to contain both JavaScript and CSS, only the JavaScript files will be required.

A simple manifest file might look like

# A comment here
require "./one.js"
require_dir "./subdir1"
#
# Another comment
require_tree "./subdir2"

Directives:

Directive Description
require "[path]" Include a single file
  • If the path starts with "/", "../", or "./", process and include the specified file. The file must be inside one of the configured asset paths.
  • If the path does not start with "/", "../", or "./", the file will be searched for in all of the configured asset paths. E.g. if there are asset paths one and two defined, require "js/test.js" will look for one/js/test.js and then two/js/test.js stopping when it finds a matching file.
  • The filename part of the path does not have to include the whole extension. E.g require "test" finds the first file that matches the name in the asset paths (for example test.js.ejs)
  • If the file does not exist/can't be resolved/isn't of the right type for the manifest, it will be ignored (will be logged in verbose mode).
require_dir "[path]" Include all the files in a directory
  • The path must be absolute, or relative to the current directory. E.g. you can do require_dir "../some/other/dir" but not require_dir "somedir"
  • If using absolute paths, or ".." in your paths, the resulting directory needs to be inside one of the configured asset paths.
  • If the directory does not exist, it will be ignored (will be logged in verbose mode).
require_tree "[path]" Include the files in a directory recursively
  • The rules for require_tree are the same as the rules for require_dir

Manifest Directories

If you create a directory, for example named foo.js.mf and put a bunch of javascript files in it (or any subdirectories under it), asset-smasher will (recursively) take all the files inside and merge them into foo.js.

Essentially, this is a time-saver so that you don't have to create a manifest file that only contains a single require_tree directive.

Using via Command-Line

Use npm install -g asset-smasher to install the asset-smasher command-line tool globally.

  asset-smasher --help

    Usage: asset-smasher [options] <output dir>

    Options:

      -h, --help               output usage information
      -V, --version            output the version number
      --compress               compress/minify the generated files
      --hash                   generate versions of the files with md5 hashes in the name
      --gzip                   generate gzipped versions of the compiled files
      --hashVersion <version>  invalidate all assets without changing file contents [1.0]
      --only <pattern,...>     only process the files matching these glob patterns (relative to any of the paths) [**/*.*]
      --paths <path,...>       list of paths to look for assets [.]
      --prefix <prefix>        prefix to append to logical paths when constructing urls. use if output dir is not served from the root of your web app []
      --helpers <js_file>      a .js module of helper functions require()s to expose to transforms []
      --plugins <js_file>      a .js plugin module []
      --amd                    enable AMD support (give anonymous modules names, support /** @amd */ pragma comments)
      --verbose                output more verbose information about what is going on to the console
      --noclean                do not delete the output directory before generating files (by default it will be removed first)

    If --only is not specified, *all* files in the --paths will be processed.
    If --hash is specified, a map.json file will be generated that maps the unmangled file name to the hashed one.

    Be careful that your shell doesn't expand glob patterns when passing them as arguments. To be safe, surround the argument with quotes.

    Examples:

      Compile all assets in the current directory to /home/me/compiledAssets

        $ asset-smasher /home/me/compiledAssets

      Something similar to what the Rails asset pipeline does by default

        $ asset-smasher --compress --hash --gzip --prefix /assets \
            --paths ./js,./css,./images \
            --only "**/*.jpg,**/*.gif,**/*.png,application.js.mf,application.css.mf" ./public/assets

      Compile assets, providing some custom helpers to the transformation

        $ asset-smasher --helpers helpers.js output

Helpers

There is a built-in asset_src helper that can be used to get the "real" (i.e. with hashed file name) path of an asset. E.g. asset_src('css/myFile.css') might return '/assets/css/myFile-c89cba7b7df028e65cb01d86f4d27077.css.

Some transformers (e.g. the .ejs one) take in a set of local variables that they can use during transformation. You can pass in the path to a JavaScript module whose exports will be included in this set of variables.

You can use this, for example, to set configuration parameters in your JS files:

helper.js

exports.serviceUrl = 'http://my.service/';

config.js.ejs

//...
var serviceUrl = '<%= serviceUrl %>';
var cssLocation = '<%= asset_src('css/myFile.css') %>';
//...

Execution

$ asset-smasher --helpers helper.js --only config.js.ejs,css/myFile.css .
$ cat config.js
var serviceUrl = 'http://my.service/';
var cssLocation = '/assets/css/myFile-c89cba7b7df028e65cb01d86f4d27077.css';

Plugins

If there's a type of file you want to pre-process that is not natively supported by Asset Smasher, you can add it using a plugin file.

For an example of what the transformer classes look like, look in the lib/compilation/transforms directory

If a plugin module is passed (via --plugins), it will be require()d and then invoked, being passed in the asset smasher library (the module defined in lib/asset-smasher.js)

To register your transformer, just add another entry to the transforms object.

E.g.

my_plugin.js

module.exports = function(assetSmasher) {
   // A stupid transformer that adds "foo" to the start and end of the contents
   var FooTransform = function FooTransform(options) {
     this.options = options || {};
   };
   FooTransform.prototype = {
     extensions:function () {
       return ['.foo'];
     },
     shouldTransform:function (file) {
       return path.extname(file) === '.foo';
     },
     transformedFileName:function (file) {
       return path.basename(file, '.foo');
     },
     transform:function (asset, cb) {
       // Transform the file name
       asset.logicalName = this.transformedFileName(asset.logicalName);
       // Get the contents
       var contents = asset.contents;
       if (Buffer.isBuffer(contents)) {
         contents = contents.toString('utf-8');
       }
       // Compile the contents
       asset.contents = 'foo-' + contents + '-foo';
       cb();
     }
   };

   assetSmasher.transforms.Foo = FooTransform;
};

If you then invoke asset-smasher with --plugins my_plugin.js it will automatically transform *.foo files.

Using via Express Middleware

Asset smasher exposes an express middleware that can:

The middleware takes in the same arguments as the Smasher constructor, with a few extras:

The middleware exposes two helpers to your views:

You must include the middleware before the Express routing middleware. Otherwise the asset helper functions will not be available for your view to use.

Example

var assetSmasher = require('asset-smasher');

Middleware config (Dev)

app.use(assetSmasher.middleware({
  serve: true,
  paths: [path.join(__dirname, 'assetDir1'), path.join(__dirname, 'assetDir2')],
  prefix: '/assets',
  outputTo: path.join(__dirname, 'tmp')
}));

Middleware config (Prod)

app.use(assetSmasher.middleware({
  serve: false,
  prefix: '/assets',
  assetMapLocation: path.join(__dirname, 'public/assets/map.json')
}));

Middleware config (Alternate Prod config not using precompilation, but instead compile on first access)

Note that if you use this configuration, you will not be able to use "hashed" filenames.

app.use(express.staticCache());
app.use(express.static(path.join(__dirname, 'public')));
app.use(assetSmasher.middleware({
  serve: {
    individual: false
  },
  paths: [path.join(__dirname, 'assetDir1'), path.join(__dirname, 'assetDir2')],
  compress: true,
  prefix: '/assets',
  // This will make the files be served once by asset smasher
  // and then by the express "static" middleware thereafter.
  // You can then also use something like "staticCache" to cache the files if you're not
  // using a reverse proxy cache on the public dir
  outputTo: path.join(__dirname, 'public/assets')
}));

View (ejs here, but could be others)

<!DOCTYPE html>
<html>
<head>
  <title>Test</title>
  <%- css_asset('application.css') %>
  <%- js_asset('application.js') %>
</head>
<body>
  This is a test
</body>
</html>

Using via Programmatic Interface

You can invoke Asset Smasher programmatically by requireing it. You can also plug in additional transformers this way.

The Smasher object has the following methods:

The Asset object returned by getAssetByLogicalPath has the following properties (and one method):

Example

var assetSmasher = require('asset-smasher');
var Smasher = assetSmasher.Smasher;

// Plug in a custom transformer
assetSmasher.transforms['MyAwesomeFormat'] = require('myAwesomeFormatTransformer');

var sm = new Smasher({
  paths:['/path/one', '/path/two'],
  only:['**/*.{jpg,gif,png}', 'application.js.mf', 'application.css.mf'],
  prefix:'/assets',
  compress:true,
  hash:true,
  hashVersion:'1.0',
  gzip:true,
  outputTo:__dirname + '/public/assets',
  helpers:{
   my: 'helper',
   another: 'helper'
  },
  amd:false,
  verbose:true,
  noclean:true
});
sm.compileAssets(function(err) {
  if(err) {
    console.log('An error occurred', err);
  } else {
    console.log('Compilation done!');
  }
});

Transformer Notes

To use a transformer you must install the associated node module.

AMD Support

Asset-smasher can wrap JavaScript files that follow the CommonJS module format with Asynchronous Module Definition define calls.

To enable the wrapping of a JavaScript file, pass the --amd option to asset-smahser and put the following at the top of it:

/** @amd */

This will cause asset smasher to:

Example (say the file is in scripts/foo.js)

/** @amd */
var x = require('x');
var y = require('../y');

exports.foo = function (bar) {
  return x(bar) + y(bar);
};

This will be transformed into:

define('scripts/foo',
       ['require', 'exports', 'module', 'x', '../y'],
       function (require, exports, module) {
  var x = require('x');
  var y = require('../y');

  exports.foo = function (bar) {
    return x(bar) + y(bar);
  };
});

The /** @amd */ pragma can also take a comma-delimeted list of module ids after it, which will be added to the list of dependencies. This is useful when you need a dependency to be loaded, but aren't using it directly (e.g. a jQuery plugin)

Example (say the file is in scripts/foo.js)

/** @amd ./foo,./bar,./baz */
var x = require('x');
// ...

Will be transformed into

define('scripts/foo',
       ['require', 'exports', 'module', 'x', './foo', './bar', './baz'],
       function (require, exports, module) {
  var x = require('x');
  // ...
});

If you have any anonymous AMD modules specified (i.e. with no module id) or use the simplified commonjs wrapper, the --amd option will give these modules names (relative to the asset path root)

Example (say the file is in modules/test.js)

define(['foo','bar'], function (foo, bar) {
  // ...
});

Will be transformed into

define('modules/test', ['foo', 'bar'], function (foo, bar) {
  // ...
});

And

define(function (require, exports, module) {
  var x = require('x');
  //...
});

Will be transformed into

define('modules/test', ['require', 'exports', 'module', 'x'], function (require, exports, module) {
  var x = require('x');
  //...
});

You can then use an AMD loader (like require.js) to load the modules.

LESS/Stylus

ejs

dust and Handlebars