• Part 1: Setting up Gulp with Visual Studio
• Part 2: Getting SASSy
• Part 3: Deployment: minification and concatenation
• Part 4: Code quality: tests and linting
• Part 5: Thoughts on Gulp

In this post I want to talk about JavaScript code quality.

Code quality: tests and linting

JavaScript is a notoriously quirky language –  the tricky “this” operator, lack of block level scoping, the ability to use variables before declaring them, “==” vs “===”, etc etc.

We can do a number of things to tighten up our code quality. We can run tests to ensure our code works as expected, and we can run a linting program to catch any syntatic mistakes before running our code. Combining this with Gulp gives a nice workflow for development.

Linting with JSHint

Linting is a process of running a program to analyse code for potential problems, catching bugs before the program gets executed. Douglas Crockford created JSLint in 2011, and of course there is a gulp plugin for it. However, I prefer to use JSHint – a community offshoot of JSLint that allows for more customisation and is slightly more relaxed in its defaults. You can find a good comparison of the two here.

First, I would seperate out any third-party libraries or frameworks you have into their own folder(s). In my opinion this is good practice anyway, but especially if we are running linting – we don’t want failures on code we haven’t written! Next, we want to add the plugin to our package.json:

{
  "scripts": { "install": "echo Done" },
    
  "devDependencies": {
    "gulp": "~3.9.0",
    "gulp-load-plugins": "~0.10.0",
    "gulp-jshint": "~1.11.2",
    // other plugins here
  } 
}

Then, add the Gulp task to our gulpfile:

var sources = ['./Scripts/*.js', '!./Scripts/libs/*.js'];

gulp.task('watch-lint', function () {
    gulp.watch(sources, ['lint']);
});

gulp.task('lint', function () {
    return gulp.src(sources)
        .pipe(plugins.jshint({
            strict: true,
            predef: [""]
        }))
        .pipe(plugins.jshint.reporter('default'))
        .pipe(plugins.jshint.reporter('fail'))
    ;
});

Here’s a line by line breakdown of the important stuff:

• Line 1: Define the JavaScript files we want linting, excluding any third party libraries and frameworks
• Line 3-5: Set up a watch to run the linting task whenever we make a change to our JavaScript
• Line 7: Set up our linting task
• Line 8-12: Send our files to JSHint. JSHint has a number of options you can set to define how lenient or strict you want it to be. You can look at the docs to get an idea of what you can set. Here I’m only using the strict option (here is a good blog explaining strict mode). There is also a section to define any global variables you’re using that should be ignored – for example if you’re using Angular you’ll want to add “angular” here
• Line 13-14: Send the output of the linting to some inbuilt reporters – one to output the results to console and one to raise an error for any failures

Now we can add the watch task to our default task, and we’ll have linting every time we change our code.

Tests with Karma

One of the great things about Gulp is that you can get a Gulp task to run other Node based packages directly. For example, you can run Karma within a task, without having to use a plugin. Combined with PhantomJS, a headless browser, you get a great unit testing framework for your JavaScript. First, add the packages:

{
  "scripts": { "install": "echo Done" },
    
  "devDependencies": {
    "gulp": "~3.9.0",
    "karma-jasmine": "~0.3.6",
    "karma": "~0.13.9",
    "karma-phantomjs-launcher": "~0.2.1"
    // other plugins here
  } 
}

Then, we’ll need to add our karma config file (karma.conf.js) at the root of the project :

module.exports = function (config) {
    config.set({
        files: ['./Scripts/*.js',
                './Scripts/Specs/**/*.js',
        ],

        exclude: ['./Scripts/libs/*.js'],

        frameworks: ['jasmine'],

        autoWatchBatchDelay: 1000,

        reporters: ['progress'],

        browsers: ['PhantomJS']
    });
};

One thing I’ve found using Karma in Visual Studio: often Karma can crash in Visual Studio with an “EBUSY” error. I haven’t found an adequate explanation for this yet – I think it’s because when you modify and save a file Visual Studio writes a temporary file to replace the file you are saving, which doesn’t sit well with Node. This can be mitigated by setting the “autoWatchBatchDelay” to 1000, which waits 1 second after any change before running tests.

Finally, our gulp task:

var Server = require('karma').Server;

gulp.task('runTests', function () {
    new Server({
        configFile: __dirname + '/karma.conf.js',
        singleRun: false
    }).start();
});

Here, we are simply firing up Karma from within a Gulp task and passing in our karma config file. We are also telling Karma to watch our files for any changes and re-run our tests if they change. If you are going to run this task on a build server (and you should), you’ll need to configure the singleRun variable here. This can be done by using yargs to pass in a variable to switch this behaviour.

There we are! There is a commit of all the changes here, which includes a sample test. I highly recommend using both linting and unit tests when writing JavaScript code, and Gulp is great at running these tasks for us.

Next, in the last past I want to share my thoughts on Gulp, specifically within the .NET ecosystem.

• Part 1: Setting up Gulp with Visual Studio
• Part 2: Getting SASSy
• Part 3: Deployment: minification and concatenation
• Part 4: Code quality: tests and linting
• Part 5: Thoughts on Gulp

Small is beautiful

To increase the speed of a site, and to save bandwidth, we want to:

• Have as few requests for the smallest files possible
• Cache these files for as long as possible

We can achieve this by concatenating and minifying the files, then adding a hash of the files contents to the filename so they can be cached indefinitely.

There are a number of different ways of approaching minification and concatenation with Gulp, and a lot of different plugins and plugin combinations. The following is a style that has worked for me, you may need to adjust it to suit your own workflow.

Plugins

There are a number of plugins we will need:

• gulp-useref for concatenation
• gulp-uglify and gulp-minify-css for minifcation of JavaScript and CSS respectively
• gulp-rev and gulp-rev-replace for revisioning
• gulp-filter for filtering files to process them independently

Lets add these to the package.json:

{
  "scripts": { "install": "echo Done" },
    
  "devDependencies": {
    "gulp": "~3.9.0",
    "gulp-load-plugins": "~0.10.0",
    "gulp-useref": "~1.3.0",
    "gulp-uglify": "~1.2.0",
    "gulp-minify-css": "~1.2.1",
    "gulp-rev": "~5.1.0",
    "gulp-rev-replace": "~0.4.2",
    "gulp-filter": "~3.0.0"
  } 
}

Referencing scripts and CSS

Now we’ll want to add our CSS and JavaScript files to our pages. These are put in commented build blocks that let gulp-useref know where a section it should concatenate is. So, at the top of the page (here it’s the _Layout.cshtml page) we reference the CSS:

<!-- build:css /dist/site.css -->
    <link href="/Content/bootstrap.css" rel="stylesheet" />
    <link href="/Content/Site.css" rel="stylesheet" />
<!-- endbuild -->    

In the first comment, we define two things:

• The first part, “build:css” tells useref that this be a CSS block
• The second is the location of the concatenated file that will eventually be referenced (more on this later)

We do the same at the bottom with the JavaScript:

<!-- build:js /dist/libs.js -->
    <script src="/Scripts/Libraries/jquery.js"></script>
    <script src="/Scripts/Libraries/bootstrap.js"></script>
    <script src="/Scripts/Libraries/respond.js"></script>
<!-- endbuild -->

We can have multiple build blocks of the same type (e.g. multiple JavaScript blocks) so we can have one for libraries (Angular, jQuery etc) which will change less frequently than the code we write.

The Gulp task

Now for the biggie – the actual task itself. I’ll put it here and go through it in more detail below:

gulp.task('minifyFilesForRelease', function () {

    var cssFilter = plugins.filter('**/*.css', { restore: true });
    var jsFilter = plugins.filter('**/*.js', { restore: true });

    var assets = plugins.useref.assets();
    gulp.src('./**/*.cshtml')
        .pipe(assets)

        //Process JavaScript
        .pipe(jsFilter)
        .pipe(plugins.uglify())
        .pipe(plugins.rev())
        .pipe(assets.restore())
        .pipe(jsFilter.restore)
        
        //Process CSS
        .pipe(cssFilter)
        .pipe(plugins.minifyCss({
            keepSpecialComments: 0
        }))
        .pipe(plugins.rev())
        .pipe(assets.restore())
        .pipe(cssFilter.restore)

        .pipe(plugins.useref())
        .pipe(plugins.revReplace({
            replaceInExtensions: ['.js', '.css', '.html', '.cshtml']
        }))
        .pipe(gulp.dest(function (data) {
            return data.base;
        }
        ));
});

Here’s a line by line breakdown of what’s happening:

• Line 1: Define the task
• Line 3-4: Setup some filters we will use later to separate out the CSS and JavaScript for individual processing
• Line 6: Reference the assets object of useref which will return the build blocks we setup earlier
• Line 7: Get all our cshtml files
• Line 8: Get all our build blocks in these files
• Line 11: Filter our build blocks to just the JavaScript
• Line 12: Minify the JavaScript
• Line 13: Revision the JavaScript
• Line 14-15: Remove the filters, so we have access to all our build blocks again
• Line 18: Filter our build blocks to just the CSS
• Line 19-21: Minify the CSS removing any and all comments
• Line 22: Revision CSS
• Line 23-24: Remove the filters again
• Line 26: Replace the commented, unprocessed build block in our cshtml files with the processed ones
• Line 27-29: Add the revision number to the processed files (note we need to include cshtml files here)
• Line 30-32: Replace the original cshtml files. We need to specify a destination here, and as we have views in multiple  folders we can use this technique by Yiling to overwrite a file regardless of its location

Phew, there’s a lot there! If you run this task each build block in our cshtml files will be replaced with a single, minified, revisioned file.

Before:

After:

Running in Release mode

We only want to run this task when we’re putting together a release, whether that’s manual or (much better!) automated. To do this, add a PreBuildEvent that runs the Gulp task before a release build. Go to the project properties, then Build Events and add the following to “Pre-build event command line”:

if $(ConfigurationName) == Release (
   cd $(ProjectDir)
   npm install
   gulp minifyFilesForRelease
)

This will then run our minifying task only on a Release build.

Lastly, as we are adding the minified files to a “dist” folder that doesn’t exist during development, we’ll need to add this folder after each build. Open up the csproj for the web project, and at the very end add the following:

<Target Name="AfterBuild">
    <ItemGroup>
      <Content Include="dist\*.*" />
    </ItemGroup>
 </Target>

This will include the contents of our dist folder after each build.

Wrapping up

We now have an automated process for minifying and concenating files that doesn’t intefere with the development process. The commit for everything is here.

Happy Gulping!

In my previous blog I showed you how to set up Visual Studio for use with Gulp. Now I want to show you how you can use it in everyday development, including processing SASS, running tests, linting your code and getting ready for deployment.

In this series of posts I’ll talk about how you can use Gulp with Visual Studio to run your common front end tasks:

• Part 1: Setting up Gulp with Visual Studio
• Part 2: Getting SASSy
• Part 3: Deployment: minification and concatenation
• Part 4: Code quality: tests and linting
• Part 5: Thoughts on Gulp

The code I’ll be showing is based off a standard MVC 5 project. If you want the code I’ve created a GitHub project for it here.

Basic Gulpfile + package.json

Throughout these examples I’ll be using gulp-load-plugins – an awesome meta-plugin that takes all your Gulp plugins from your package.json and attaches them to an object. At the start, the package.json file looks like this:

{
  "scripts": {"install": "echo Done"},
    
  "devDependencies": {
    "gulp": "~3.9.0",
    "gulp-load-plugins": "~0.10.0"
  } 
}

N.B. I like outputting something to the console after installing packages as the default is to not output anything, which can be a little confusing!

The basic gulpfile looks like this:

var gulp = require('gulp');
var plugins = require('gulp-load-plugins')();

gulp.task('default', ['']);

Now, any plugins we add to our package.json will immediately be available on the plugins object.

Processing SASS

I’m a big fan of SASS for developing CSS as it helps keep my CSS clean and DRY. SASS files need processing, and Gulp can do this for you. As a bonus, we can add sourcemaps so that when we inspect the page in Firefox, Chrome or IE 11 we refer back to the original SASS rather than the processed CSS. As a double bonus, we can use LiveReload to reload the page and automatically see our changes whenever we make a change to the SASS (note with LiveReload on Windows you’ll need the Chrome extension for the best experience).

First off, we add the SASS file. For this demo project, based on MVC 5, this is simple – rename the Site.css file to Site.scss. As SASS is a superset of CSS, all valid CSS is valid SASS.

Next, we add the plugins we’ll need to the package.json:

{
  "scripts": { "install": "echo Done" },
    
  "devDependencies": {
    "gulp": "~3.9.0",
    "gulp-load-plugins": "~0.10.0",
    "gulp-sass": "~2.0.4",
    "gulp-sourcemaps": "~1.5.2",
    "gulp-autoprefixer": "~2.3.1",
    "gulp-livereload": "~3.8.0"
  } 
}

Next, add the gulp tasks – one to process the SASS into CSS, and another to watch the SASS files and recompile whenever we make a change:

var gulp = require('gulp');
var plugins = require('gulp-load-plugins')();

gulp.task('sass', function () {
    gulp.src('./Content/*.scss')
        .pipe(plugins.sourcemaps.init())
        .pipe(plugins.sass())
        .pipe(plugins.autoprefixer())
        .pipe(plugins.sourcemaps.write())
        .pipe(gulp.dest('./Content'))
        .pipe(plugins.livereload())
    ;
});

gulp.task('watch-sass', function () {
    plugins.livereload.listen();
    gulp.watch('./Content/*.scss', ['sass']);
});

gulp.task('default', ['watch-sass']);

Here’s a line by line breakdown of what is happening within our ‘sass’ task:

• Line 5: Grab all our SASS files from the Content folder. At the moment this is just the Site.scss file, but as we build this up we’ll want to add different SASS files for different parts of the project
• Line 6: Initialise the sourcemaps. Note that this must be done before they get processed
• Line 7: Process the SASS to generate the CSS
• Line 8: Use autoprefixer to add any vendor-specific prefixes to the CSS. The defaults for the plugin target any browser with >1% share, which is eminently sensible
• Line 9: Insert the sourcemaps into the generated CSS
• Line 10: Write the generated CSS back to the Content directory
• Line 11: Notify LiveReload that we’ve made changes

The ‘watch-sass’ task simply tells LiveReload to start listening, watches our SASS files for any changes, and when they are changed runs our ‘sass’ task. This watch has also been added to the default task, so will run automatically. Now, when we make a change to the Site.scss file the css will automatically get generated, and we get all the other goodness from our other plugins too.

Sourcemaps

Below I’ve changed the Site.scss file so that the text in the Jumbtron is blue, and have inspected it in Chrome. You’ll notice the red box in the inspector refers to the Site.scss file (not the generated Site.css file) , thanks to sourcemaps.

LiveReload

If you open the site in Chrome and start the LiveReload extension, any time a change is made to the SASS the web page will automatically reload with the changes. Here the color of the jumbotron text is switching between red and blue, and automatically updating:

Wrapping up

So there we are! Automatic processing of SASS with Gulp, along with a few other goodies. The commit for eveything I’ve shown is here.

One thing to note is that in this task we didn’t do anything minification to the CSS – I’ll come on to that in my next post.

What is Gulp?

Gulp is a build process for front end development. Gulp automates all of the repetitive tasks you have to do to prepare your front end files (such as JavaScript and CSS) for development and deployment. Gulp uses small, simple, composable plugins to that let you define exactly what happens to the files in your build.

Common tasks you can get Gulp to automate

• Minifying JavaScript/CSS/images
• Concatenating JavaScipt/CSS files
• Running JavaScript lint tooling
• Compiling your SASS/LESS
• Compiling TypeScript/CoffeeScript
• Running JavaScript tests (eg Karma, Jasmine, Mocha)
• Transpiling ES6 into ES5
• Minifying Angular without breaking Dependency Injection
• …and many more

To install and use Gulp, you’ll need npm. Which begs the question…

What is npm?

npm is the package manager for [simple_tooltip content=’Node is a runtime environment built in JavaScript. It is somewhat equivalent to the .NET CLR – and for getting set up it is not necessary to know much more than that’]Node[/simple_tooltip]. It’s a front-end equivalent to NuGet – you can use it to install and update packages and their dependencies. We’ll be using it to install Gulp and all its plugins.

How to set up Gulp in Visual Studio

Currently you can set up Gulp in Visual Studio 2013 and 2015. If you have 2015, skip the first 3 points as they come with Visual Studio 2015. Note that the screenshots are taken using Visual Studio 2013 with a standard MVC 5 project template.

1. Install Node/npm.

   You can install Node from here. What we’re really after here isn’t Node, it’s npm (which comes installed with Node).

2. Install Gulp.

   After installing npm, open up a Console window, and enter:

   npm install gulp -g

   This will install Gulp. The “-g” option installs it globally, so you can run Gulp commands anywhere.

3. Install the Grunt Launcher Visual Studio Extension.

   [simple_tooltip content=’Grunt is another build process for front end development. Grunt favours configuration, while Gulp favours code. I am not going to get into the merits of each here!’]Grunt Launcher[/simple_tooltip] provides menu options to install npm packages and run Gulp tasks from within Visual Studio.

4. Add Gulp to your project

   At the root of your web project, add a new JSON file called package.json with the following contents:

   {
       "devDependencies": {
           "gulp":  "3.9.0"
       }   
   }
   

   The [simple_tooltip content=’Note a standard package.json will contain other information such as a description, author and a repository for the code’]package.json[/simple_tooltip] is a npm file we use to define metadata about our project. Here we are listing the (front end) dependencies for developing our project – this will include plugins for running Gulp tasks. For now, we just want to add Gulp to our project.

   Now, right-click on the package.json file and click “NPM install packages” (if you’ve installed Grunt Launcher) to add it to our project.

   

   You can ignore those warnings for now – you can add the metadata it is complaining about later.

   This will add a node_modules folder in your project folder, where all our npm packages will be kept. Within that will be Gulp.

   

5. Add a Gulpfile

   Next we want to add a Gulpfile, where we will define the steps for our build process. Add a JavaScript file called gulpfile.js to the root of the project, and add the following code:

   var gulp = require('gulp');
   
   gulp.task('default', function () {
       console.log('Hello world!');
   });
   

    

   
   In line 1 we load Gulp. On line 3 we define a new Gulp task. A task is a step, or a set of steps, that Gulp will execute. A task takes a function that it will run – in this case we simply log to the console. A task also has a name – here it’s called “default” and as you might guess it runs automatically when we run Gulp.

   We can now run this task by right-clicking on the gulpfile.js and clicking the “Gulp” menu item added by the Grunt Launcher extension.

   

    

   Now Visual Studio is running our task, and outputting “Hello world!” to the output window. We’re done! Well, not quite…

Doing something actually useful

Hello World is all well and good, but what about automating something useful? One common task is minifying CSS files, and that’s what we’ll do here.

1. Add the plugins

   Change your package.json file to the following:

   {
       "devDependencies": {
           "gulp": "3.9.0",
           "gulp-uglifycss": "1.0.4",
           "gulp-rename": "1.2.2"
       }
   }
   

   Here we’ve added two new packages, gulp-uglifycss (which minifies css) and gulp-rename (which renames files passed into it). We can now use these in our task.

2. Modify the task

   Change the gulpfile.js to the following:

   var gulp = require('gulp'),
       uglifycss = require('gulp-uglifycss'),
       rename = require('gulp-rename');
   
   gulp.task('default', function () {
       return 
            gulp.src(['./Content/*.css', '!./Content/*.min.css'])
           .pipe(uglifycss())
           .pipe(rename({
               suffix: ".min",
               extname: ".css"
           }))
           .pipe(gulp.dest('./Content'));
   });
   

   Here’s a line by line breakdown of what the task is doing:
   Line 1-3: Add references to the plugins we just added in our package.json
   Line 7: Get the CSS files from our Content directory (excluding any .min files)
   Line 8: Pipe (i.e. pass) the files to gulp-uglifycss, which minifies them
   Line 9-12:  Rename the minified files to include a .min suffix
   Line 13: Save the minified files back to our Content folder

   Now, we can run this task to automatically minify all our CSS files.

   And there we are! We now have a process that will save us from a repetitive boring task, and we can build on this to automate lots of other tasks.

Next steps

Note that for brevity I have excluded everything that isn’t necessary for getting Gulp set up in Visual Studio. There are a few things you will want to do next to get fully set up:

1. Install the Task Runner Explorer extension

   You can install the Task Runner Explorer Visual Studio extension to run tasks automatically on each build or every time the project opens.

2. Install NPM Package Intellisense

   NPM Package Intellisense gives you package intellisense when installing packages and plugins.

3. Add more plugins

   Nearly any task you want to automate will have a plugin here – there are over 1500 at time of writing.

Happy Gulping!