It has again been a while since I posted. I've been working on some projects but nothing I really felt like blogging about until now. One of the projects I was working on introduced me to Hugo, which is a static website generation engine that takes metadata and templates and generates a set of HTML pages from them so that it's ready to be uploaded to a simple web server such as S3. On another project, I used Gulp for the first time to automate some repetitive build tasks such as minification and also to run BrowserSync so that I could make changes and have them appear automatically in-browser after I saved them. For my latest project, I want to combine the two so that I can generate a Hugo site and then post-process it, and also have that processed site appear in-browser whenever I make any changes. I'm happy with how this has turned out and I didn't see any blog posts about how to do this so I figured it was a good time for a post.
The Goal
By the end of this post, what we will have is:
- Hugo generating index and content pages based on a simple template and some simple content, and updating that whenever the content or template changes.
- ES6 code will be automatically transpiled to JS.
- JS, HTML, and CSS will be automatically minified when building for production.
- It will be possible to run a browser and have it automatically refresh whenever any of the HTML, JS, or CSS changes (and is automatically processed).
The project is up on GitHub if you want to see the finished product, but I'll spend the rest of the blog post going through the process step-by-step.
Hugo
The first step is to install and set up Hugo. They have a great quickstart guide on their site, so it's best to just go through that if you haven't already. What you should end up with is a basic site with an about page and a post, and you should be able to spin up a simple server that will watch for changes and automatically update the site. However we want to go another step and process that with Gulp, so we won't be using the Hugo server. We do want to use Hugo to watch for changes and generate the updated site, and for that we use the command:
hugo -w -s .\hugo-site -d ..\hugo-generated --disableRSS
This will (-w)atch the (-s)ource folder, which I put into a hugo-site subfolder, and generate it into the (-d)estination of another top-level subfolder hugo-generated. I've also disabled the generation of RSS feeds and haven't used a theme since I used a simple layout instead, but you can add or change these options as needed.
It's a bit annoying to have to remember this command, and we're going to be using npm anyway, so we might as well make a package.json file and add the command as a script, so we just need to remember "npm run hugo".
{
"private": true,
"engines": {
"node": ">=0.12.0"
},
"scripts": {
"hugo": "hugo -w -s .\\hugo-site -d ..\\hugo-generated --disableRSS"
}
}
Gulp First Steps
Now that we have a Hugo generated folder, what we want to do is use Gulp to pick up any files from there, transform them as needed, and output them to a final distribution folder I'll name gulp-dist. It might be possible to instead make a Gulp task to run Hugo and do everything in one go, but this way is simpler and it lets me see exactly what both Hugo and Gulp are doing in case of any problems.
The first thing we'll get Gulp to do is to simply copy the hugo files to the new gulp-dist folder, and make some helper scripts to clean up the gulp-dist and hugo-generated folders. We'll clean the gulp-dist folder before each build, but leave the cleaning of hugo-generated to be on-demand since we would need to manually trigger a Hugo generation after we do that. So what we need to do is install Gulp and the del library using npm.
npm install --save-dev gulp del
And then make a small gulpfile.js with the build and clean tasks.
var gulp = require('gulp');
var del = require('del');
var hugoBase = './hugo-generated';
var distBase = './gulp-dist';
gulp.task('clean', function() {
del([distBase + '/**/*']);
});
gulp.task('clean-hugo', function() {
del([hugoBase + '/**/*']);
});
gulp.task('build', function() {
return gulp.src(hugoBase + '/**/*')
.pipe(gulp.dest(distBase));
});
gulp.task('default', ['clean'], function() {
gulp.start('build');
});
I won't spend too much time explaining how the gulp scripts work - hopefully it's clear enough by itself but if not you can dig into the Gulp docs. Basically though, the clean tasks are just deleting files, the build task is just copying files, and the default task first runs clean then build. This means we can just run "gulp" instead of "gulp clean" then "gulp build".
BrowserSync
Now let's get Gulp to do something useful - let's make a task that will open a browser and automatically copy the source files and then refresh the browser whenever any changes are made. To do this, we need to use BrowserSync and file watchers.
npm install --save-dev gulp del
// ...
var browserSync = require('browser-sync').create();
gulp.task('serve', ['build'], function() {
browserSync.init({
notify: false,
server: {
baseDir: distBase
},
reloadDelay: 1000,
reloadDebounce: 1000
});
gulp.watch(hugoBase + "/**/*", ['build']);
gulp.watch(distBase + "/**/*").on('change', browserSync.reload);
});
// ...
When we run "gulp serve", a browser should be automatically opened, and if we still have the Hugo generator running in the background, we can make changes to the Hugo content, see Hugo generate that into hugo-generated, then see Gulp copy that to gulp-dist and refresh the browser. Magical! The BrowserSync options we are using are notify, which removes an annoying popup on the browser, server, which says where to serve from, and reloadDelay/reloadDebounce, which helps to avoid multiple refreshes when Hugo regenerates all the files. The watcher on hugo-generated tells Gulp to re-build whenever that changes, and the watcher on gulp-dist tells BrowserSync to refresh the browser whenever that changes.
ES6 Transpiling
The next thing I wanted to do was to have Gulp convert ES6 code into plain JavaScript, so that I can use the new features but the code continues to work on all browsers. To do this we'll use Babel, and before writing any new code, we can change the gulpfile itself to ES6. So install babel and the ES6 (a.k.a. ES2015) preset:
npm install --save-dev babel-core babel-preset-es2015
Make a simple .babelrc file to tell Babel we want to transpile ES6.
{ "presets": ["es2015"] }
And then rename gulpfile.js to gulpfile.babel.js and convert it to ES6.
'use strict';
import gulp from 'gulp';
import del from 'del';
import bs from 'browser-sync';
let browserSync = bs.create();
let hugoBase = './hugo-generated';
let distBase = './gulp-dist';
gulp.task('clean', () => {
del([distBase + '/**/*']);
});
gulp.task('clean-hugo', () => {
del([hugoBase + '/**/*']);
});
gulp.task('build', () => {
return gulp.src(hugoBase + '/**/*')
.pipe(gulp.dest(distBase));
});
gulp.task('serve', ['build'], () => {
browserSync.init({
notify: false,
server: {
baseDir: distBase
},
reloadDelay: 1000,
reloadDebounce: 1000
});
gulp.watch(hugoBase + "/**/*", ['build']);
gulp.watch(distBase + "/**/*").on('change', browserSync.reload);
});
gulp.task('default', ['clean'], () => {
gulp.start('build');
});
Hopefully everything is working exactly as it was before, but now we're in a better position to write some ES6 and both transpile that to JS and generate sourcemaps at the same time. We'll make a simple JS file under hugo-generated/scripts and add that to the layouts so it's automatically included in all generated HTML.
let test = "testing"; setTimeout(() => console.log(test), 1000);
Now we need to install the Gulp plugins that will allow us to transpile and generate sourcemaps.
npm install --save-dev gulp-sourcemaps gulp-babel
And update the gulpfile to do the transpiling and sourcemap generation. At the same time we'll make the file selectors more specific so we can tell Gulp to process each type of file differently.
// ...
import sourcemaps from 'gulp-sourcemaps';
import babel from 'gulp-babel';
gulp.task('html', () => {
return gulp.src(hugoBase + '/**/*.html')
.pipe(gulp.dest(distBase));
});
gulp.task('extras', () => {
return gulp.src(hugoBase + '/sitemap.xml')
.pipe(gulp.dest(distBase));
});
gulp.task('styles', () => {
return gulp.src(hugoBase + '/styles/**/*.css')
.pipe(gulp.dest(distBase + '/styles'));
});
gulp.task('scripts', () => {
return gulp.src(hugoBase + '/scripts/**/*.js')
.pipe(sourcemaps.init())
.pipe(babel())
.pipe(sourcemaps.write('.'))
.pipe(gulp.dest(distBase + '/scripts'));
});
gulp.task('build', ['html', 'scripts', 'styles', 'extras']);
gulp.task('serve', ['build'], () => {
browserSync.init({ ... });
gulp.watch(hugoBase + "/**/*.html", ['html']);
gulp.watch(hugoBase + "/scripts/**/*.js", ['scripts']);
gulp.watch(hugoBase + "/styles/**/*.css", ['styles']);
gulp.watch([distBase + "/**/*.html", distBase + "/scripts/**/*.js", distBase + "/**/*.js"]).on('change', browserSync.reload);
});
// ...
And that's all we need to do! You should now be able to see Gulp taking the ES6 code from hugo-generated and using it to create JS and mapping files in the gulp-dist folder.
Linting
Another thing that's useful to have in the world of JavaScript is linting, so that you can have a bit more confidence that the code you're writing is good quality and free of any obvious bugs. This is another thing that is very easy to do when using Gulp. We'll be using ESLint, and all we need to do is install it and create a simple Gulp task.
npm install --save-dev gulp-eslint
// ...
import eslint from 'gulp-eslint'
var lintOptions = {
extends: 'eslint:recommended',
rules: {
quotes: [2, "single"],
"no-console": 0
},
env: {
"es6": true,
"browser": true
}
};
gulp.task('lint', () => {
return gulp.src([hugoBase + '/scripts/**/*.js'])
.pipe(eslint(lintOptions))
.pipe(eslint.format())
.pipe(eslint.failAfterError());
});
You can leave this task as something completely separate to run when you want, or make it a prerequisite for the build task, or even have separate tasks for development lint options and production lint options. I just have it alongside the clean script and run it when I run the default Gulp script, before doing any other processing.
Minification
The last thing I wanted to do was to have an option to build everything minified, so that it was ready to be uploaded to my webserver. The way I decided to do this was just to have an option I could pass to the "gulp" or "gulp build" commands which specified that I was building for production, while leaving everything else as-is. This means that when I'm developing, everything will stay un-minified, and I'll only do the minification when I'm ready to deploy. To do this, we'll need a few different Gulp plugins:
- gulp-util allows us to access arguments we pass to the gulp commands (we'll be using --production only)
- gulp-if allows us to perform processing only if that argument is passed
- gulp-uglify minifies JavaScript
- gulp-cssnano minifies CSS
- gulp-htmlmin minifies HTML
As with most Gulp code in this post, doing this is fairly simple and self-explanatory.
npm install --save-dev gulp-util gulp-if gulp-uglify gulp-cssnano gulp-htmlmin
// ..
import util from 'gulp-util';
import gulpif from 'gulp-if';
import uglify from 'gulp-uglify';
import cssnano from 'gulp-cssnano';
import htmlmin from 'gulp-htmlmin';
gulp.task('html', () => {
return gulp.src(hugoBase + '/**/*.html')
.pipe(gulpif(util.env.production, htmlmin({collapseWhitespace: true})))
.pipe(gulp.dest(distBase));
});
gulp.task('scripts', () => {
return gulp.src(hugoBase + '/scripts/**/*.js')
.pipe(sourcemaps.init())
.pipe(babel())
.pipe(gulpif(!util.env.production, sourcemaps.write('.')))
.pipe(gulpif(util.env.production, uglify()))
.pipe(gulp.dest(distBase + '/scripts'));
});
gulp.task('styles', () => {
return gulp.src(hugoBase + '/styles/**/*.css')
.pipe(gulpif(util.env.production, cssnano()))
.pipe(gulp.dest(distBase + '/styles'));
});
// ...
Now all we need to do is run "gulp --production" and our dist folder will be cleaned and then populated with fully minified JS/HTML/CSS!
Conclusion
And that's it! We have done everything we set out to do, and it was pretty easy. Gulp is pretty great like that. Hopefully this post has helped someone out there, and remember that the full code is available on GitHub in case anyone wants to use it as a starting point.
