tl;dr

Do not use npm update with any package which use custom dist-tags.

Dist-tags

This package has officially those 3 last versions: 1.0.0, 1.0.1 and 1.0.2. But inside the dist-tags (read more), we have two of them: the classic latest which refers to the latest stable release of the package and a custom one, named canary, indicating the last non-stable release of the package. If you are wondering if real projects are using such tags, the answer is yes, the npm package is using latest and next tags for it’s weekly pre-release (read more).

Currently, latest points to 1.0.1 and canary to 1.0.2. Meaning that if you run npm view test-npm-update, you will have something like:

{
  "name": "test-npm-update",
  "description": "...",
  "dist-tags": { "latest": "1.0.1", "canary": "1.0.2" },
  "versions": ["1.0.0", "1.0.1", "1.0.2"],
  "version": "1.0.1"
}

Versions

It’s important to realize a few things here. 1.0.2 is a released version just as 1.0.0 and 1.0.1 and we will say it’s the greatest one (as in the biggest number according to semver) but not the latest one (as in the one tagged with the latest dist-tag). We need to make such distinction to fully understand what will happen after that.

So, when you run npm view test-npm-update, it actually runs npm view test-npm-update@latest, meaning it will grab the informations of the latest version. But maybe some other versions have been released with a custom tag after this one. For me, so far, so good. NPM is doing exactly what I would expect. If I want a custom release such as the canary one, I can run npm view test-npm-update@canary and it will display infos about the 1.0.2 version. In fact, but I might be wrong but I except NPM to always use the latest version (aka the latest dist-tag) by default if I don’t specify anything. That’s what you can mostly read all over the NPM documentation.

But remember, 1.0.2 is inside the versions array just like any other version. So first warning, if you use such metadata for whatever stuff you are doing, do not assume that the greatest version inside the versions array is the latest one.

Install

Now, what if I run npm install test-npm-update? What would you expect to be installed? 1.0.1 right? And of course it will be this version, the latest one. That’s normal, after all, latest is the default one. All good here.

What if I clean my folder and then run npm install test-npm-update@^1.0.0? Guess what, 1.0.1 will be installed. And I’m totally ok with that. I asked for the best 1.x.x version and I’m glad to have the latest one since it matches.

package.json

But most of the time, you don’t install or update from command line, you have a package.json file with a range inside it. Let’s say we have the following one:

{
  "name": "awesome-project",
  "version": "0.0.0",
  "dependencies": {
    "test-npm-update": "^1.0.0"
  }
}

Pretty classic, right? Now, for the purpose of the demo, let’s say we currently have the 1.0.0 version of test-npm-update locally installed. If you want to reproduce, just create an empty folder, then create a package.json inside it with the previous content and run npm install test-npm-update@1.0.0 to force the install of an old version.

Done? Cool, let’s move forward. NPM has a command to test if you have outdated versions locally installed. Which is our case. Let’s check that by running npm outdated. You should have something like:

Package          Current  Wanted  Latest  Location
test-npm-update  1.0.0    1.0.2   1.0.1

Wait a minute? I’m ok with current (the locally installed) being 1.0.0 and latest (matching the dist-tag) being 1.0.1 but wanted is supposed to be the best matching version I should install according to package.json. How can it be greater than latest?

Actually, it’s all ok according to the NPM documentation. After all, the package.json range is ^1.0.0 which means the greatest possible version without changing the first non-zero digit. And among all our versions (see the versions array from npm view), both 1.0.1 and 1.0.2 match this range, but since 1.0.2 is greater than 1.0.1, the wanted version is 1.0.2.

I didn’t expect that to be honest. That’s not wrong but I can’t help myself finding that strange.

Install again

Quick mention to the fact that if I run npm install with my package.json in an empty folder (aka without the 1.0.0 version already installed), it will still install 1.0.1 version. That’s ok according to latest being the default one. Back to our outdated 1.0.0 version.

Update

Things start to get really ugly now. So, npm outdated just told me I have an old local version. I should probably update it, and NPM has a command for that. Let’s run npm update. To be honest, I wasn’t sure anymore what would be installed locally. I mean, I would have normally expected the 1.0.1 version. My brain was like “It should be the greatest stable version which match the range”, with stable meaning lower or equal to the latest tag, but for NPM, it’s more like “It should be the greatest version which match the range. Period.”. And it makes all the difference. My brain stops at 1.0.1 as the latest stable but NPM browse all version, including any custom dist-tags, including the canary version.

At the end, running npm update will install 1.0.2 version. This is wrong. According to documentation:

This command will update all the packages listed to the latest version (specified by the tag config), respecting semver.

I read that as the latest version according to latest dist-tag. But we just updated to a version beyond this latest version. In any case, this is super dangerous! It means you can update to non-stable versions without even noticing it.

What if we didn’t have the 1.0.0 already installed? Since npm update also install missing packages, it will indeed install test-npm-update according to package.json and, of course, to the 1.0.2 version.

Conclusion

IMHO, I think this is way too dangerous, npm update should be capped by the latest version, and so should npm outdated. By default, no command should target versions beyond latest dist-tag. Also, it seems inconsistent to have install and update both capable of installing a missing package from a package.json file but not to the same version.

I raised an issue on Github, we will see. Be careful from now on.

Thanks for reading! Spread the word.

Personal ad

It might be a bit too early to speak about that, but if you need an outdated command which is actually capped by the latest tag and also support other package managers (like Bower), please check my outdated project. It’s not ready at all yet but it will be in the next few days, promise.

TL;DR ‘coz I’m a h4k3r

Full source code of the demo is on GitHub with a nice and simple README on how to bootstrap the project.

What are you talking about dude?

Play Framework is an HTTP server written in Scala. It is super cool and you should probably give it a try. Let’s say you are using it, you really enjoy how productive it is, only having to refresh your browser to see all your code modifications, but you want to go further, you want live-reloading! There are several tools doing that.

Introducing BrowserSync. It provides 2 killers features. One is live-reloading: it can monitor resources, and when it detects some modifications, it will reload the browser page to load them. Even better, if possible, it will hot deploy them, meaning they will be loaded without reloading the page. That’s the case for CSS files which are loaded and the page repainted to display the new design without any refresh. The second one is to keep synchronized several browsers across devices. And we are not talking about resources only but about all user actions. If you scroll on one browser, it will scroll on all connected browsers, same for clicking on a button, and so on…

Let me say it again with a concrete example. You are on your computer, one screen with your source code, another one with Chrome opened and a last one with Firefox. You also have an iPad tablet and an Android smartphone connected to your computer. All of those have your application main page opened. Each time your edit your code, they will all reload and display the new result (hot deploy in case of CSS). When you are ready to test interactions, you just go to, let’s say, your Chrome browser and start scrolling and clicking. All your screens will start moving and doing the actions everywhere. Talk about increasing productivity!!

What about assets that require compilation? Such as SCSS, LESS, Stylus, CoffeeScript, etc… No biggy bro (or sis), we have you covered. In this demo, we will use Gulp to monitor those resources, compile them, and pass them to BrowserSync for live-reloading (obviously not losing the hot deploy if possible). You can freely use whatever build tool you want, be it Grunt, Broccoli, …

Magic proxy

BrowserSync could be used as the web server, that’s the easiest way to use it, and I’m actually doing it when my Play server is only here to provide a REST API. Consider doing that when you don’t need Scala templates or when you cannot use them (for example if you want to embed all your standalone HTML files inside a Cordova / PhoneGap application).

But here, we want to use Play as the web server, meaning it will be responsible to serve all our assets: HTML, CSS and JavaScript. For that, we will need to use the proxy feature of BrowserSync. Long story short, it will start another web server, on its own port, and will display the exact content from the Play server but adding a bit of JavaScript magic in it so it can enable all its features. It is just like opening your Play application, just on a different port. By default, you would use the port 9000 for Play, in this demo, we will set the proxy on port 9001 (it would have been around 3000 by default, but I didn’t want anyone to freak out so I put it as close as possible to Play default one).

We also want to live-reload some of our resources. I kept it simple for this demo, but you are free to add as many as your want. We will use the files property of BrowserSync configuration and set and array of files to monitor (it supports wildcards). Here, we are monitoring all CSS files from public/stylesheets, JavaScript files from public/javascripts and HTML files from app/views. There are two important points to notice here. First, we are referencing files based on their path on the source code, not on their actual url. For example, public/stylesheets/main.css will be served by Play as assets/stylesheets/main.css, but BrowserSync only needs to know the real path, after that, you can map it to whatever url you want. Second, we are monitoring Scala templates, and it’s fine because when BrowserSync will detect a modification, it will reload the page, and Play will re-compile the template before serving it, so it will display the new version. It works just fine with both run and ~run , the latest being faster.

Here is a super small configuration for BrowserSync to enable such a proxy. You can read the online documentation to extend it.

var browserSync = require('browser-sync');

browserSync({
  // By default, Play is listening on port 9000
  proxy: 'localhost:9000',
  // We will set BrowserSync on the port 9001
  port: 9001,
  // Reload all assets
  // Important: you need to specify the path on your source code
  // not the path on the url
  files: ['public/stylesheets/*.css', 'public/javascripts/*.js', 'app/views/*.html'],
  open: false
});

What about compiled assets?

Right… right… there are assets that you cannot serve directly, you need to preprocess them. The easiest way to do that is using a build tool, just pick one among the best ones (Gulp, Grunt, Broccoli, Brunch, …) and enjoy. There is a really simple separation of concern here: the build tool only manage assets that are not served directly (LESS, CoffeeScript, …) and compile them into assets that are actually handled by BrowserSync (CSS, JavaScript, images, …) which then live-reload them.

It’s so easy that it’s nearly frustrating. In the demo, the main.css file is generated from main.less. I have two Gulp task to handle that: one for compilation and one to monitor any modification.

var gulp        = require('gulp');
var less        = require('gulp-less');

// Registering a 'less' task that just compile our LESS files to CSS
gulp.task('less', function() {
  return gulp.src('./resources/less/main.less')
    .pipe(less())
    .pipe(gulp.dest('./public/stylesheets'));
});

// Let's watch our LESS files and compile them at each modification
gulp.task('watch', function () {
  gulp.watch(['./resources/less/*.less'], ['less']);
});

Wait, where are sbt-web and webjars?

Yeaaaaah… so lately, Play has introduced all those stuff on managing assets using SBT, some plugins and webjars. It works just fine but for me, it’s just as strange as if I would install Play from NPM. It’s cool if people want to try to merge two super different worlds (front-end and back-end), be it Node.js or scala.js, but I’m totally not ready for that. As a full stack developer, I want to enjoy each world with its own ecosystem, not trying to bend one into the other.

That’s why in all my projects, front-end assets are handled by front-end tooling (mostly Gulp and NPM lately) and my back-end resources by back-end tools (SBT, Ivy, Maven, …). Not only do I get the best of each, but I do not have some limitations. For example, the other day, I came across a bug on a JavaScript library I was using. No biggy: fork the repo, correct the bug, do a pull request. And while waiting for the author to merge it (which can take some time), I am free to directly use my fork inside my package.json so it’s not blocking at all. It tooks me less than an hour. I would be curious on how to do that using webjars? Anyway, sorry for not planning any proof of concept using webjars if that’s what you are using.

And that’s it! As I said at the beginning of the article, the demo is on GitHub, feel free to clone and play with it. Enjoy and thanks for reading.