Neil Middleton
Doing lots of things - fast
Recently, we’ve been spending a lot of time looking into the corners of the web that a lot of other agencies don’t often get the chance to – that of the high performance web. In essence, I’m talking the concept of doing lots of little things fast. Very fast.
You may not have really seen this stuff out there, but you’ve certainly used it. Google manages over 1 billions searches per day. Twitter processes 200 million new tweets per day. Facebook deals with 30 billion new pieces of content each month.
So, how do these companies deal with this volume? The key lies in processing queues, doing lots of little things fast, which is where we’ve been picking up some experience for some time.
As the current land lies there are a few options available to the budding developer – traditional web programming, specialised job queues built using tools such as Java, Erlang or Scala, or the somewhere-in-between option of frameworks such as Joyents Node.js.
As we’re web developers we like to stick with what we know and focus on the tools that let us use the tools already at our disposal such as Ruby and Javascript, the two languages that every single Kyan developer lives and breathes.
The key with these hybrid frameworks that makes them different to a traditional web framework, such as Rails, is one of architecture in that they are designed to be ‘non-blocking’.
A blocking framework, such as Rails suffers from a performance point of view in that it spends a fair amount of time waiting for other processes to complete. Database queries need running, DNS lookups need to be made, files need to be read – all of which take time, and during this time the process is sat there twiddling its little electronic thumbs. Whilst this makes the application build super simple and stable, it’s not the quickest to run.
So, what about non-blocking options? A few options exist out there already. Tools such as Cramp, Sinatra-Synchrony and the current Web 3.0 (there, we said it) darling Node.js. The key to all of these is that when we need to do something which takes time such as a database query, instead of sitting around waiting for the query to come back, the process will go off an amuse itself doing part of another request, meaning that your throughput and CPU utilisation go through the roof, making more use of the machine at hand and reducing cost.
However, there are drawbacks. Node.js for instance is very new, but maturing at an alarming rate. Due to this the testing and monitoring stories aren’t fully fleshed out, but are getting there fast. With the adoption of some large scale applications from the likes of Github and LinkedIn it’s going to be a strong platform very soon.
At Kyan, we’re Ruby developers at heart, and are well practised with Ruby so this is where we like to keep things at the moment as we know how to test and monitor our Ruby applications better than most, but we’re also sure that it won’t be long before we’re rolling out the next generation of platforms and adding tools such as Node to our core.
Tags: kyan rails javascript webapplications ruby sinatra node performance application
Tweet