Managing Javascript & CSS source files

3 September 2010
Early on in the development of Hive, we realised we needed a better way to manage javascript and CSS. As the javascript files got bigger we started having source code merges (shudder) and we occasionally would ship a version with debug code still in it.

There are some really good systems out there for optimising javascript & CSS, but we thought we’d start simple and see how it went. We’ve now rolled this system out over four applications and its working great.

To avoid source code merges, small and frequent check-ins are good, so using a source control system like Accurev where each developer has their own workspace is perfect. Its also easier to maintain code and avoid merges if you can have one javascript class per file – but you end up with lots more files, which can make loading a page slower.

To solve this, we create as many javascript and CSS files as we need, then merge them on the fly. This allows a single url /scripts?version=x to download all required scripts and an XML file manages which scripts will be merged and returned. We also threw the Yahoo YUI library in there to minimize the javascript and CSS and reduce the overall size.

The other issue we were having was with debug code, swapping between debug and production libraries and remembering to remove console.log commands. To solve this our XML file specifies a target, so the web.config can control which javascript files will get included. We also use some REs to strip out debug commands on the fly.

Overall, a dead simple solution to solve 5 problems:

1. More files means easier to maintain and fewer source control merges
2. On-the-fly merging means only one javascript & one CSS per page, resulting in faster page loads
3. Minifying the code means smaller javascript and CSS
4. Stripping out console.log and other debug code means we can leave it in and not worry about it in production
5. Swapping between debug and production versions of libraries is automatic

We can highly recommend the YUI compressor for .NET – which we use in this code snippit:

public string GetJavascript()
{
    try
    {
        // Get the files from the XML index
        List<string> scriptFiles = GetJavascriptFiles();

        // Load the contents and put into a string
        var javascript = MergeFiles(scriptFiles);

        // Optimise
        if (_Target.ToLower() == "live")
        {
            // Strip out console.log, etc
            javascript = RemoveDebugCode(javascript);

            // Minimise using Yahoo YUI
            javascript = JavaScriptCompressor.Compress(javascript);
        }

        // return the result
        return javascript;
    }
    catch (Exception e)
    {
        return e.Message;
    }
}

Hive in BETA

26 August 2010

Its a relief to see Hive finally out there in BETA and to see the feedback starting to flow in.  Hive is a project we’ve been working on this year and has been a lot of fun.

Hive is a SaaS app for scheduling projects and people.  Its perfect for people with a team and a bunch of projects to manage.  Its based on the premise that most people hate project management and hate project management tools even more.  Hive lets you do in a visual, fun and collaborative way what people revert to spreadsheets for – to get visibility of what projects are on the go, who is working on them and to make sure you’re not over (or under!) capacity.

Hive is built as a one page web app and its always fun to run it in full screen mode and have people not believe its a web application.  We’ve had a lot of fun working with Darryl Gray who designed this stunning application.  Its been nice to have a project where we can make use of  the shiniest new CSS and javascript techniques with lots of drag-drop and animations, integrating SVG and wiring it all up to give a seamless user experience.  Seamless except for the bugs of course – but that’s what a BETA version is for!

As far as features go – the best of Hive is still to come.  Under the hood we’ve implemented live collaboration and its quite fun watching another user’s changes animate on your screen (this isn’t available in the current version).  We toyed with allowing viewstate changes to be sync’ed across users too – which is fun, but I’m not sure if its that useful.  Because Hive is based on an event model, we’ll be adding undo and redo support soon, but I’ll have to save details of that for another blog post.

We’ve also got some great reports, sharing, publishing and integrations planned, but the most important features to add going forward will be the ones asked for by users, so make sure you sign up at www.tryhive.com, have a play and send us your feedback.

p.s. The screenshot above features a slightly older version of the UI.  The latest version is even nicer.


Charts: buy, build or borrow?

6 August 2010

Its the same old question everytime you need some software. Do I buy it, build it myself, or find someone who’s already got it and convince them to share it?

Like any software developer, my first instinct is to build it – but then the business side of my brain kicks in and I start thinking about stuff like opportunity cost and cost/benefit.

We had this choice to make recently with charts and the answer was obviously to use one of the many great solutions out there. But, in the end it didn’t stack up and I was surprised that the right choice was just to build our own.

We were looking for something that …

- Could render to HTML, PDF, iPhone, iPad (rules out the great looking flash and javascript options)
- Could get 20 charts in a PDF with high enough resolution to show each chart full screen and without blowing out the size of the PDF (rules out using images)
- Would give us complete control over styling including being able to change style mid-series (rules out half the options)
- Is configurable via an XML template (rules out the other half of the options)

Actually, configuring any chart from XML is pretty standard when it comes to presentation, but not when it comes to the data. Charting solutions are generally not aware of the context of the report – they are only aware of the data values being plotted. We wanted a solution that could be aware of the full report, so that we could quickly change the data being charted. Like in Excel – when you have the data in one sheet and you can pivot/sum/group/etc and chart whatever you like.

So, our solution was to build a .NET SVG charting library that is report aware. SVG gives us full control over the style of each chart, series and data point.  Instead of just outputting SVG, every aspect of the chart can be styled using SVG.  It also allows us to render numerous scaleable charts to HTML and PDF without blowing out the file size.  Most importantly though, the charts are aware of the report context – so a chart no longer plots points – it plots a formula over the data. For example, to show a conversion rate, the formula “SALES / VISITS” could be used in the chart (where sales and visits are two predefined metrics). This gives us a few advantages:

1. The chart can be customised via XML or a UI to show any visualisation of the available data. A quick update and we’re now showing SALES and VISITS as separate series and comparing them to last year, or filtering by region.

2. Charts can be synchronised across a report, ensuring a consistent use of colour and allowing some smarts for visualising the data best.

3. Charts can be interactive on the web. A user can drill into the conversion rate to see how it was calculated, or highlight an area of interest, or filter the report by a particular region, or pivot the chart. We borrowed the dimension/metric/date model from Google Analytics to give data as many dimensions and metrics as we needed and introduced a hierarchal subject, allowing data to be easily rolled up and summerised.

It turns out that drawing the charts is the easy bit and managing the styles and data behind the charts is the high value bit that is sorely lacking. The false ecomony in our case would have been to buy the bit that draws the charts and then to have our options severly limited for managing the styles and data.

We’ve already deployed these charts over three projects with more to come, so the investment is paying back fast too.


Are dynamic languages the next best thing?

31 October 2007

We had a preview of Ivan’s talk this morning comparing the up-and-coming IronRuby dynamic language in .NET with C# 3.5.

My first impressions of Ruby on Rails and IronRuby are that they give you fast results now, but at the cost of maintainability later (since everything’s a tradeoff). But, maybe maintainability doesn’t matter so much, because its faster to just re-write it.

In C# & Java, I like the fact that an interface is a contract and the implementation is abstracted away from caller. Dynamic languages lose this explicit interface-driven-design in favour of a test-driven-design.

At the end of the day, its what gets better results for less effort that matters so it will be interesting to watch if dynamic languages can improve project success rates and deliver cheaper and better quality software.

Are dynamic languages the next best thing? Ivan definately thinks so and will be writing more about IronRuby on his blog.


Thanks MBUnit

18 July 2007

Unit testing can be very satisfying, especially when tests fail.  If your tests always pass, then you start to wonder if the tests are working.  I was wondering this yesterday, so I added some extra assertions, thinking that one of these two tests must fail:

Assert.AreNotEqual(firstInvoiceNumber, secondInvoiceNumber); 
Assert.AreEqual(firstInvoiceNumber, secondInvoiceNumber);

Wrong.  They both passed.  After some more testing,

string firstInvoiceNumber = string.Format("{0}", "TEST"); 
string secondInvoiceNumber = string.Format("{0}", "TEST");
string firstInvoiceNumber1 = "TEST"; 
string secondInvoiceNumber1 = "TEST";
Assert.IsTrue(firstInvoiceNumber.Equals(secondInvoiceNumber)); 
// PASSES
Assert.AreEqual(firstInvoiceNumber, firstInvoiceNumber1); 
// PASSES 
Assert.AreEqual(secondInvoiceNumber, secondInvoiceNumber1); 
// PASSES
Assert.AreEqual(firstInvoiceNumber, secondInvoiceNumber); 
// PASSES 
Assert.AreNotEqual(firstInvoiceNumber, secondInvoiceNumber); 
// PASSES
Assert.AreEqual(firstInvoiceNumber1, secondInvoiceNumber1); 
// PASSES 
Assert.AreNotEqual(firstInvoiceNumber1, secondInvoiceNumber1); 
// FAILS

So, don’t trust your unit tests any more than you trust your code.  “It must work because it compiles” doesn’t apply to unit tests either.


Follow

Get every new post delivered to your Inbox.