Managing Javascript & CSS source files

3 September 2010
Early on in the development of Hive, we realised we needed a better way to manage javascript and CSS. As the javascript files got bigger we started having source code merges (shudder) and we occasionally would ship a version with debug code still in it.

There are some really good systems out there for optimising javascript & CSS, but we thought we’d start simple and see how it went. We’ve now rolled this system out over four applications and its working great.

To avoid source code merges, small and frequent check-ins are good, so using a source control system like Accurev where each developer has their own workspace is perfect. Its also easier to maintain code and avoid merges if you can have one javascript class per file – but you end up with lots more files, which can make loading a page slower.

To solve this, we create as many javascript and CSS files as we need, then merge them on the fly. This allows a single url /scripts?version=x to download all required scripts and an XML file manages which scripts will be merged and returned. We also threw the Yahoo YUI library in there to minimize the javascript and CSS and reduce the overall size.

The other issue we were having was with debug code, swapping between debug and production libraries and remembering to remove console.log commands. To solve this our XML file specifies a target, so the web.config can control which javascript files will get included. We also use some REs to strip out debug commands on the fly.

Overall, a dead simple solution to solve 5 problems:

1. More files means easier to maintain and fewer source control merges
2. On-the-fly merging means only one javascript & one CSS per page, resulting in faster page loads
3. Minifying the code means smaller javascript and CSS
4. Stripping out console.log and other debug code means we can leave it in and not worry about it in production
5. Swapping between debug and production versions of libraries is automatic

We can highly recommend the YUI compressor for .NET – which we use in this code snippit:

public string GetJavascript()
{
    try
    {
        // Get the files from the XML index
        List<string> scriptFiles = GetJavascriptFiles();

        // Load the contents and put into a string
        var javascript = MergeFiles(scriptFiles);

        // Optimise
        if (_Target.ToLower() == "live")
        {
            // Strip out console.log, etc
            javascript = RemoveDebugCode(javascript);

            // Minimise using Yahoo YUI
            javascript = JavaScriptCompressor.Compress(javascript);
        }

        // return the result
        return javascript;
    }
    catch (Exception e)
    {
        return e.Message;
    }
}

Hive in BETA

26 August 2010

Its a relief to see Hive finally out there in BETA and to see the feedback starting to flow in.  Hive is a project we’ve been working on this year and has been a lot of fun.

Hive is a SaaS app for scheduling projects and people.  Its perfect for people with a team and a bunch of projects to manage.  Its based on the premise that most people hate project management and hate project management tools even more.  Hive lets you do in a visual, fun and collaborative way what people revert to spreadsheets for – to get visibility of what projects are on the go, who is working on them and to make sure you’re not over (or under!) capacity.

Hive is built as a one page web app and its always fun to run it in full screen mode and have people not believe its a web application.  We’ve had a lot of fun working with Darryl Gray who designed this stunning application.  Its been nice to have a project where we can make use of  the shiniest new CSS and javascript techniques with lots of drag-drop and animations, integrating SVG and wiring it all up to give a seamless user experience.  Seamless except for the bugs of course – but that’s what a BETA version is for!

As far as features go – the best of Hive is still to come.  Under the hood we’ve implemented live collaboration and its quite fun watching another user’s changes animate on your screen (this isn’t available in the current version).  We toyed with allowing viewstate changes to be sync’ed across users too – which is fun, but I’m not sure if its that useful.  Because Hive is based on an event model, we’ll be adding undo and redo support soon, but I’ll have to save details of that for another blog post.

We’ve also got some great reports, sharing, publishing and integrations planned, but the most important features to add going forward will be the ones asked for by users, so make sure you sign up at www.tryhive.com, have a play and send us your feedback.

p.s. The screenshot above features a slightly older version of the UI.  The latest version is even nicer.


Charts: buy, build or borrow?

6 August 2010

Its the same old question everytime you need some software. Do I buy it, build it myself, or find someone who’s already got it and convince them to share it?

Like any software developer, my first instinct is to build it – but then the business side of my brain kicks in and I start thinking about stuff like opportunity cost and cost/benefit.

We had this choice to make recently with charts and the answer was obviously to use one of the many great solutions out there. But, in the end it didn’t stack up and I was surprised that the right choice was just to build our own.

We were looking for something that …

- Could render to HTML, PDF, iPhone, iPad (rules out the great looking flash and javascript options)
- Could get 20 charts in a PDF with high enough resolution to show each chart full screen and without blowing out the size of the PDF (rules out using images)
- Would give us complete control over styling including being able to change style mid-series (rules out half the options)
- Is configurable via an XML template (rules out the other half of the options)

Actually, configuring any chart from XML is pretty standard when it comes to presentation, but not when it comes to the data. Charting solutions are generally not aware of the context of the report – they are only aware of the data values being plotted. We wanted a solution that could be aware of the full report, so that we could quickly change the data being charted. Like in Excel – when you have the data in one sheet and you can pivot/sum/group/etc and chart whatever you like.

So, our solution was to build a .NET SVG charting library that is report aware. SVG gives us full control over the style of each chart, series and data point.  Instead of just outputting SVG, every aspect of the chart can be styled using SVG.  It also allows us to render numerous scaleable charts to HTML and PDF without blowing out the file size.  Most importantly though, the charts are aware of the report context – so a chart no longer plots points – it plots a formula over the data. For example, to show a conversion rate, the formula “SALES / VISITS” could be used in the chart (where sales and visits are two predefined metrics). This gives us a few advantages:

1. The chart can be customised via XML or a UI to show any visualisation of the available data. A quick update and we’re now showing SALES and VISITS as separate series and comparing them to last year, or filtering by region.

2. Charts can be synchronised across a report, ensuring a consistent use of colour and allowing some smarts for visualising the data best.

3. Charts can be interactive on the web. A user can drill into the conversion rate to see how it was calculated, or highlight an area of interest, or filter the report by a particular region, or pivot the chart. We borrowed the dimension/metric/date model from Google Analytics to give data as many dimensions and metrics as we needed and introduced a hierarchal subject, allowing data to be easily rolled up and summerised.

It turns out that drawing the charts is the easy bit and managing the styles and data behind the charts is the high value bit that is sorely lacking. The false ecomony in our case would have been to buy the bit that draws the charts and then to have our options severly limited for managing the styles and data.

We’ve already deployed these charts over three projects with more to come, so the investment is paying back fast too.


Finishing

28 July 2010

We’re in the middle of the “finishing” phase of three v1 projects right now, and since my software lifecycle on this blog is sorely missing any comment on finishing, I thought I should share a few thoughts.

Firstly, what is finishing? Its what used to be called QA or testing, where the developers hand over to the test team and then retrospectively update the specification to match what’s been built.

I’m not that fond of the QA at the end of a project approach. If you look at risk based testing, then you identify risks up front and work throughout all stages of the project to mitigate the risks. When you get to the end of the project, there shouldn’t be that much QA left to do – just finishing. We all know that the last 5% of a project takes far longer than 5% of the time and this is a really important time where hundreds of small decisions are made.

The other thing I’m not fond of is this tension between QA being at the end of the cycle so there’s always time pressure, but also the idea that developers should hand over completed bug-free code. I tried that once and all it meant was that I had to go through the entire QA process myself (which I’m really bad at) – just to hand it over to be QA’ed again.

Finishing is about shipping working software. Its when the feature development is complete (or when you hit the 80/90% mark in your cycle or budget) and when developers, testers, BAs and everyone else involved start to focus on shipping instead of building. There will be bugs to fix, there will be missing functionality to build, there will be enhancements, there will be style changes, there will be a lot going on and its not all QA. The goal is to turn a feature-complete project into a quality shippable project and it doesn’t rest on the shoulders of the test team while the developers move onto the next project.

So, finishing is actually quite fun, I just wish we stopped setting up our projects to converge like this!


Zoho’s end game

29 June 2008

Some interesting comments from Zoho CEO Sridhar Vembu over on Gopal Shenoy’s blog.

On the Zoho suite, we have believed from the beginning that a lot of the value of collaborative productivity is integration across the products. Traditionally database oriented offerings (such as CRM) and document-centric offerings (the office suite) have existed in their separate silos. We believe the key to next generation productivity is integration across these silos.

Salesforce seems to agree, because they have partnered with Google for the Office suite part. Before that partnership, they tried to acquire us, which tells you what they would have liked to do, but we politely said no because we didn’t believe there was a cultural fit. They even tried to get us to drop our Zoho CRM, in return for a “deal” to integrate our office productivity apps with their CRM suite. It is our diversified business & product mix that allowed us the freedom to resist such unfair demands.

So, its about the integration more than the collaboration & other SaaS goodness.  This comes through not only in the integration within their own apps, but in the integration model they offer with third parties.  I’ve had a play with their spreadsheet integration and they allow it to be run as an embedded spreadsheet within another app.  Data can be loaded from and saved to the third party app with no Zoho account required.  This is quite powerful, but wouldn’t be nearly so interesting if their apps weren’t online and quite good too.


Build what users want

26 January 2008

Jessica Livingston has interviewed a number of successful entrepreneurs in her book ‘Founders at work‘. One of the interviews is with Paul Graham, founder of Viaweb, one of the first SaaS providers who made it big.

Paul makes some good comments about software startups, the first of which is:

“If you make something users want, they will be happy, and you can translate that happiness into money. That is the basis of a startup. A startup is a company that builds some kind of technology that people want. The mistake that a lot of founders make is to build something they think users want, but that users don’t actually want.”

There’s often a big gap between what users ask for and what they want and if you’re building what your users are asking for, then you’re still playing catch-up with your competition. Most successful software stories have pre-empted what users want and have led the way by innovating in their field.

Most of my favorite software takes me beyond what I can already do and gives me higher productivity/enjoyment/success. I rely on experts to lead me in their field, so I can focus on becoming/being an expert in my field. I don’t want software to give me what I ask for, because then I either have to pretend to be an expert at everything, or I just won’t grow my capabilities because I’ll ask for what I’m familiar with.

Innovating in your field of expertise is leading users to higher productivity, enjoyment and success and in turn, letting them focus on what they do best.


Now’s a good time …

22 January 2008

… to upgrade your small business accounting system to Xero.

Xero have just released their 19th update within the last 12 months and its only getting better and better.

To make it even easier, Xero will also get you up and running from your existing accounting system for $NZ199, so now’s a perfect time to get setup in time for the new financial year.

Shameless Xero advertising follows …

matthewxero.jpg
How can you resist? Sign up here.

SaaS for commoditization

6 September 2007

Ben has two classifications of SaaS – SaaS/s and SaaS/v, differentiating offerings by those that (s)ubstitute existing software and those that add (v)alue. The SaaS/v classification makes sense, but SaaS/s seems to be based on the comparison with existing software, not on the characteristics/model of the SaaS offering itself and how it compares with other SaaS offerings. Maybe better classifications would be SaaS/c and SaaS/v?

SaaS for commoditization:

  • Drivers are quality, cost and service
  • Audience is mass market (long tail)
  • Indirect sales needed to keep price competitive
  • Not necessarily a commodity yet, but being commoditized through innovation in improved delivery and optimizing established features

SaaS for value-add:

  • Drivers are added value, integration, collaboration
  • Market is niche
  • Direct sales needed to educate market, so must be priced accordingly
  • Innovation is in new markets and new products

Typically low cost SaaS will commoditize a product so that high value add-ons can be sold on the low cost platform, but SaaS/v shouldn’t be low cost because its not a sustainable business model. Unless of course the business model is buy a customer base and then be acquired.


The core value of software is changing

28 July 2007

The Herald reported today on Steve Ballmer’s comments about the future of software.

“Every piece of software – the basic core value in the way software gets created – will change in the next three, five or 10 years,” said Ballmer, adding that future software will all factor in some aspect of desktop, web and server elements.

However, he rejected the idea the software industry would shift entirely to an internet delivery model.”

There’s a lot of talk about SaaS being the future of software, but I think a lot people still see SaaS as just delivering software as a web application. I believe SaaS is the next major business model for software, but if you look at existing SaaS providers, the web app is just part of the service.

SaaS systems today include:

  • The web application
  • The service delivery platform (part web based, part server based and part client based)
  • Server modules to provide integration with third parties
  • Client modules to provide integration with client based software, such as Outlook
  • Client side reporting using Acrobat and Excel
  • Rich client side UI using Flash and Silverlight
  • Brower plugins to provide offline or regular services (such as Wesabe’s firefox uploader)
  • Core client side software such as provided by skype, google, virus scanners
  • Vista gadgets

To limit SaaS to a web based delivery model is to overlook the richness of how a web application fits into the bigger picture of software as an integrated web/client/server service.


What SaaS isn’t

3 May 2007

Following on from my thoughts about webware & software, I’ve been hearing more and more chatter about the pros and cons of web apps. Having worked on a webapp for four months at Xero with some top people (including Craig, Kirk, Fletch, Jeff, Adam, Grant, Phil, Ivan & Vicki) in their fields – I’m convinced that webapps can provide more powerful UIs than software can.

Have a look at some of the Xero screens – which are possible in software – but you’d never see stuff like this for the same reasons as I mentioned in my last post … software typically follows well defined UI patterns and constraints and therefore limits the creativity of the UI/design gurus who can make webapp UIs look and work this good:

Xero dashboard

What still annoys me though, is SaaS marketing like this:

“Avoid the costly IT expenditures, prolonged time-to-market and outdated technology associated with traditional software”

“Nowadays, companies need flexible technology solutions with low upfront costs, reduced risks, and higher probabilities of project success. In response to this need, we developed our solution to be “Software as a Service” that is hosted in a centralized data center and delivered on a subscription basis to clients using a web-based interface”

This is all true of “traditional software”, but its not the reason for building a webapp. SaaS can be delivered as a smart client or webapp or both (or xaml or …). There are some clear advantages of doing each, but all that really matters is that the data is stored in a data center and accessed via standard web protocols. Whether you run a javascript/html client or a thicker client such as a .NET windows forms – both can provide “low upfront costs, reduced risks, higher probabilities of success” etc, but both can also provide none of these. I’ve seen plenty of webapp services delivered with the wrong business model, poor UI and poor delivery/support and they can have longer “time-to-market” and higher “hidden costs” than delivering software.

Although this disagrees with wikipedia, I think SaaS isn’t the software or technology – its the business model, which has to be backed up with a strong capability to deliver & serve users through great UI, great functionality and great business & software support.


Webware vs Software

23 April 2007

I recently found a couple of blogs that are pro and anti webware and it made me think about just how much webware has changed recently. Eighteen months ago the reasons I used for deciding whether to build webware (ie SaaS) or software (ie standalone app or a smart client) were basically: if the audience is the general public, then build webware. Otherwise, build software.

This was because software UI could be far more powerful than web UI, so you could build a lot more functionality with a lot higher usability for the same price. Of course this was also based on my preference for software, but I had the golden rules of UI design to back me up.

Now, with decent (or at least better) web browsers, AJAX, javascript and all those 2.0′s, webware can compete with the basic stuff that software has always been able to do. Its like Windows finally doing what Apple has been doing forever. Feedback and error handling can be handled live, controls can be interacted with and updated on-the-fly and users don’t have to wait for a server roundtrip after every click. On top of this, webware gives complete freedom to the user interaction gurus, so they can design an app with consistency, closure, keeping users in control and reducing memory load.

So, with webware and software pretty-much equal when it comes to UI and the cost of developing and deploying webware and software being very similar, its often not a technical decision anymore between building webware or software – its a business decision. Is the business model to license or subscribe? How will the software be sold and maintained? etc.

However, while webware technology has come a long way, there’s still a lot of work to do to achieve good UI and I’ve been very impressed at how the team at Xero have managed to build webware with better usability than any accounting software out there.

The golden rules …

1. Consistency
Software UI is well defined. It all looks the same. Menus, toolbars, shortcut bars, modal windows etc, its all boring and consistent and you don’t have to put any thought or time into how to make the windows and controls look – just copy Outlook. Of course, you can still screw up the consistency within your app, but software has a huge head start over webware when it comes to consistency.

2. Shortcuts
Frequent users need shortcuts, including right click menus, single vs double click, keyboard shortcuts and menu shortcuts. These are all taken for granted in software and users get pissed off when they’re missing, but they’re usually missing from webware and no-body notices because no-body expects them there because it takes time to add them, so most developers don’t.

3. Informative feedback
Good feedback should be fast and contextual, so having to make a round trip to the server, then reload a page just to validate a field, or confirm an action is poor. Handling feedback in software is a non-issue – you still need to provide good informative feedback, but its much easier to do it in software than in webware.

4. Closure
The user needs to know when they’ve completed an action. When using software, have you ever seen the message “Don’t click Save twice otherwise your order will be processed twice”? This is typical of webware because you click the button and wait and then you’re not sure whether you’ve just saved or lost your last 10 mins of form-filling, so you click it again. What happens when you go back, then forward, or refresh. Software doesn’t have these issues, so you don’t have to spend time building workarounds.

5. Error handling
Like providing good feedback, error handling should be instant, contextual and preventative where possible. Who wants to post a form, then receive a list of error messages that need to be matched back to each control? Error handling in software is easy, in webware its harder, so its usually not done well.

6. Undo
This is an area that most webware lacks in, but something that users expect.

7. Keep users in control
Happens by default with software and requires more thought, design and workarounds in webware.

8. Reduce short-term memory load
Not too long ago, updating info live on a webpage was not cross-browser, so webware was pretty poor at this, but to update information on a form in software is simple and you’ve got lots and lots of memory for storing the session info for just one user.


3.2.1. Xero

6 March 2007

Sir David Tweedie (Chairman of International Accounting Standards) once said, “Accounting is a primitive subject still. There are lots of things we haven’t got right. Much decision making and many businesses sit uncomfortably in the framework of the traditional accounting model, which is a creation of the manufactoring era … balance sheets are like haggis, if you knew what went into it you wouldn’t touch it”.

So what’s the solution? Should all business owners go out and take a degree in accountancy so you can run your business on valid and relevant info? No, its simple really – we just need smarter business tools and this is where Xero comes in…

http://www.drury.net.nz/2007/03/06/xero/

“Xero is an online accounting solution for small businesses, delivered on a Software as a Service (SaaS) basis. I believe it is one of the biggest market opportunities out there and a global opportunity we can credibly go after from New Zealand.

We’ve been heads down on this for some time. We have a number of beta customers on board and will soon move into a limited release. A challenge for SaaS offerings is building out operational capability. As the software is getting to where we need need it we are now beginning to now ramp up that investment so we can deliver a customer experience in line with the quality of the software. For that reason we’re bringing in customers in small batches, learning where they need guidance, rinse and repeat.

Xero is all about people. Making a difference to the people that run small businesses and providing an opportunity for our best talent to build a world class company. The next step in my career is to build a long term company and culture that attracts and fosters talent, to earn export revenue.

Xero is design led. We recruited early some of the top interaction designers we could find Philip Fierlinger and Grant Robinson.

We’ve put an awesome technical team in place with Craig Walker, Kirk Jackson, Andrew Butel, Fletcher Brown, Adam Burmister and Jeff Wegesin.

We’ve invested heavily in product management and customer care with an exceptional group of customer advocates: Michelle Perera, Andy Leeb, Catherine Walker (Orange Girl), Catherine Robinson, Donna Wylie, Larissa Paris and Louise Roebuck.

We managed to attract Kate McLaughlin from the National Business Review to run Xero marketing and communications and the very talented Darryl Gray is in house brand guy.

Small business accounting identity Hamish Edwards brings the domain expertise to ensure what we deliver is not only technically world class but really does improve the performance of a sector that makes up 95%+ of all businesses.

It’s early days but we think we’re doing something special. “

Check it out: www.xero.com


Follow

Get every new post delivered to your Inbox.