The two sides of Web scale

Web scale is certainly a concept that has taken hold in the library realm, not least due to OCLC’s efforts to build the Web scale library management system. In a recent talk (in German), I took a fairly broad swipe at OCLC’s project. I think their project is going in some interesting new directions, but it is hard these days not to see OCLC turning into something of a libraryland Google.

My concerns with their project boil down to questions of how one defines scale, and where one places one’s emphasis with scale. On the one hand, OCLC is getting scale fairly right with this project. Take local library data that on its own is rather uninteresting (it tells no stories, to paraphrase a colleague of mine) and cannot be used to drive  innovative workflow processes or user services, put it together in a large aggregated database, and leverage that scale to rewrite how the library management system works. No library nor single library consortium is in a position to do this, not least since we lack data standards for much of the data we generate, which makes it difficult and expensive to aggregate such data.

On the other hand, however, OCLC is ignoring what I posit in my talk as the more important (and vexing) aspect of scale, namely organizational scale. While their approach is revolutionary, their organizational approach is well known: a vendor developing a proprietary product behind closed doors that will be sold to end users who have no access to the underlying code. Even if OCLC had the most brilliant programmers, usability experts, interface designers, etc., it would be impossible for them–based on our experiences not only with library software, but most software ever developed–to build a product that would be significantly better than software provided by other vendors. Sure, it may be a leap forward, but that leap forward will, over time, shrink and OCLC will lack the flexibility and ability to respond to that, which is what has happened to most library software vendors.

At this point one should anticipate that what I argued is that open source is the only way to go if one wants the “sea change” Andrew Pace seeks with OCLC’s project. There are myriad examples of software applications, not least when it comes to Web scale technologies, where open source products dominate their sector: Apache, MySQL, WordPress, Mediawiki, et al. What is significant about this is that these products dominate not because (or at least not only) they are free, but because they are the best product in their respective markets.

What does one get with open source? More developers, for one. Extensibility. Plug ins, themes, extensions galore. A community that buzzes with activity and contributes actively, regularly, and openly to the product in many ways. Not least, one gets insurance against the whims of a single vendor, and against the possibility that a vendor changes direction due to financial crises, acquisitions, buyouts, etc.

OCLC is certainly acting in good faith, just as Google with its “do no evil” philosophy has acted in good faith. Just as there are many people dissatisfied with Google for a variety of mostly valid reasons, it would be naive to assume that OCLC will not run into many of the same problems.

As a librarian, I do not want to be caught making the mistake again of hitching my wagon to the vision of a single vendor’s dream. Do we not have enough cautionary tales to avoid this fate? Moreover, we collectively have more software development capacity than any single vendor could ever afford. Where things break down is in the management of our libraries, where safety is valued over innovation, and where we view collaboration with suspicion in many instances. If we had more visionary library directors such as Mark Leggott at UPEI and others who have dedicated the resources of their organization to supporting and developing community-based open source solutions for libraries, we would not be in the rut in which we have been stuck for the entire Web age in libraries, waiting for some cavalier to pull us out of the muck.