Archive for the ‘FulfILLment’ Category

A new role for the Equinox community librarian

Tuesday, September 29th, 2009

It’s with mixed emotions that I write this (from the 25th floor of the lovely Amway Grand Plaza in Grand Rapids, where a team of Evergreeners has been planning the 2010 conference), but very shortly I am changing roles in the Evergreen community.  I have an opportunity to return to library administration and to Northern California, two opportunities I (and my family) had hoped would be in our future at some point.

So to Brad and team I bid a fond adieu. In a short time I was able to be part of Evergreen’s first-ever user conference, help nurture into being a fabulous community-driven documentation project, and help build bridges of communication among all the Evergreen stakeholders.  I picked up some skills I knew I needed (nothing like a little XML to tide a gal over), and some skills I didn’t realize I needed (once I learned how to ask sponsors for conference funding, it became almost intoxicating!). I was able to give talks about open source from Indiana to Australia. Not a bad gig at all!

But I am by no means bidding farewell to the Evergreen community. You will continue to hear my voice as a strong advocate for open source software in libraries, and for Evergreen in particular — not just the software, but the wonderful magic bus that is carrying it forward, a bus overflowing with passionate developers, librarians, users, and other champions of engagement.

Youre either on the bus, or...

You're either on the bus, or...

History will prove us right

Those of us who “get” open source have sometimes been denounced for our support of what open source means in LibraryLand. We have sometimes  been condescended to as if we were cultic simpletons — as if passion and knowledge could not go hand-in-hand.

But the reality is that open source is here to stay. It’s a necessary alternative to the way we’ve done things, and not simply because it has forced new thinking from traditional software sources (who in many cases, despite their own public comments, use open source themselves within their applications).

Get in, Put the Keys in the Ignition, and Drive

I have spoken many times in the last several years about the bad state of affairs we librarians fell into in the late twentieth century, when we moved from designing and building the tools we use to a passive relationship with other folks’ software.

This sad situation has not been healthy for anyone involved — not librarians, vendors, or users.  It threatened the very existence of librarianship as a legitimized profession. If we are not engaged with tool creation, then who are we? (For more on this, read The System of Professions.) This passive relationship also turned vendors into vending machines, forcing them to produce software that reflected individual punchlists, not community vision.

Open source restores the tool engagement we need at a crucial point in our future, when it is clear that we are in the middle of a massive shift, and that everything related to information access, transfer, and ownership is on the table. We must be stewards of our future. We must drive that bus!

Last thoughts

I have had a great time at Equinox Software as Community Librarian. It’s been a wonderful journey, and I respect Equinox for having the insight to create the position in the first place.  I look forward to watching Evergreen continue to evolve, and to watch the community stride forward into the future.  I won’t say “I’ll miss you all,” because I’ll be right here — just in a different role.

Posted in FulfILLment, People | 4 Comments »

Generations

Monday, April 21st, 2008

ILSs have evolved over time. What started as as a way to manage the printing of catalog cards (1950s and 1960s) eventually morphed into a system for tracking inventory, and then finally into a full-lifecycle management system for general library operations. None of this is news, of course.

What is news is that Evergreen is the first and only ILS to go into production that does not suffer from decisions made 20, 30 or even 40 years ago. Too often I see quotes from respected luminaries in the library world suggesting that Evergreen is simply duplicating what has been done for the last 30 years of ILS design. That means two things, actually:

  1. We’ve done a very good job of creating an interface to the Evergreen ILS that feels like what librarians are used to using. This means a shorter training cycle for front-line staff, greater ROI and lower TCO.
  2. We may have done too good of a job creating an interface that makes librarians comfortable with traditional ILSs feel comfortable with Evergreen, and haven’t explained well enough just how the core Evergreen is designed and shown just how easy it is to expand services in ways no one has yet considered in terms of an ILS

Looking Back

What we now call the ILS, a product purchased from and maintained by vendors, first emerged in the late 1970s and early 1980s. This is, for all intents and purposes, where history begins.

Over the last thirty years we’ve seen, more or less, three architectural generations of ILS arise from the original proto-ILS. Each generation learns from the lessons of those that came before, as well as from the wider world of software engineering. At each stage there are specific changes in architecture which provide obvious differentiating features that are wholly new so far as the ILS is concerned, but there are also concepts and constructs carried over from previous generations. In addition, there are non-architectural design components that are typical of each stage of evolution, and while the basic architecture of systems at each evolutionary stage may not be directly relevant, there is an identifiable correlation between the overall architecture and these higher-level design decisions.

The Client-Server Age

In the beginning (the late 1970s to mid 1980s) there was the monolithic client-server system, and it was good. Mostly it was good because there was nothing else, and something was needed. These systems were designed to run on the computers of the day; that is, large (for their time) central computers with dumb terminals, or later, thin clients on PCs, for data entry and output. They were (and are, it’s amazing how some of these systems persist) consolidated 2-tier systems that contained all of the logic in a single program on the server.

The back-end storage systems, and in many cases the display engine, were integrated directly into the main program, and required that this one program run on one server. The only way to scale such a system is to purchase a larger server. It is far from impossible to outgrow the largest servers on the market, even today, unless an institution is willing to dedicate a very large portion of their budget to such a system.

Another design constraint of theses systems, though not related to their client-server architecture but still endemic to the age, is that they were never meant to run the operations of more than one organizational entity. There is little or no consideration for location independence. If a staff member has a permission, they have that permission everywhere within the system. In other words, there is no concept of location-specific privilege separation.

Case in point: PINES is the state-wide consortium in Georgia at which Evergreen was initially developed. Although functionality was lacking in their existing solution, an equally concerning problem was the performance of the server during normal business hours. The 48 processor E10K they were running on was straining under the load of a state-wide consortium, and regular maintenance jobs that would not be a problem for smaller scale installations were requiring a mid-day restart of an essential, core service, and a nightly restart of all services. To upgrade out of the red zone would have meant investment in a 2 million dollar E20k, or larger. The PINES budget for one year is around 1.5 million dollars. Putting aside the cost and scaling factors in running such a system, there was no way for PINES to separate privileges based on which library or library system a staff member worked in. Among other things, this meant that PINES could not implement centralized Acquisitions.

The Web Application Age

Then in the late 1990s, scurrying around the feet of the lumbering, antiquated 1- and 2-tier client-server dinosaurs, there evolved the 3-tier web application. These systems are characterized by a nod towards database independence, the use of non-library standards for core functionality, and interface rendering handled by a generic third-party application such as a web browser. Such systems can be made to scale horizontally — that is, the full spectrum of functional areas can be scaled up — by adding more web servers in a dumb cluster, but because all of the web servers must run all the business logic for the ILS, as well as generate web pages for display, this scaling is not linear — one sees less usable per-server capacity added with each new server in the cluster. A related scaling problem is that there is no opportunity for balancing base-line resource utilization against the specific needs of the installation — in a low-circulation, high-search environment the server still has to load and run all of the heavy circulation logic with each instance of the search logic, sapping memory and CPU resources that could be used elsewhere.

These systems also carried over the lack of location-aware privilege separation from the earlier generations. While adding new services to such systems is somewhat simpler than in a client-server architecture, they are still basically a monolithic code-base built to run the operations of a single organization. They are also targeted at smaller library systems lacking the need for some of the more sophisticated business processes of larger institutions. As such, scaling is not a primary concern — and rightly so, with their small-system focus — and is not addressed directly but comes as a side-effect of the architecture, to the level that such scaling exists.

The SOA/Distributed/Dis-integrated Age

In the mid-2000s we see the rise of REST and SOA. New software systems in most industries are being modeled on distributed and modular architectures that allow the dis-integration of services and interfaces. ILSs designed and developed in this time period have the specific characteristic of incorporating concepts of SOA and distributed computing from the ground up. Each functional unit of the ILS is implemented as a service with a well-defined API, and these services can be replaced or augmented over time without the fear of disrupting the rest of the code-base. New services snap in like Legos and are trivial to implement in comparison to new services in previous generations, and service APIs stay consistent across time and even development and implementation methodology. Because ILSs built this way have intentional separation between all service layers (UI, business logic, storage) they can present many different workflow-optimized interfaces to the user, or to other services, while leveraging existing code.

Because of the focus on service dis-integration, resources can be applied precisely where they are most needed, and horizontal scaling is linear in the worst case. Scaling in this architecture is managed through the addition of simple, replaceable commodity hardware, making it extremely cost effective for initial capital outlay and providing the possibility of “just in time” cluster expansion using hardware that delivers the most bang for the buck at the time of need.

Greener pastures

Evergreen is the first production ILS, and only we are aware of as of this writing, to be designed with Service Oriented Architecture (SOA), Representational State Transfer (REST) and “n-tier” architectural concepts specifically in mind. Evergreen achieves this by building services and applications on the OpenSRF (Open Service Request Framework, and pronounced “open surf”) platform. OpenSRF handles all of the details of implementing a stateful, decentralized service architecture and the Evergreen code supplies the business logic and UI framework that matters to libraries.

To provide a specific example of the ease of extension that Evergreen and OpenSRF provide, the online bill pay functionality coming in the next major release was designed and developed by someone that had only passing familiarity with the underlying of Evergreen, or with the OpenSRF framework on which it is built. It was created in about 4 hours, and adds integrated, ILS-wide support for credit card and PayPal payment processing.

Another mistake I hear repeated all too often is the fallacy that Evergreen requires uniform policy definition across participating institutions. This could not be further from the truth — Evergreen supports the most flexible and location aware circulation and hold policies of any ILS available, and can even be extended with ad hoc exceptions to any policy definition via simple rules. This is, of course, unrelated to the architecture of Evergreen, but as it is the only example of a system based on such an architecture, this feature is also endemic to this pattern.

But wait, there’s more!

Another natural consequence of the SOA/dis-integrated architecture of Evergreen, facilitated by OpenSRF, is that most if not all of its code can be re-purposed largely without modification. We at Equinox Software are currently beginning development on FulfILLment, a large-scale, cross-ILS borrowing platform to facilitate the functionality of large ILL consortia, such as state or region-wide networks. This system will leverage much of the organization-aware business logic that Evergreen already provides to support cross-institution circulation in a completely ILS agnostic fashion.

The Future

While neither I nor anyone else can predict what the Next Big Thing will be in terms of application architecture, we can draw some clues from the current state of the art. Bleeding edge application development in 2008 seems to be moving towards refining the SOA model. Efforts such as SOA 2.0 and Service Component Architecture (SCA) point toward a near-term future where SOA and dis-integrated services become mainstream, and service-on-demand and mashups are the norm. If these trends are any predictor, Evergreen and OpenSRF are in for a long ride, especially as more features and functionality are incorporated over time.

Note: Updated 2008-04-22T10:00:00-04 to add some term clarification.

Posted in Evergreen, FulfILLment, Thinking about Evergreen | No Comments »

Deep in the heart of Texas

Monday, April 14th, 2008

In preparation for the Texas Library Association conference this week, I have been looking at data from public libraries in Texas using the same method I used in a post on the open-ils blog on library characteristics: A Riff on Big. Not surprisingly, Texas public libraries show similar patterns as we saw there for all U.S. public libraries: a few very big libraries and many small ones.

Taking total circulations as in that earlier post, we find the following summary figures for Texas from the same fiscal year 2005 data used there:
The maximum number of annual circulations at any Texas public library was 10.4 million at Harris County. The fourth quartile, though begins at 87,800. That is, one fourth of the libraries in this sample had total circulations greater than that number including, of course, Harris County. One fourth had between 31,732 and 87,800. The central value is that 31,732 which means that half of the libraries had circs greater than that number and half had less. The first quartile—the lowest fourth—had fewer annual circulations than 12,050. As I said, the central value is 31,732 but the mean, that is, the arithmetic average, is 180,574. The big libraries are so big that they pull this average way above the median.

This kind of relationship with circulations exists with most other variables. For example, total expenditures: median (or central value): $ 116,000, while the arithmetic mean is $655,856. And so on through most of the variables that relate to size. The big public libraries in Texas are very big and the small ones are very small.

There are implications to the fact that the distribution of libraries by their sizes and resources have these characteristics. I discussed the profound and worrisome policy implications in more detail in that earlier post but, briefly, differential resources like these numbers indicate have a lasting effect on our notions of a foundational premise in our form of government, that is, an informed citizenry. In addition, in an era where so much is changing in our economy and so fast, continuing education is vital. Smaller libraries have fewer resources to aid a population that is adapting to continuing change and is trying to stay informed. What I neglected to mention earlier, however, is the fact that libraries provide their users entertainment as well as edification—and sometimes both at the same time.

In the Transforming Texas Libraries initiative, there is a clear background of realizing that the changing information environment faced by libraries and the public they serve are changing. Google and this new information environment are affecting libraries.

I have an idea:

Why not put all Texas public libraries in one resource sharing network like the folks in Georgia did? The technology exists now with Evergreen and a related development effort, dubbed FullfILLment ™, that will use the proven Evergreen back end processing to connect to library catalogs that are not using Evergreen through opportunistic connectors and, voila!, you have a PINES-like consortium. FullfILLment is discussed in other blog posts. We are working with some other large consortia to get development funding.

What would happen? Well, all sorts of things as the network filled out.

Right now, users of libraries face an information environment with their libraries that resembles “silos” as they are called in the IT world: separate and barely communicating collections of information. In a Google world, library users and potential library users find library resources broken up in small lots while they are now expecting to see the whole panorama of what is available. Interlibrary loan is a sluggish, time consuming, and awkward way of sharing materials between libraries; its cost in time and complication reduces its use and its effectiveness.

Within PINES, however, ILL become “holds” that result when someone searching the local catalog can’t find what he or she is looking for locally and the ILS offers a simple means to request it from another library. The material is already verified and the user authenticated. Without going into too much detail, I discussed elsewhere what I dubbed “The Evergreen Effect” which is what I called the increase in holds coming from the result that Evergreen was easier to use than the legacy software. Holds were up in the 30-50% range and total circulations were up 10% or so. This increase was after the consortium already existed so we have no easy way to estimate how much of a circulation increase came about as a result of a consortium alone. What would all this mean to Texas with its 100 million circs (again, 2005 figures)? Would a 25% increase be outlandish a projection if Texas were to emulate the PINES experience?

Q. Can Evergreen really handle the circulations for all the public libraries in the state?
A. Yep. That is what it was designed to do.

Q. Is Evergreen perfect?
A. Nope. It will do a lot but acquisitions and serials won’t be working until late this year or early next year. But, it will take longer than that to get the libraries loaded anyway.

Q. But won’t this expected increase cause problems?
A. In the short run, very probably. People like libraries and if you make it easier to use them, count on it: they will. Folks will be coming in your library more to avail themselves of a virtual collection that is substantially larger than what they have access to currently. In Georgia, we have seen that library users do not care about our politics but they do love access to the long tail—or at least the “slightly longer tail” than they find at their libraries. The public that uses libraries also uses Google and information silos pain them. These users have been trained to expect better in this era.

More broadly, there are good problems in libraries and bad problems. A bad problem is what happened to many libraries in Louisiana after the hurricanes: they’re gone. A good problem is when folks want to use your library so much that they bust down the doors. Now they tell the mayor what a great job the library is doing so the library director doesn’t have to. The budget meeting without that problem. In Georgia, the legislature has just passed a bill increasing the PINES budget by a third. It is on the governor’s desk and we will see if he signs it. But, the support PINES received in the legislature, I believe, is a result of the support it receives among its many users for having done a good job. A good job means more support.

People like what libraries do and if given the opportunity to have libraries that make more materials more available, the users of libraries will visit them more often. That will likely mean more work in the short run but in the long run stronger libraries and a better informed citizenry that uses libraries. We have the ability to run very large, very dispersed networks now and our users like them. What are we waiting for?

Bob Molyneux

Posted in Announcements, FulfILLment | No Comments »

Out where the buses don’t run

Tuesday, March 18th, 2008

What follows began as a summary of my VALE symposium talk but I have been thinking about the talk and related items and, well, here goes. For those interested, I have been told the slides and videos will appear here shortly:
http://www.valenj.org/newvale/ols/symposium2008/.

My talk touched on several themes but I want to elaborate on that part of the talk that discussed consortial software and particularly FulfILLment(tm) because there has been interest in it since VALE.

Evergreen’s OpenSRF is the service-oriented architecture that provides the structure that allows Evergreen to run a system on a laptop. It is the same architecture that allows Evergreen to scale across hundreds of servers to handle the large transaction loads of a distributed resource sharing network like PINES.

Consortia are a tool used by libraries to do many things as Mike’s earlier post outlines. There are consortia that are used for purchasing, those that are loose combinations for other purposes. Only a few historically have attempted to do what PINES has done: to combine their catalogs into one and run a large network with a statewide borrowers’ card. Union catalogs have been much more talked about than actually observed because they are difficult to do in the card catalog era and not much easier in the digital era until recently. And, of course, before Evergreen handling the transactions load on a large, distributed, and busy network was beyond available software. True, there are three or four (depending on final figures) US and one Canadian library that circulate more than PINES based on 2005 figures but from many fewer locations, fewer bibliographic items, fewer database edits from fewer catalogers, and not spread over a state, with all that entails. I also discussed the notion that there is a latent demand for software with such a capability in that post.

Our experience with such a resource sharing network has led us to conclude that:

  • Library users like access to the large virtual library that such a consortium gives them.
  • They don’t care about our politics or our difficulties under the hood.
  • They will bypass libraries without access to the virtual library in favor of those with easy access to those consortial resources.

Welcome to the long-tail, Google world. Our users can see a world without Balkanized information resources and they want it.

Of course, with the open source movement drawing attention, we find that the vendors of proprietary ILSs are more open than we are—just ask them. Similarly, everyone does resource sharing now from individual libraries which do not have a single union catalog. Ask them how many instances of their software they run at the consortium and have them describe how they migrate records from one system into the union catalog.

Be that as it may, how did we come to this place where we have all these individual libraries doing their thing? The pattern is an old one and the term of art to describe it is “information silos.” I defined information silos for the talk as separate and barely communicating collections of information. Interlibrary loan is a sluggish, time consuming, and awkward way of sharing materials between libraries. Its cost in time and complication reduces its use and its effectiveness. ILL is an example of bad communication between libraries where separate libraries barely make materials available to other libraries.

PINES breaks information barriers between member libraries and their users and removes the awkwardness of ILL by making it a type of internal circulation transaction, termed a “hold.” Not a kludge bolted on between library silos but an integral part of each library’s system and accessible to a library’s users as they search the catalog. Holds are much easier than ILL so we find users availing themselves of the option in astonishing numbers. You are seeing not only the long tail but that part of the tail that is just slightly longer than an individual library user’s collection—the slightly longer tail, if you will. Why do we have silos? In this talk, I offered these possible explanations:

  • Legacy vendor’s lack of vision
  • Our lack of vision and/or politics
  • Until now there was not software robust enough to run large consortia

There may be others. For instance, I fear in my darker moments that some of our colleagues do not like to share but my sunny disposition will not countenance that thought at this time.

One of the questions that folks ask us is: aren’t you just copying the legacy vendors? It looks that way, doesn’t it? Because in this market, librarians expect “integrated” systems. That is how Evergreen presents. But, underneath, it is completely different. It is modular and built on the OpenSRF architecture I mentioned.

Of course, everyone is modular these days—just ask them. We mean modular in the sense of distributed nodes in clusters not merely modular as a means of monetizing components. An example: If your Evergreen installation outgrows its servers, you buy another commodity server, install the software, and the new box and old boxes start talking and away they go. You don’t have to go out and by another million dollar server because the software you are using was designed when mainframes were in vogue and that is how they did things way back then. The future will give librarians a host of different solutions that we can only speculate about. One we discussed at VALE was that if you didn’t like the Evergreen’s circulation module, you could replace it with one that you liked better. Mix and match of independent modules seems very likely will be a part of the library software world of the future.

However, we can do a bit more than speculate now about one development. Mike Rylander, VP for Research and Design, and I gave a talk on January 17 at the State Library of Ohio in response to a Request for Information about upgrading a resource sharing model they have used, MORE. Our proposal was called FulfILLment and Mike’s earlier post has the diagram and more discussion of it. It was an attempt to get past the silos in another way. This one uses the proven Evergreen backend for real-time processing with opportunistic connectors to legacy vendor software. “Opportunistic” because many things that we would hope would work don’t.

A user of a FulfILLment(tm) library would have a better experience now because FulfILLment would simulate much of the capability of Evergreen to make a virtual catalog available at the time of search. Would it be as good as Evergreen? No, but it would be pretty close—certainly better than what most libraries have now.

FulfILLment is Mike’s idea and his thinking is now beyond where he was in January and we are seeking development money to build it. After it was developed, it would be released under an appropriate open source license for use by the community.

The Ohio State Library was facing a dramatic budget shortfall and that is where the project stood until VALE. Now, we seem to have more interest. I certainly hope so. If libraries are to build or to have a part in the information environment of the 21st century, we better get going real quick, now. I submit that if the ARL libraries, for instance, were reachable through a FulfILLment(tm) interface, it would have a profound effect on scholarship and the learning environment at ARL member institutions—as it would at any set of academic libraries using FulfILLment.

And why not public libraries? If all or a substantial number of libraries in PALINET or SOLINET were reachable through that kind of interface, it would also have a profound effect on the use of member libraries and be a big step to moving libraries closer to the center of the information society instead of out where the buses don’t run.

Bob Molyneux

Posted in Announcements, FulfILLment | No Comments »

The Path to FulfILLment™

Thursday, March 13th, 2008

FulfILLment OverviewOne of the smartest data guys I know, Equinox’s Bob Molyneux, is in New Jersey as I write this talking to the New Jersey VALE group about Evergreen and consortial ILS deployment, a topic that we at Equinox know a bit about — we manage a State-wide consortium in the US and a growing Province-wide system in Canada, and there are several other installations well under way.

Why are large consortia forming around and with Evergreen? To answer that question we must first define what a consortium is. There are different types of consortia, and each consortium of each type has its own subtle variations from the broad definition, but basically they can be broken down into three main groups

  • ILL consortia like GIL — These consortia build union listings for broad search. Some ILL consortia will attempt to smooth the way for interlibrary loan by mediating requests through a central system, though require that all participating libraries use the same ILS for seamless ILL and are notoriously difficult to configure and maintain.
  • Purchasing consortia like WALDO — These are generally meant to provide leverage when negotiating with vendors. Direct resource aggregation is not their primary concern. As such, these consortia do not directly leverage economies of scale for implementation, and they generally end up being as expensive as taking the ILL consortia route to resource sharing.
  • Consolidating consortia like PINES — These consortia work to realize the full benefits of interdependent cooperation through broad resource sharing initiatives, such as a single shared ILS implementation.

Evergreen directly addresses the needs and concerns of Consolidating consortia by providing a way to consolidate hardware, software, support and administrative services, while still allowing local control over policies and procedures. This works, and works well, and is inspiring the creation of other Consolidating consortia throughout the US and internationally.

(As an aside: yes, I know you’ve heard that Evergreen requires adopting uniform policies, but that’s simply not true — it never was and never will be. Every participating library in an Evergreen installation can implement their own local policies, though there are benefits, especially for patron experience and staff training, to adopting similar policies.)

But what do all of these types of consortia have in common, in terms of what they attempt to address?

Markets. It’s all about markets.

Things

First, there is the internal market of resource acquisition within a set of cooperating libraries. Resources are scarce and must be allocated in the most efficient and effective manner possible.

Because ILL consortia do not attempt to consolidate, but rather duplicate ILS installations, this market is not addressed in any direct way.

The issues here are partially addressed in the Purchasing consortium model, though only on the procurement end and not on the resource requirement end.

However, this is one of the core goals of Consolidating consortia, and one of the driving forces behind the initial development of Evergreen. The reduction in direct costs that come with an Evergreen powered consortium is undeniable and amazing. What were once repeated, per-system costs for licensing, hardware and support becomes a single shared investment which can be 90% smaller, a full order of magnitude, than the aggregate for existing individual systems. This economy of scale is unprecedented in library-run software, and presents wholly new opportunities for self-direction and resource distribution.

People

Next, we have the external market for patrons. Libraries, for better or worse, are traditionally silos. In practice, they build local collections, limited by available resources, based upon what they believe will most benefit their patrons. Every community is different, and so every collection is different. Patrons, however, are not concerned about local library politics or the intricacies of collection development. Patrons want to retrieve the items they need, and they want to do so as efficiently as possible. They vote with their feet, and will follow the path of least resistance to the items they desire.

ILL consortia attempt to address this issue by building a new system around and on top of existing standalone ILS installations. These union listings, though, are not authoritative and are generally not updated in real- or even near-real time.

Purchasing consortia do not address the issues of the patron market in any direct way, as their constituent libraries do not (within and for the purposes of the purchasing consortium) aggregate collections.

Consolidating consortia, on the other hand, address the issue by being the authoritative source for information. They can provide a seamless experience to the patron and drive circulation of the “long tail,” helping patrons discover resources they never knew were available. In PINES we have seen a trend whereby patrons from areas served by non-participating libraries will travel out of their way to a PINES library because PINES has a larger, broader collection — nine million items on 1.8 million titles — than any one participating library could hope to amass. Put another way, patrons will seek out the larger, less convenient collection in preference to the smaller local collection, period.

So we have shown that one way to positively position libraries in both of these markets is to consolidate services in a single large, scalable system. To date, Evergreen as implemented at PINES is the largest single-instance public library ILS deployment in production in the United States with 275 outlets in 49 distinct library systems. To the best of our knowledge it is also the largest single-instance public library ILS deployment in the world. Evergreen is also the first, and currently only, ILS built to deal with the complexity of such a consortium, and is proof positive that Consolidating consortia work and that patrons and staff reap the benefits of addressing the issues in both of these markets.

TIMTOWTDI

There Is More Than One Way To Do IT — the Perl motto

Consolidating consortia, even implemented with Evergreen, are not without costs and great effort, however. It takes strong leadership and great political will to build a large Consolidating consortium. It requires buy-in from all parties of interest, and a willingness to work together and compromise. Even with all this, timing and existing contracts can make building such a consortium nigh on impossible. So what’s to be done?

Enter: FulfILLment™

FulfILLment is a new Open Source product being designed and developed by Equinox Software to help address cost and patron issues from the point of view of an ILL consortium by applying the lessons learned building Evergreen for the Consolidating consortium model. At the same time, FulfILLment aims to provide enhanced services to both patrons and staff, lowering the barrier to entry for all users.

FulfILLment leverages the underlying architecture of Evergreen and many of its concepts and algorithms. Some of the greatest strengths of Evergreen — its hold and circulation policy flexibility, its open and extensible OpenSRF architecture, and its easy integration with external services — are at the heart of FulfILLment and can be applied directly to the problem of inter-system ILL.

FulfILLment does this by redefining the problem space. Conventional wisdom says that ILL is conceptually distinct from circulation. The problem of automated ILL has thus far been addressed mainly through the use of union catalogs. This eases the search and discovery portion of ILL, assuming that the union catalog is up to date, but many union-catalog based systems do not address automated mediation well, if at all. In fact, many systems are little more than cross-lending agreements on paper that require full staff mediation. The reasons for this are clear if you consider these points of traditional ILL:

  • Little or no a priori knowledge of item or patron status
  • Little or no surrounding context for the ILL request
  • Either: forced ILS uniformity to provide seamless ILL; or spotty interoperability of NCIP and related protocol implementations, or in the worst case, so-called “paper ILL”

All of which lead to complicated data gathering, retention, aggregation and reporting.

However this conceptual distinction between ILL and circulation is not necessarily correct, given the proper circumstances. Consider these counterpoints from the view point of a circulation system:

  • Complete global state knowledge
  • Complete context of hold requests
  • Implementation based on “best and most appropriate” protocols

FulfILLment brings the benefits of a circulation and hold system based on the core algorithms in Evergreen to the ILL problem space. By encouraging and facilitating the participating institutions to collect and enter all relevant information about ILL policy and system definition, FulfILLment can provide not only truly automated mediation of ILL request (holds) but also full ILL transaction (circulation) management and automated transit management.

If the infrastructure and algorithms of Evergreen are the heart of FulfILLment, the FulfILLment Next Generation Discovery Interface (NGID) is its public face. The FulfILLment NGDI is a hybrid physical/virtual union catalog which automatically loads and deduplicates bibliographic records from all participating institutions for central search. Records can be pulled in using standard protocols such as OAI-PMH, Z39.50 or SRU, or can be automatically pushed into the NGDI from the local systems by whatever means are available.

FulfILLment leverages the hold targeting and capture algorithms from Evergreen — arguably the most advanced in the world in terms of policy and process modeling — and uses them to find the best item to fulfill the request of the patron based on all available information.

FulfILLment’s hands and eyes consist of the Local Automation Integrator, or LAI. This system uses the best and most appropriate protocol for each participating institution’s ILS, be it NCIP, SIP2 or a custom connector, to query each local ILS that advertises items on the requested record in real-time in order to provide status information to the hold processing mechanism. Data concerning the requesting patron is also pulled in real-time, and can be automatically obfuscated or expunged if required as soon as all open ILL requests and transactions for that patron are completed.

Once an ILL request has been accepted by the lending library and the item captured for remote circulation, FulfILLment tracks the complete life-cycle of this transaction. Both ends of the ILL, the lending and borrowing libraries, know the current status of items on loan and can generate reports based on this information using the Evergreen Reporter, one of the most advanced and flexible reporting engines available for an ILS.

The road ahead

This true end-to-end management and automation of ILL, particularly in an ILS-agnostic fashion, is something that has not yet been achieved in the library world. The potential benefit to staff in terms of reduced workload when fulfilling ILL requests might be enough on its own to make FulfILLment worthwhile, and when the cost reduction benefits of Open Source software, the advanced Next Generation Discovery Interface and the capabilities of Evergreen are brought to bear on problem of ILL, the return on investment will be immense.

We at Equinox are truly excited about the possibilities that will open up for building and growing ILL consortia on a scale not yet seen. You can expect to hear more in the near future and we hope you will all join us on this path to FulfILLment.

–miker

Posted in Announcements, Evergreen, FulfILLment | 2 Comments »