Library.nu: Modern era’s “Destruction of the Library of Alexandria”

The description above may come as a surprise to people who never sampled the wares of this fantastic resource.  But the description, made by someone named Samir Huseyn on Twitter earlier today, is a fitting one.  It was likely nowhere near as extensive as the library of Alexandria, but the last time I visited, it had thousands of pages and likely almost 100k objects in its catalog (according to the story below it was over 400k).  But for the global patrons it served its most attractive quality was that it served them all up for free.

As of yesterday morning, the library.nu is no more.  For the past 24 hours, dedicated users and community members scrambled to find out what had happened to it.  Over the past few weeks, there have been dramatic changes to the login interface.  At first, the login page would display nothing in the way of the works behind it; then a week or so ago the previous login page was replaced altogether: now library.nu pretended to be a “book review site,” displaying only the reviews members had left of the pirated works they previously dispensed.  By Monday morning, the site was dead, its member rolls purged to prevent login from the direct link, the main page (ironically?) redirected to the Google Books frontpage, and the admin email producing an automated response:

I started watching the Twitter feed for “#library.nu” or its alias gigapedia.info for any word of what had happened.  My first difficulty was finding anything about it in English. Amidst the Chinese, Korean, Continental and Eastern European languages, there was an occasional English tweet, but most of these were of the “WTF?” variety: no more info except a confirmation that something was going on.  A post on a Spanish language blog confirmed that there was something amiss, though that was last week.  Like many previous tweets - or the English ones I could read - what these actually illustrated was less that there was something wrong with the site and more that there was a deep, globally felt felt fear that something might happen to it.  With Megaupload, BTjunkie, Pirate Bay, and other sites falling (even, as it were, in a SOPA-free world) recently, it was a reasonable concern.

This morning, it finally came out (first in Italian then German then English)

An international alliance of publishers, including Cambridge University Press, Elsevier and Pearson Education Ltd, has served successful cease-and-desist orders on a piracy operation with an estimated turnover of £7m

The two platforms, sharehoster service www.ifile.it and link library www.library.nu, had together created an “internet library” making more than 400,000 e-books available as free illegal downloads. The operators generated an estimated turnover of €8m (£6.7m) through advertising, donations and sales of premium-level accounts, according to a report by German law firm Lausen which helped co-ordinate the alliance.

The other publishers involved also comprised Georg Thieme; HarperCollins; Hogrefe; Macmillan Publishers Ltd; Cengage Learning; John Wiley & Sons;the McGraw-Hill Companies; Pearson Education Inc; Oxford University Press; Springer; Taylor & Francis; C H Beck; and Walter De Gruyter. The alliance was also co-ordinated by the German Publishers and Booksellers Association (Börsenverein) and the International Publishers Association (IPA),

If the list above seems like an unlikely bunch of transnational interests, that’s only because you never visited this library.  It mostly trafficked in science and engineering textbooks (like many such sites) with an abundance of other monographs and trade titles, including enormous collections of journal articles and the latest epub and .mobi encoded editions of bestsellers.  These were all carefully cataloged, with metadata linking them to subject headings, a description telling you what kind of text it was - whether it was merely a page image capture, one with OCR making it searchable, or a fully digital, publication quality pdf.   

None of the files for these books, however, were archived on the site.  As the news reports allege - and as several astute users identified early on - there seemed to be a direct linkage between the locker site ifile.it and library.nu.  Many files were mirrored on both ifile.it and mediafire, but all of them were at least hosted on the .it site, perhaps signaling the reason the Italian Publishing Association was one of the first parties included in the crackdown (according, anyway, to my rough translation of the Italian news releases.)  And though there wasn’t advertising on the library.nu site, major advertisers (some of whom seem to be party to the crackdown, others like Hotels.com are less directly involved) placed prominent ads on the pages of ifile.it.  It’s actually quite sensible that they be connected, though I’m sure the actual legal and financial arrangement is more complicated: why go to the trouble of separating these if it could be easily demonstrated all the money was going to the same pot?

In any case, as with the Megaupload take down (and many others before that) the overall haul of the operators is hard to pin down - is £7m the turnover in one year? Throughout the life of the site? Does it include operating costs for hosting the 400,000 works?  And, since this is a coalition of international publishers (or at least an international coalition of publishers) what percentage of their income does this £7m represent?  I suspect the reason there is even mention of this amount is that this is the most important thing they can charge: simply giving away files is a tough thing to prosecute unless you can also prove there was some ill-gotten loot on the part of these IPR thieves.  On the other hand, there are clearly other relationships here.  I don’t know what ebooksclub.org is, but most of the obvious publication grade pdfs were, at least for a time, given a prefix indicating they were from that site.  That site now defaults to library.nu, so maybe this is another layer of the operation.

I don’t want to get wrapped up in the INTERPOL intrigue - though I’m glad to have some more insight into how the operation functioned behind the scenes, from here on out the media story will be mostly focus on vilifying them for trying to fill an important but underserved niche.  It isn’t merely, as the news stories so far have dutifully reported verbatim from the press release, that these are “freeloaders who make unjustified profits by depriving authors and publishers of their due reward.”   And it is laughable to claim that this takedown will create, “a more transparent, honest and fair trade of digital content on the Internet.”  As the music and movie industry have demonstrated, giving more power to rightsholders is usually the best way to create a more opaque, monopolistic and ultimately unfair trade in digital content.  

None of this is to say that the International Publishers Association isn’t perfectly correct in calling this an illegal operation and its main owners “criminal” (though “highly criminal” seems a stretch); it was very likely illegal.  Whether this is right or not is another question entirely.  But this wasn’t just a “lucrative operation” (we don’t actually know how much it cost to operate it, and if the MPEE study of torrent sites is any guide, there were likely lots of costs involved since they were paying for actual hosting. UPDATE on this from Torrent Freak says they barely covered hosting costs)  If it was in any way “organized copyright crime” it was organized in a very widespread way, with community members encouraged to submit their resources to the site to be cataloged and stored.  More importantly, it obviously served a need, particularly in what the MPEE calls “Emerging Economies.” In looking at the global outcry on the Twitter feed, it is clear that many people in foreign markets relied upon this as an important resource.

For me, it is mostly an inconvenience.  Instead of being able to easily find and quote a passage from a digital copy, I’ll have to take the time to find a print copy from my bookshelf or my library. Then I’ll have to type the whole damn thing like someone from the 1950s.   Like most first world pirates, I’m as likely to buy a print copy once I download a digital one, if I haven’t already bought it). And since most of my interests are in the humanities and social sciences, I will rarely find a book I absolutely couldn’t afford to buy (though I’ve come across a few).  But for many people in more dire straits, the loss of this library represents not just the loss of an intellectual resource, it may mean the difference between being able to afford their schooling or not.  

On this front, even in the first world students have faced increasing textbook costs and as Audrey Watters points out, etextbooks have been a very poor replacement precisely because they actually exacerbate the cost.  Not only are they no less expensive, but they can’t be shared, or sold, or bought used.  Add to that that they have less functionality than a regular textbook (for the most part you can’t annotate them well and no one supplier has all of them: instead of going to one bookstore for all classes, they have to make deals with different vendors with different interfaces) and it makes it so that even the people who could afford to prefer an e-text over a real one would be more likely to look for it elsewhere.  

The most exciting venture on this front in the legit, publisher owned e-textbook world is at Indiana University.  It is a fine project in its own way - they’ve developed an HTML5 hosting platform that allows for lots of cool highlighting and social sharing options -  but to listen to the IT head who had to negotiate the deal it had a lot more to do with lawyers than educators.  For one thing, they had to join with several other schools to make it possible and negotiate as a group.  All of the other schools were also large universities and the lawyers trying to make arrangements with just five publishers racked up a hefty bill.  With all this heft, the only way they were able to negotiate a slightly reduced price for the content they were accessing was by forcing every student to buy the book - no used books, no shared books, no borrowed books: the cost of the e-textbook is included as a course fee charged to every student in the course.  And they are allowed access to it through a University portal for as long as they are in college, then they no longer have access to the text.  

All of this for saving an “average” of $25/textbooks, meaning if you happen to be one of the 5300 students in the program who takes all the classes in the program, then you will average that savings; if you’re only taking one of them, you might be saving a great deal less.  It’s a good plan for students like Obama’s health plan will be for the uninsured: you’re forced into a plan that prioritizes corporate largess over its human priorities, but at least you get served.  As the New York Times put it, it is a program which “Focuses on Bulk Savings, not iBooks.”  Or, in the words of the IT head, Osborne

The model is agreeable to publishers and authors because they are guaranteed income that would otherwise be lost if students bought used textbooks. “We thought, ‘What if we made a model that gives money to the publisher and the author every year, provided it’s low enough for students to afford?’ ” Mr. Osborne said on Thursday.

I’m not faulting Osborne for beginning from what would be good for publishers: though his primary mission is to his students, he realizes that the only way he can even begin to imagine serving them, is if he starts by considering what would be good for the rightsholders that effectively make all the rules in this environment.  Only by beginning from what will make them happy can he hope to serve his educational mission.  

So far, we’re still in the relatively developed world. What led me to library.nu yesterday was discussions I had with a few people about the new open education courses at MIT, through their platform MITx.  It is basically an extension campus on the web, only you may or may not be able to turn it into a credential.  They don’t seem to be aiming as high as the Stanford AI MOOC that led to around 500,000 students starting the course at some point (only 30,000 or so finished.)  But it is still billed as a good, economical way to get the information.  More importantly, in the higher ed-tech-disruption circles, it is seen as more evidence that online is the wave of the future.  

But scroll through the course and you find that the textbook is written by the professor - but owned by Elsevier.  There are excerpts from the text on the website, but if you want the whole book, you have to buy it from Amazon or the publisher.  So students might be able to take a free class, which may or may not get them any closer to a college degree, but if they really want to understand the material, they have to buy an $85 textbook.  I suppose it is a small price to pay for a college class, but a cynic could point out (not me, of course) that it’s also a fantastic way to sell more books.

Again, I don’t fault the professor or the institution for trying to work this out in an above board, satisfactory fashion.  Elsevier has powerful propagandists and likely much more powerful lawyers: they consistently preach a doctrine justifying their 35% profit margin by speaking of the incredible value they add to the works they publish - and if you ever stop believing, they remind you how much you’d need to pay to absolve your wayward soul.  Even if you wanted to argue - you couldn’t.  In any case, this is a very special arrangement that has to be made for this model to work.  If the only classes schools can offer online are those where the professor has written the main text, it might be a slim set of offerings indeed: hardly the basis for a new model of education.  And, again, central to it is the problem of the textbook.

The outside option here is that schools start using open educational resources, and there is a large movement around this as well - both schools trying to develop them and educators developing texts to use.  If this was a more common practice, it might begin to eat away at the smug satisfaction textbook publishers must feel when they walk away from delivering a deal that so cravenly favors them over students.  Preventing a shred of (their own) humility is the main goal of these publishers, which is why library.nu was so important to eliminate.

Once we slip the heavily fortified borders of the so called First World, the existence of library.nu is easily explained by the same situation that the Media Piracy in Emerging Economies report does for movies and music “piracy.”  

Multinational pricing in emerging economies, [in contrast with domestic], signals two rather different goals: (1) to protect the pricing structure in the high-income countries that generate most of their profits and (2) to maintain dominant positions in developing markets as local incomes slowly rise. Such strategies are profit maximizing across a global market rather than a domestic one, and this difference has precluded real price competition in middle- and low-income countries. Outside some very narrow contexts, multinationals have not challenged the high-price/small-market dynamic common to emerging markets. They haven’t had to.

In other words, multinational companies have to protect their price gauging at home by making sure there aren’t pirated works available in the global, digital marketplace: if you are a regular old student, you might just avoid the hassle altogether and get a digital copy.  The more widespread this practice became, the more acceptable, the less money they would be able to force from each student, at least on average.  So they have to kill a service when it grows, not because it threatens their international market per se, but because it threatens the one at home.

Likewise, this means they have to be careful about even the price of legal textbooks in markets of middle- and low-income students (at home and abroad) (especially in technical fields where the cultural discount doesn’t eat into its global value, i.e. the English version could suffice worldwide).  Even if the textbook was just marginally cheaper across a border or two, core students might be tempted to have one shipped - or an enterprising businessperson could start some sort of portal where the cheapest textbooks in the world are shipped everywhere else.  

Maybe something like this already exists, but the price of US (and European) textbooks is targeted at their core market in roughly the same way are the goods of the movie and music industries that are the main subjects of the MPEE.  This means that, for people who live in a market like South Africa, political, economic, and racial restrictions and divisions make (or have made) affordable textbooks hard to come by.  As the report notes, in South Africa,

Textbook piracy remained ubiquitous and, according to the International Intellectual Property Alliance (IIPA), was responsible for larger total losses than either film or music piracy throughout the 1990s and early 2000s.

mind you, this was before digitization.  I suppose if students there want affordable, legal, knockoff textbooks, they’ll have to get Bono to beg for them like he did for AIDS generics. Or they could just head over to library.nu.  Or could have. 

I’m not defending criminal behavior, but as I’ve said before, economic realities mitigate against a clear definition of just what is criminal in this context.  Whatever the juridical shortcomings of the digital library they made possible, it was the easiest way for anyone to get digital content from these publishers, even if they wanted to pay top dollar.  And a quick glance at the nationalities or at least linguistic diversity of its mourners on Twitter should exhibit how wide a potential public it could serve.  This was a working model of the library of the future; a functional collection that made education and exploration possible in unheard of fashion, but still needed vast improvement (and, of course, a legal license.)

In some ways, its (possibly) ironic redirect to Google Books is a sly nod to its brethren.  Where Google Books - and its settlement with publishers - fails, it picks up the slack.  Hate searching for passages in a book and only being treated to snippets because a publisher demanded it: search in Google Books, then stroll over to the library.nu and actually be albe to read a couple of pages around it - like all of them.  

In enormous servers, in buildings around the world, are all the books.  Archive.org has them; Google has them; Hathi has them; and those they don’t have they’re adding every day.  But instead of being able to magically call them up on your device using the contemporary technology in all its glory, like a chump you still have to putter around trying to find where a digital version can be legally bought in a format you can load onto your e-reader with a set of rights even remotely resembling those you would get for roughly the same price in a physical book, i.e. you can mark it, save it, sell it, lend it,  give it away to Goodwill or send it to poor kids in South Africa.  In other words, most of the time you have to sit back and bide your time until the pirates really sink this ship and make it possible for literature, libraries, scholarship, and reading to move into the 21st century for the bulk of the world’s population.  Today, for all its faults, we lost one important trailblazer for that future.  As one Tweet put it:

The upside of total human annihilation for creative alien scavengers of the future

I would like to start a journey - towards thinking about the library of the future - with what might be a peculiar example.  P.D. James begins her novel The Children of Men with an explanation of the tragedy that has befallen humankind in the late twentieth century: “For all our knowledge, our intelligence, our power, we can no longer do what the animals do without thought.”  Namely, they are unable to reproduce.  The story opens in the year 2021 with the observation that the last human born of natural birth – to a mother in the Brazilian favelas 25 years earlier – has died.  The finality of this young man’s death, what it signifies as a milestone in the history - and the future - of the human species, is part of what prompts the narrator’s reflection.  

The future, in this case, is one in which the human species no longer exists.  This presents a vexing question that we often pose in relation to individuals, but rarely in relation to entire cultures, societies, or even species: what would you do if you knew you were going to die?  Live it up with all the drugs and meaningless sex you can muster. Travel to all the places you always meant to go. Finally and fully engage in a deeper relationship with a supernatural diety or earthly soul mate.  The answers to these questions animate James’ account in lively ways, particularly in relation to the inhumanity that seems only possible on a grand scale of an entire society – when fascism, violence, hate, and xenophobia see little need to hide themselves for the sake of posterity.

But in a sense the exhaustive variety of possible answers to this question would likely find their representation somewhere among the myriad members of the species.  So it isn’t surprising that one of the inclinations James highlights is something like that of several Early Modern sonnets: to attempt to guarantee immortality through archiving one’s thoughts (or the youthful beauty of one’s lover) in writings and one’s writings in a folio. When Shakespeare records the living memory of his beloved, for “the eyes of all posterity” or Spenser uses verse to eternize the virtues of his, they did so with the reasonable belief that, even if no one found his writing or was inspired by it, someone COULD find it. Someone alive, later.  

For those living through the parallel universe of James’ book, there could be no such promise.  Recognizing that this might make the situation sound even more desperate, James presents, if ever so briefly, a source for hope on this front.  In the late 1990s, shortly after the last natural born human took his first breath, scientists had discovered alien life on a far off planet, thousands of light years away.  There is no contact with this lifeforce, no communication with its beings, no guarantee they had or ever would have the capability (or desire) to visit the literally posthuman planet earth.  Yet the glimmer of hope gives those inclined to speak to the ages the motivation to carry on.  As James tells it, “We are storing our books and manuscripts, the great paintings, the musical scores and instruments, the artefacts.” 

Every piece of human knowledge and culture would be carefully archived in as uniform a fashion possible, with as many redundancies as the system would allow.  Shortly, they would be sealed up as the last humans died off, so that when the alien life comes to earth, they will be able to learn what we were, what we did, what we thought, and, perhaps most importantly, how we felt about it.  It would likely take generations (assuming aliens live about as long as humans) of dedicated alien scholars poring through these archaic codexes of whatever variety they took before they could finally claim to understand us.  But to the hypothetical crafter of (hopefully) universal metadata standards, this possibility was enough.

James doesn’t give insight as to why the human race is dying - except to say that its inability to discover the cause of this disaster is the crowning insult of its misplaced faith in science. But it is not outlandish to imagine a time when something like this scenario could come to pass.  In previous years, nuclear holocaust topped the list - here the insult to science was that man was too active, discovered too much and then did things to cause it.  Still, at least there is agency involved.  In our current era, the most likely doomsday scenario is caused by us doing nothing at all: we know a great deal about climate science; we can predict with computer models what the sea-rise in Bangalore will look like; can calculate the human cost of this otherwise unimaginable catastrophe; yet for the most part, we do nothing.  This is the recurring lesson science fiction has to teach us: that we place too much or not enough faith in science, that we had the answer, but our hubris and greed kept us from turning the tide: in other words, our base humanity keeps us from using our science to its most enlightened potential.

This is a grim foundation on which to build the library of the future: a masoleum for the interstellar archeologists who might one day be interested in our culture.  But we really should look on the bright side: just think, when those aliens discover this vast trove of information they will be able to do whatever they want with it. While this could mean they use most of the storage media for fire wood or a coltan salvage yard, in the best case, they would be even more advanced than us, technologically, socially, and culturally.  In that case, they could create infinite copies to distribute throughout their society, drawing upon the networked knowledge of everyone to help interpret and understand the lost civilization they had happened upon in their travels.  

They would be free to reinterpret works using their own cultural framework, making sense of them despite (or even because of) the distance in time and space.  And this reinterpretation could take any form they liked and be redistributed and reinterpreted until the ideas and history we had bequeathed to this galactic posterity merged with the great mind of the universe and we achieved an immortality Shakespeare could not dream of.  All of this would be possible whether the works were from Shakespeare’s time, published in the nineteenth century, recorded in the twentieth, or released as an ebook in 2012.  All of this would be possible not just because diligent archivists kept track of it, or brilliant coders fabricated a Rosetta Stone legible to the stars, or curious extraterrestrials plugged away at understanding it: no, all of this would be possible because, for these incredibly fortunate future invaders, all the information in the history of the planet earth will be in the public domain.

Roundup post

  • danah boyd reprises her infamous manifesto on open access publishing, saying “Save Scholarly Ideas, Not the Publishing Industry.” I feel a little more sympathetic to junior scholars who send work to journals of the evil corporate variety, but she usually makes waves so its worth noting.
  • Andy Baio’s post on young people’s perceptions of copyright set the Twitter abuzz last Friday, surveying Youtube videos and users and finding that they have an almost comical misunderstanding of what copyright means.  The phrase “No Copyright Intended” or “No Infringement Intended” shows up often, indicating that many users seem to think their release of any claims on the material they have created (using others copyrighted material) is enough to shield them from legal action.  From this Baio proposes a thought experiment (on the lines Lessig proposes in Remix) of what will happen when these people reach voting age and are able to get elected to office. Rob Beschizza, on BoingBoing, notes that 

Isn’t it also interesting how many young artists still instinctively honor the idea, as they see it, of copyright? Respect for other artists comes naturally. People don’t stop respecting copyright until they see how little the claimed principles have to do with the reality of enforcement—especially when it’s used to condem their own creative expressions as a form of theft.              

  • Heather Morrison, an ABD student at Simon Fraser and powerhouse of information about Open Access resources, provides her “early year end” report on the success of open access.   From her perspective, the emerging challenge of Open Access is the challenge of success: “How can we track all these resources and make it easy for people to find and use them?”  This seems to connect Open Access with the library and information sciences in a very direct way, namely, while creating new, legitimate platforms (and, as boyd points out, practices) is essential, just producing open access scholarship isn’t enough: we need to develop new forms of curation and data management for that scholarship to be useful.  
  • On that topic, it took this story from across the pond (Economist) to alert me to the CHNM initiative Occupy Archive, which is collecting materials from and about the Occupy movement for the purposes of historical preservation.  

Without some system for organising, collating and preserving the Facebook pages, YouTube videos and blogs the movement is generating, the materials may be lost. For archivists, the question is not whether Occupy movement has political legs, but if its history has a future.

  • Michael Robertson analogizes digital music with hot dog vending to show both the absurdity of contracts with record labels and in reason Spotify and other platforms will never make a profit in the current environment.  In the process, he reveals information that had previously been shrouded by nondisclosure agreements (just as Big Deal journal subscription packages with libraries have been for years). For me, this is further evidence that the morass of licensing books for a massive e-library would likely be an unsustainable nightmare in the current legal environment.  These initiatives would be made unsustainable by the people with the most to gain from their success, namely the copyright holders like the record industry. As Robertson describes it:

The supplier will always elect the formula that captures the largest amount of money for themselves, completely disregarding the financial viability of the store. If the store miraculously managed to generate a profit, the landlord would simply raise the rates after two years.

  • As if to cement this instinct, Librarian in Black reveals the fact that Amazon’s Overdrive catalog for ebooks is different for different libraries.  The Analog Divide asserts that this true of other vendors as well. Since these arrangements are likely shrouded in the same sorts of NDAs, it becomes hard to compare the deal you are getting - or even to understand what the deal you are getting entitles.

This is what happens when we well-intentioned librarians are expected to negotiate deals with these companies – and their experienced contract lawyers. We expect them to share our values of open access and sharing, while they’re beholden to their own profit motives. Essentially, we’re bringing hugs to a knife fight.

  • More on Libraries as Hackerspaces, this time from NPR
  • Canadian artists are lobbying to legalize file sharing, which should make Canadian torrent site isohunt happy.  It would be paid for by a P2P tax, which might make Canadian internet users sad.

more, soon

Contrary to many futuristic projections—even from bibliophiles who, as a group, enjoy melancholy reveries—the recent technological revolution has only deepened the affection that many scholars have for books and libraries, and highlighted the need for the preservation, study, and cherishing of both.

- William Pannapacker in The Chronicle, talking about the continued relevance of books as material objects and bookstores and libraries as sites of serendipitous discovery.  The need to preserve or approximate serendipitous discovery is an ongoing theme I hear often in relation to the potential transformation to a purely digital ecosystem for books.  Pannapacker spends most of his column discussing the continued aesthetic appeal of books as objects (a fact publishers are keen to), he is also concerned with how those objects work as components of a system of discovery.  Library stacks and bookstore shelves provide a different sort of organized chaos from Amazon’s “recommendations”  which tend, in my experience, to favor more recently published books rather than the “deep tracks” you might find sitting on a library shelf.  The nostalgia people feel for this is also pragmatic: finding unexpected books in this way is a mysterious, organic process which defies algorithms alone.  

And that’s part of the problem: most people’s everyday experience with e-books and e-libraries is based on like Amazon.com or Google Books, where, on the one hand, browsing the stacks is impossible and the adjacent texts you can discover are likely chosen by a mathematical formula rather than a more thoughtful set of metadata.  While these are relatively decent platforms for reading and even searching ebooks, they are poor methods of serendipitous discovery, at least as we normally experience it.  Amazon, for instance, provides on a sort of social network function, recommending books other people have also bought, and certainly I’ve found new books this way.  Goodreads recently launched a similar book recommendation engine which, as more people sign on, will likely become smarter at making predictions. The predictions, on the other hand, are based more on what other people like rather than what a bibliographer might see as a deeper relationship.  And anyway, seeing a few recommendations is not like browsing a shelf.  Google is beta testing its own version of a virtual bookshelf, which would be great if its metadata didn’t remain a complete disaster.  

 Like digital libraries more generally, the solutions (and problems) of serendipitous discovery seems to fall into three categories.  These are aside from the problem of actually getting and using the book or text once you discover it, but are related.  Going from back to front, the biggest problem is having content in the database.  In some ways, this is where Google has stopped with its process.  Get all the world’s information and put it online.  (I know they claim they will organize it, but when they say that they usually mean they will tell computers to do it).  This is a big problem, to be sure, as even I could create a perfectly organized, beautifully designed digital library of 45 books.  Populating that library with more volumes than the average physical library is one of the few things that makes it more appealing than walking over to the academic library.  Google and Amazon start from the content and work forward.  Their insufficiency is in the other two realms.  

One of these is metadata.  Though there is much more to it, if we are thinking of how to replicate the library experience, we should consider the backroom coding that helps organize the information on those shelves.  Metadata is a area I’m coming late to, being new to the library world, but it is also an area that is in flux.  The Library of Congress, for instance, recently announced that it felt the dominant model of metadata, MARC, should be slowly phased out.  This fits with the move towards forms of linked data, using Resource Description Framework (RDF) codes for every piece of information on the web.  CLIR, Mellon and Stanford recently held a workshop on this topic in relation to libraries and several of the DPLA Beta Sprint projects focused on either linked data or metadata interoperability. Importantly, this is the area where, despite its claims to the contrary, Google is in dire need of help from more invasive, expert forms of organization.  

But even with good content, well organized in the back end by good metadata, the average user will expect elegance and functionality in the solution to the most obvious problem: the interface problem.  Namely, how do you create an interface that provides the tactile satisfaction of walking through a bookstore or library, while having it still remain digital or virtual.  

Joe Esposito, writing at Scholarly Kitchen, takes a short cut.  He has proposed the idea of what he called a “Metadatatorium” where a physical space would remain to house the books, but the purchasing (or in the case of libraries, borrowing) would actually be done by waving your e-reader in front of a ISBD or QR code on the physical object.  In this, he’s inspired in part by this French video

Owners of these spaces would get a cut of purchases and could create more energy around their spaces by opening them up to performances, readings, book clubs, and so on. This would increase the possibility of serendipitous discovery as well as the communal and social aspects of bookshops.  Esposito doesn’t discuss how these books would be organized (i.e. what kind of metadata he’d rely on for their organization) and he seems to fall back on the organic, experiential sensibilities of the bookshop owner or library bibliographer for populating the content.  (Even more problematically from a legal and economic perspective, he prefers the agency model currently used by Apple’s iBookstore.  While the latter may have some appeal to publishers, this week both the EU and the US Justice Department are investigating this model for anti-trust violations.)

For a more virtual experience, researchers at the University of Calgary have elaborated several possible models of what they call “Bohemian Bookshelf Supporting Serendipitous Discoveries through Visualization.”  They begin by elaborating a complex understanding of how serendipity works, using a combination of personality traits (observation skills, open-mindedness, perseverance) and environmental factors (coincidence and prior categorizations of other people); then they speculate on what is needed to facilitate serendipitous discovery in a system, settling on four main factors: Multiple access points, multiple pathways, juxtaposition of information, and the flexibility to allow for curiosity and play.  They then speculate on how to organize books along a variety of axes and present them in a more organic manner according to these factors. As they describe it: 

The Bohemian Bookshelf consists of five individual visualizations: the Cover Colour Circle, Keyword Chains, Timelines, the Book Pile, and the Author Spiral. Each of the five visualizations provides a unique overview of the dataset from a particular perspective, as we will describe below in more detail. At the same time, the individual visualizations are interlinked with each other: the selection of a book in one visualization changes the views of the other four visualizations to in relation to the newly selected book.

They provide some illustrations of what this might look like in practice, helpfully collected here by someone else.

Their elaboration of a variety of interlinked interfaces is probably one of the best overall ideas—though it doesn’t seem all that deeply informed by metadata issues as the closest they get to a thematic arrangement is the keyword chain.  Add to this the fact that many useful manuscripts have no cover art at all (at least in hardcover) and it is clear the books this model has in mind are mostly trade books of some kind.  Still, the dynamic nature of the interface would allow for discovery of those volumes, provided they are part of the database and have the right keywords. 

The interface problem seems to be the other major concern of the DPLA beta-sprint projects.  The project bearing the greatest similarity to the Bohemian and Google Bookshelves is the project from Harvard.  It is a combination of a multilibrary, metadata-userdata “LibraryCloud” with a front end interface called ShelfLife.  Right now it suffers from having less content, but eventually it could be very effective as an engine of serendipity.  

Perhaps more innovative is the ExtraMuros project, which helps not only visualize collections geographically, but also allows users to curate data in a variety of collections. The DPLA page is here; and the demo can be found here.

My own ideal would actually require even more intervention, but would likely create an invaluable resource in and of itself. Not only would it aid discovery, but it would provide a visualization of deep connections between works.  I see the digitization of libraries as a unique opportunity to construct the outlines of another kind of “distant reading.”  The goal of this project would be to visualize the scholarly (and cultural) impact of a given work.  Ideally, a library of this kind would work more like Google Scholar, where books  and articles are included in the same framework.

I call it the Citemap. The proposition is this:  if/as we scanned/linked in the documents, we would pay special attention to the works cited and/or the footnotes.  On a basic level, the final project would show us, visually, what works each individual work had cited in making its argument, and more importantly, what the cultural importance (again, measured by citations) of those works was.

For example, take Raymond Williams classic examination of the idea of The Country and The City.  The reference section begins on page 167 of this pdf. Here the work in question, Williams’ book, would be the Node Work, the center of the node.  Depending on the filter you were using, you would be able to visualize either all the works that cited this work or all the works that he cited.  The size of any node would be determined by the number of citations.  According to Google Scholar, Williams’ would have a pretty good size node, as he has over 3000 other works citing his.  Likewise, most of the early works he cites (e.g. Virgil, C. Marlowe, P. Sidney) would have large nodes.  In each of these cases, it would be clear that both the work itself, and the works it draws upon are of some larger cultural significance. You could scroll over each node, ideally, recentering the study on a new node, checking how it fit within the larger network of both literature and secondary literature.are some central canonical texts—and what those are or have been at any given time (ideally you could also control for the timeline, rolling back the network to a snapshot of publication date to see when a work became popular or culturally/critically significant).  As we go through the data, figuring out a way to visualize these as a network would help to show the way these central texts are related to one another (or not.)

While there are a great deal of critical projects a resource like this could spawn, for the present purposes it would replicate in a virtual, visual way the most common source of my own serendipitous discovery: other authors’ bibliographies.  It wouldn’t have quite the tactile appeal of sitting with leather-bound volumes of cream colored paper, but there’s nothing to say you can’t go find a printed copy of the book you discover in the citemap (or the bohemian bookshelf, for that matter.)  

joshrushing:

I was on The Stream today to discuss drones. I also filmed an episode on the topic for Fault Lines. It airs December 26th at 2230 GMT. I’ll be live tweeting during that episode, so if you tune in, it will be like we’re watching it together. If you can’t tune in, it will be posted for eternity here.

Rushing and AJEnglish manage to squeeze a very comprehensive overview of developments around drone technology, from the many new practical uses (by both big brothers—government and industry—and citizens) to the ethical issues of privacy, legality, and increased incidence of PTSD by military drone pilots (a key issue which Rushing will likely explore in his longer episode of Fault Lines later this month.) 

Drones are definitely in the news and on people’s minds much more these days.  Yesterday Glen Greenwald had a pointed analysis of an NPR segment which he says, “purported to be a news story on the domestic use of drones but was, in fact, much more akin to a commercial for the drone industry.”  Greenwald is right to be leery.  The FAA will soon release new rules that will make it easier for drones to fly legally - making it easier and cheaper to acquire them for whatever use you desire.  As I pointed out a few weeks ago, only our continued vigilance and concern will help us avoid a future where police use drones to spy on us, hunt us down or stifle dissent.  On the other hand, the segment also casually mentions the more likely possibility (about which I also speculated) which would ensue before we would all be ruthlessly policed by a hovering equivalent of the telescreen: what if this technology were hacked by someone with different intentions than its legitimate user. 

While I relish the protest/resistance angle to this, the legal and ethical problems this might cause are the stuff of sci-fi writers navigating the ethical quandaries new technologies force us into.  i.e. my drone got hacked by a malevolent cabal of some sort, yet I’m on the hook for its actions.  In some ways, this would be more horrifying than the option a viewer speculated about—that the drones will become intelligent and turn on us #skynet style.

UPDATE: I can’t believe I didn’t link to the other story breaking this week in the drone category, following from this last point.  Though there is still dispute about what happened, Iran either hacked and brought down or at least collected in tact a downed US drone earlier this week.  Experts seem to differ on if it crashed itself, was brought down by hacking, or if it is even real.  Either way, the cultural attention it garnered seems to indicate the era of the drone is upon us.

Top