Week 6: Preservation in Digital Libraries

Research Challenges in Digital Libraries

We must research digital libraries in order to get a grasp on where we can take them. They are too widespread and heterogenous to really understand anything that's going on at the moment. We also need to figure out how to preserve the digital libraries as they are now for future study.

Big Issues:
1. We must figure out how to deal with all the digital libraries and preserve them while using humans as infrequently as possible.

2. We must protect the digital archives now. They require a lot of effort to maintain, so we must find a way to do that while, again, using humans as infrequently as possible.

3. We need to look at economic and business models of digital libraries to see how we can maintain these things in ways beyond technology. How can we afford to keep them up?

4. In order to expand the usefulness of digital libraries, new technologies need to be created. This needs to happen in order to make DL's cheaper while using humans as infrequently as possible.

5. We need shared and scalable infrastructure to support digital libraries. Sequestering them within institutions prevents interoperability and scalability, which hinders the usefulness of digital libraries.

Open Archival Information System Reference Model: An Introductory Guide

Open: reference model was developed in an open public forum: anyone could participate.
Archival Information System: people and institutions who agree to preserve info and make it available.

An OAIS must:
1. Get the appropriate information.
2. Make sure they have long term control of the information.
3. Know their user community.
4. Have appropriate metadata for the user to understand the info.
5. Make sure information is totally preserved.
6. Make it available to the user.

Tasks of OAIS:
-Ingestion (of data)
-Preservation Planning
-Data Management
-Archival Storage
-Administration
-Access (of data to user)

Types of information packages:
-Submission Information Package
-Archival Information Package
-Disseminated Information Package

This model provides a formula for digital library producers to follow. By doing so, they could produce an efficient, effective digital library. The paper does not provide any guidance on the technology or infrastructure to make this happen, but it does provide the guideposts of what sorts of things the technology and infrastructure must do.

Preservation Management of Digitized Materials
- The authors state that guidance is needed for digital preservation. It seems to be a recurring theme.

This book is to extensive to takes notes in much detail. However, it is an extremely interesting, useful guide for a novice in digital libraries to get a handle on the field. It introduces the reader to the vocabulary, provides reasons on why this information is vital, and explains how digital libraries are made, who uses them, what the rules and requirements are, and provides models for institutions to follow as they delve into this realm. Since this is a very new world, and many librarians are long out of library school, having this sort of resource, perhaps with additional instruction, they can get up to speed. Staying abreast of technological developments is important, and digital libraries are a huge part of that.

Actualized Preservation Threats
The National Digital Newspaper Project is an effort to "Chronicle America" by digitally preserving printed newspapers. It "also has a digital repository component that houses the digitized newspapers, supporting access and facilitating long-term preservation. Taking on access and preservation in a single system was both a deliberate decision and a deviation from past practices at LC." They wrote this paper to discuss the work done so far. Specifically, they discuss the preservation threats encountered by the project in 2 years.

Types of failures:
Media- Failure in the portable hard drives transporting the digital images from the awardees to LC. Fixed using 'fixity checks' as part of the transfer process and keeping a copy at the awardees until it was verified that LC had received it.

Hardware- Internal hard drives failed. They avoided data loss by using multiple HD arrays in a RAID 5 array with a hot spare. This prevented data loss in case one failed. Data was only lost when a second event occurred in the array while the system was rebuilding the harddrive using the hot spare.

Software- Three software problems occurred. The first involved a validation problem: records were put into the NDNP repository that had passed validation but 'did not conform to the appropriate NDNP profile'. This was fixed with new validation rules. The second was more problematic. During transformation, the newspaper title record had stripped the original METS record of the XML, and also, was producing invalid METS records. This broke the application, and also made parts of the data unreadable. The third problem occurred when the XFS file system was corrupted. This caused data loss. In a large, complex system such as this, it is harder to prevent problems, and to diagnose them when they occur. This is a serious flaw of huge digital libraries.

Operator- One error occurred when a series of files were deleted accidentally. Another occurred when the operator accidentally ingested the same batches multiple times, or perhaps did not purge a successful ingest before re-ingesting it. Many duplicates were produced.

The conclusions of the paper are that in a huge task such as this, errors are going to occur in many different ways, no matter what one does to protect against them. This makes performing a large digitization project extremely daunting, since one of the tasks is to make sure that the files are not only accessible but also permanently preserved.


This is Katie's favorite person in the world. His name is Kevin. Yes, all 3 of us have K names. It was not planned: Katie came prenamed, and we didn't have any choice in our names.

Week 3 Readings

Lesk Ch. 2

Computer typesetting:
1. Printers
2. Word processing
a. exact appearance of the text
b. content of the text

Text Formats
1. ASCII standard: 7-bit code for 26 Latin letters
2. Unicode is gaining popularity: covers all characters for all major languages in 16-bit-per character
3. Higher level descriptive systems: characters are marked for meaning
a. MARC: Machine-Readable Cataloging
b. SGML: Standard generalized Markup Language
c. HTML: Hypertext Markup Language

Document Conversion: analog to digital forms
1. Keying in: expensive
2. Scanning: less expensive
a. Optical character recognition: improving reliability
3. Converted documents can then be made online: digital libraries!

Arms Ch. 3
1. Structure: elements of the document: font, characters, paragraphs, etc
2. Appearance: How the elements are arranged on the page
3. Page-description languages: describe appearance on the page. TeX, PostScript, PDF
4. Encoding characters: ASCII, Unicode, transliteration, SGML, HTML (simplified SGML), XML (bridge between SGML and HTML)
5. style sheets (formatting on screen/printed page)
a. Cascading style sheets (CSS): used with HTML
b. Extensible style language (XSL): used with XML
6. Page description languages: layout
a. TeX: focus on mathematics
b. PostScript: graphical output for printing, with support for fonts
c. Portable document format (PDF): from PostScript. Similar attributes to reading paper, but on the screen. Can limit unlawful printing. Adobe provides excellent, free PDF readers, making the format widely accepted.

Identifiers and Their Role In Networked Information Applications
1. ISBN, ISSN, OCLC, RILN: make locating a given object easy.
2. New identifiers are emerging the electronic world: URLs and URNs
a. URLs: not long lasting locators, very ephemeral.
b. URN: naming authority identifier and object identifier
c. OCLC persistant URL (PURL): maintained for a much longer time than regular URLs- less likely to produce dead links.
d. Serial Item and Contribution identifier (SICI): using ISSN, can identify individual journal or article.
e. Book Item and Contribution Identifier (BICI): can identify individual volumes or chapters within a work.
f. Digital object identifier (DOI): based on the URN idea. Can allow copyright limitations to control who has what kind of access

Digital Object Identifier
1. DOI is the digital identifier of an object, not the identifier of a digital object. It is a persistent identifier.
2. It includes: Syntax (name), resolution of the name to the object, metadata describing the object, and social networking of the object through interoperability
3. DOI does not preserve the object: it merely finds a way of sharing information about the object.


These 4 readings are all centered around communicating meaning about a given object or text. The characters on the page don't mean anything to a computer, so it is necessary to tag them and use appropriate languages so that you can convey that meaning to the computer. When you do that, the computer can organize it in the way you want.

Affixing meaning also applies to identifiers. Without a good identifier, a given object will be very difficult to find. Providing an identifier like a DOI not only helps the user to access the object, but it also provides other information about the object that is translatable across a variety of mediums. This means that the record will be persistent.

All of this applies to digital libraries. What is the point of having a digital library if you can't find what you are looking for? Or if you may have found what you're looking for, but you're not quite sure if it is without looking at the entire object? Providing information about a given object is absolutely vital in any library, including digital libraries.

And, here is an entirely gratuitous puppy picture, for those who are interested.

We took her camping in Fayette county a few weeks ago. There was a lake there and she swam and swam and swam. She's a water dog, you might say.

Look at those little paws paddling! awwww.

Week 2 Response

First of all, the over arching theme of these readings is interoperability. A large emphasis is on interchangeable parts: different tools that can be exchanged and used as needed by multiple types of digital libraries. This makes sense, and is a concept that has been around for a long time. Car manufacturers save time, money and effort by building their engines and cars with a lot of parts that can be used in as many of their products as possible. By making sure that every car in their 2008 fleet uses widget A to complete task 1, they can make a whole lot of widget A's all at once and put them in every car. If some cars used widget A, others used widget B and the rest used widget C to complete task 1, they would have to make widget A's, B's and C's, and each of them would require a different factory or machine to produce. That raises the cost of completing task 1. It's what one might call 'reinventing the wheel'.

With this in mind, it is completely logical to take this concept into the digital library environment. Why reinvent the wheel? Obviously, different digital libraries are going to have different requirements, so they can pick and choose their given widgets cafeteria-style. This lowers the cost of developing the digital library. Hence, this is why the the Suleman article discusses producing software toolkits for producing digital libraries.

Furthermore, it allows for different digital libraries to talk to each other if there is a common language. This is a concept that is not new to libraries. Much of the technology that they produced before the digital age was focused on sharing information between libraries. Union catalogs filled this purpose by letting people know what various libraries had available. Bibliographies helped libraries know what's new in their particular field. In the digital universe, libraries being able to share what they have and have the collections communicate is a logical extension of this philosophy. The Payette article gives definitive protocols and evidence of their success for the interoperability of digital library systems.

Now. Is the Internet a digital library? It is a collection of data and information, in a digital format, that is stored on various servers and can be searched and accessed. By that definition, it is a digital library. However, the Internet is not maintained by a given body or individual. It is full of wrong information and a lot of the good information is hard to find. Much of it has restricted access. Amazon.com has servers storing a lot of personal data, but users can't access it using Google.

One might say that the Internet is a 'bad' digital library. It has many characteristics that the authors of these articles are specifically trying to avoid, and problems that they are trying to overcome in digital libraries. It seems unfair to declare something as a non-entity just because it is not a good example of it. It is akin to saying that your daughter is not your child anymore because she misbehaved.

However, learning how to overcome these problems and develop robust digital library systems could revolutionize the Internet. Perhaps one day the recalcitrant child will grow up to be a fine, upstanding citizen!

Week 1 Responses

While this is my first foray into thinking and learning about digital libraries, I have used them before. Pitt has one that I have looked at, and my local library (Mt Lebanon Public Library) has a digital library that they produced in connection with the local historical society to place historical photographs online.

I appreciated the definitions and preliminary explanations of digital libraries. I had not thought that the phrase 'digital library' really is a nebulous phrase. It can mean anything in any situation! What a given organization defines as a digital library might be entirely different from another digital library.

The other major idea that intrigued me was mentioned at the end of the Paepcke article, when the author discusses how computer scientists and librarians are disillusioned and disappointed with how the technology has not met their expectations in the past 10 years. They expected it to change the world, and it really hasn't done that.

Logically, however, we should be aware that things really aren't going to change that much when a new technology is introduced. The human race is not exactly known for embracing change quickly, and generally abides by the theory, "if it ain't broke, don't fix it". The library system has been this way for 5,000 years with only minor changes. Librarians and library users are not going to embrace a drastic revolution in the way libraries are structured. The old system must integrate the new technology into is extant architecture, not the other way around.

Gradual change is for the better anyways. When Vatican II changed the entire structure of the Catholic mass in 1962, the Catholic population (the users!) reacted poorly. Many people left the church because it wasn't what they were used to, and those who didn't were still disgruntled. To say the least, it didn't go well. We don't want to anger our patrons by changing everything around on them with no warning, introduction, or trial period. Then we will surely be out of jobs.