Wikidata and BHL Update: Part 1

This is a fairly incomplete post about the work that’s going on regarding adding BHL bibliography metadata to Wikidata. I hope to have several more of these posts before the end of the year! 

Following some productive conversations on donating BHL bibliographic metadata to Wikidata, it was discovered almost immediately that BHL’s data is not terribly useful without some serious munging. One of the biggest problems with BHL bibliographic metadata is that it comes from lots of different libraries and museums, legacy cataloging systems, and various types of authority work. For example: BHL attaches Creator IDs to Author names, which is useful for identification and connecting titles and items to their Authors, but they are assigned automatically according to the character strings imported from specific fields in a library catalog’s MARC record. Despite (and perhaps because of) the use of varying authority files to control Author name strings in institutional catalog records, different libraries have contributed items by the same author whose names are are spelled, punctuated, and identified differently. BHL does not conduct authority control on BHL metadata, choosing instead to focus on improving access to items based on content rather than metadata. Fortunately, there are several different ways to go about reconciling and disambiguating data, and one of them is crowdsourcing.

BHL can use Wikidata to tell its users that “Packard, Alpheus S” (Creator ID: 82636), “Packard, A” (Creator ID: 59850), “Packard, A S” (Creator ID: 48286), “Packard, A. S. (Alpheus Spring), 1839-1905” (Creator ID: 1592), and “Packard, Alpheus Spring” (Creator ID: 56087) are all the same person without editing the spelling or legacy metadata from the catalog record.

Screen Shot 2017-08-17 at 4.24.11 PM

Dr. Packard’s Wikidata Item viewed in Reasonator

One way is to use Wikidata as an identifier by adding a property for a BHL Creator ID in Wikidata (P4081) and adding a table in BHL for Wikidata Identifiers that can be associated with those same Creator IDs. By adding identifiers to Wikidata, it becomes a more robust knowledge base that will improve the discoverability of BHL’s content by enriching its metadata externally and solving some metadata problems internally. While some of the reconciling can be done computationally using (still more) authority files, it often misidentifies strings and isn’t very helpful when an author is not in that particular database. These errors are best caught by humans, who WIkidata invites to directly edit mistakes and add identifiers. By adding Creator IDs to Wikidata and in turn adding Wikidata IDs to BHL, BHL can leverage the wisdom of the crowd to reconcile its author metadata.

In order to test this idea and attempt to start down a path that will hopefully lead to more BHL data in Wikidata, I worked with Andy Mabbett (User:PigsOnTheWing) to add a representative set of 1000 BHL CreatorIDs to Wikidata; the first step of which was to disambiguate these authors. In order to procure a sample of 1000 representative authors, I used the rbhl R package to interface with the BHL API and pull a random sample of authors with associated DOIs.1 The rbhl package is an rOpenSci tool and can be found on their GitHub. The R script I used can also be found on GitHub at: https://github.com/kmika11/BHL_Wikidata/blob/master/CreatorID/rbhl_CreatorIDScript.R . Once I was able to generate a table of Author Strings, CreatorIDs, an associated Title, and its DOI I headed over to OpenRefine to start reconciling BHL CreatorIDs. As you’ll remember from a few paragraphs ago, BHL doesn’t conduct authority control and relies instead on the work of partner institutions. This means that there are no external identifiers for authors in BHL. We chose to reconcile against VIAF IDs because VIAF has the most identifiers in Wikidata (for library resources at least). Once there were VIAF IDs, the CreatorIDs could be added as a P4081 property statement to author QIDs. The tool Mix n’ Match makes the part of this process that requires some human thought pretty simple and somewhat fun!2  

Now, my next steps are figuring out what that next steps are. There is some interest to add New York Botanical Garden’s herbaria type specimen to Wikidata along with protologue literature from BHL and perhaps field notebooks and other relevant collecting event items. BHL also has quite a long list of taxon names (3,732,986 names) with metadata for the pages they’ve come from. I don’t think it’s appropriate to push all of this data to Wikidata, but it is a significant dataset that could be useful in varying ways. Another issue is that resolving author strings to VIAF IDs is not an insignificant amount of work. Gerard Meijssen has brought up the idea of using Open Library IDs, which are already resolved to VIAF and often Wikidata, and which may be a solution. BHL hosts its content on the Internet Archive, which is the creator of the Open Library. One would imagine that is a simple hop, skip, and a jump from BHL CreatorIDs to OpenLibrary IDs, but I’m still investigating whether that is, in fact, the case.

Please jump in with any thoughts about Wikidata + BHL or what I’ve described above. I know that WordPress is not terribly conducive to discussion, but that’s how we’re set up for now. I do not claim to have an expert level grasp of Wikidata yet (or BHL for that matter), but this collaboration seems to be a constructive Open Data pursuit!

1. During this step I incorrectly assumed that BHL minted DOIs for all its content including individual articles. BHL does mint DOIs for monographs, and worked with BioStor to add 12,000 DOIs for articles.

2. The manual for using Mix n’ Match can be found at: https://meta.wikimedia.org/wiki/Mix%27n%27match/Manual

Workshop Roundup

In the past month, I’ve been able to participate in several workshops relevant to my project and to BHL in general. On February 15th I attended a training session on the Expanding Access to Biodiversity Literature (EABL) project presented by Mariah Lewis of the New York Botanical Garden. The workshop was held at the Los Angeles County Arboretum and Botanic Garden, one of BHL’s newest affiliates.

IMG_20170215_093337953_HDR

Arboretum grounds

Mariah gave an overview of the EABL project and of BHL’s collection development policy, digitization overview, and copyrights and permissions workflows. This was a great refresher from the BHL Bootcamp at the Smithsonian Libraries, and it was good to confirm that I have a solid grasp on how BHL works. Continue reading

Hello World!

Welcome to the NDSR at BHL blog!

Over the next 11 months we will be collaborating as National Digital Stewardship Residents on several projects to develop recommendations and best practices for enhancing tools, curation, and content stewardship for the Biodiversity Heritage Library. As recent graduates of Master’s programs in Library and Information Science, we are excited to contribute to the field of digital stewardship through our work on the Biodiversity Heritage Library and develop leadership skills through the Residency model.

Alicia Esquivel is the Resident at Chicago Botanic Gardens where she is working on completing a content analysis of the quantity of literature in the field of biodiversity, the amount of that literature in the public domain, the representation of each discipline within BHL and an exploration of the methodologies to scope the collections and areas where BHL may target development to better serve the research population.

Marissa Kings is the Resident at the Natural History Museum, Los Angeles County, where she is focusing on identifying high value tools and services used by large-scale digital libraries which might be applied to the next generation of BHL. She will also be exploring digitization workflows at NHMLAC and identifying items to be contributed to BHL.

Pamela McClanahan is the resident at Smithsonian Libraries where she will be conducting a user needs and usability analysis working with the larger taxonomic and biodiversity informatics community to determine user needs and services for providing increased value to BHL content. Pam will analyze this information and input to define recommendations and requirements for expanding the BHL digital library functionality.

Katie Mika, Resident at the Harvard University Museum of Comparative Zoology’s Ernst Mayr Library, is developing tools and methodologies for crowdsourcing full-text transcriptions and structured data from BHL’s manuscript items, including field notebooks, specimen collection records, correspondence, and diaries. Katie’s background is in Archives Management and building digital repositories to support description and access to digitized and born digital photograph, multimedia, and software collections.

Ariadne Rehbein is the Resident at the Missouri Botanical Garden, where she is focusing on natural history illustrations sourced from digitized biodiversity literature. Building upon the successful work of the “Art of Life” team members and citizen scientists, her project will incorporate user research and knowledge of digital scholarship to produce user interface requirements and a report on image discovery best practices.

As a cohort, we residents are collectively tasked with proposing options for substantial improvement to version 2 of BHL on the understanding that the underlying data structures and metadata schemas will be somewhat, if not completely, rebuilt. We therefore have quite bit of latitude to introduce cutting edge technology and incorporate various “wish list” features that BHL staff have collected over several months.

This blog will function as a dynamic record of our work with BHL and the NDSR program through December 2017. You can expect to read posts about our projects’ successes, challenges, and probably some failures in the next several months as well as some interesting discussions about biodiversity librarianship and content and data management in digital libraries. Occasionally we’ll also be posting about attending and presenting at professional conferences, participating at workshops, and engaging in other activities within the wider digital libraries community.

We also hope that this blog will serve as a tool to facilitate communication with other librarians and archivists and anyone interested in the future of BHL. To learn more about BHL or the NDSR program head over to the About page, which includes an overview of the IMLS supported “Foundations to Actions” grant that is funding each of our Residencies and the mission of the Biodiversity Heritage Library as well as some useful links.