A few months ago, I was invited to have a talk at the European Computer Science Symposium on an aspect of my research I particularly care about, that of open citations. What I tried to address during the presentation concerned the current status of open citation availability in a particular domain, Informatics, by using two open datasets, i.e. DBLP for gathering bibliographic metadata about relevant publications and OpenCitations’ COCI for identifying citations where such publications are involved. This post briefly introduces the preliminaries and results obtained from the material used to prepare the talk.
Open citations and where to find them
A citation is a conceptual directional link between a citing entity and a cited entity which is defined by means of specific textual devices contained in the text of the citing entity, e.g. a bibliographic reference denoted by an in-text reference pointer (e.g. “” or “(Doe et al, 2021)”). While reasons for citing may vary, citations are used in academia for acknowledging others’ work and enabling building trails of relations defining how science evolves in time.
The data needed to describe a citation should include, at least, a representation of such a conceptual link and the basic bibliographic metadata to identify the citing and cited entities, i.e. those typically used for defining bibliographic references such as authors’ names, year of publication, the title of the work, venue of publication, pages, identifiers, etc. We say that a citation is open when these citation data are in the public domain and can be retrieved freely (via the HTTP protocol) in a structured and machine-readable format (e.g. JSON or RDF) without accessing the source of citing article defining it, which, potentially, could be behind a paywall.
OpenCitations [full disclosure: I am one of its directors] is one of the founders of the Initiative for Open Citations (I4OC) and one of the open scholarly infrastructures providing open citation data through several channels (REST APIs, SPARQL endpoints, Web interfaces, full dumps in different formats). As of 31 December 2021, it makes available more than 1.2 billion open DOI-to-DOI citation links between more than 69.5 million bibliographic resources, which are mainly journal articles but also include books, book chapters, datasets, and other DOI-identified resources. The entities involved in such citations come from different domains, spanning from Medicine articles to Humanities publications, and have recently approached parity with those included in well-known proprietary services such as Web of Science and Scopus.
What about Informatics
Such a huge mass of open citations available enables us to analyse citation coverage in different scholarly disciplines, e.g. to understand which publishers contributed to the availability of open citation data in a discipline and to check what are the citation trails between different disciplines. However, to compute such citation coverage, we need to have some information that allows us to identify when a particular bibliographic resource involved in a citation belongs to the particular discipline we want to analyse. We can use information about the subject categories of publications (e.g. that of Web of Science), if included in citation indexes, to identify the discipline(s) of a given bibliographic resource. Unluckily, OpenCitations does not provide this information and, as such, we need to rely on external repositories for gathering subject categories of publications, e.g. collections of bibliographic metadata of disciplinary publications.
In the context of Informatics, there is at least one well-known resource gathering and exposing bibliographic metadata of a large part of Computer Science publications, i.e. DBLP. As of 30 December 2021, DBLP contains more than 5.9 million publications published in 1,781 journals and in the proceedings of 5,621 conferences, involving more than 2.9 million authors that are manually curated (and disambiguated) by the DBLP team.
DBLP can be used as a proxy to understand if a particular publication belongs to the Computer Science subject category. Through it, it is possible to understand how many citations in OpenCitations involve Computer Science publications by comparing the DOIs of citing and cited entities with those available in DBLP. In particular, using the OpenCitations’ COCI September 2021 dump and DBLP October dump, I found that more than 80 million citations in COCI involved at least one of the 4,637,865 entities in DBLP (considering only journal articles, conference proceedings papers, books and book chapters). As shown in Figure 1, only 39% of these citations are between citing and cited entities both included in DBLP, while the rest of them either come from or go to publications not listed in DBLP – that, potentially, could not be Computer Science publications.
Additional information about the publishers of such DBLP entities, retrieved by querying the Crossref API and the DataCite API with entities’ DOIs, are shown in Table 1. IEEE is the publisher with the biggest number of entities of those considered for this study, and its entities are involved in more than 18.9 million incoming and 21.5 million outgoing citations. The other bigger publishers, in terms of entities and citations, are Springer, Elsevier, ACM and Wiley. It is worth mentioning that the two publishers responsible for publishing mainly Computer Science journals and a relatively low number of conference proceedings (if any), i.e. Elsevier and Wiley, are those providing the highest number of openly-available references per publication (on average, around 29 and 37 cited works for each publication respectively).
|Publisher||DBLP entities||COCI incoming citations||COCI outgoing citations|
Of course, this study does not provide full coverage of open citations in Computer Science but just a preliminary insight. First, as anticipated below, DBLP does not have the complete coverage of all CS-related publications since there are some venues that are not listed there (yet). Thus, some relevant open citations could not be extracted from COCI if these involve as citing and cited entity non-DBLP publications that belong to the CS domain. However, it is worth mentioning that no bibliographic and citation database (including commercial and proprietary ones) has a full disciplinary coverage anyway and DBLP is, probably, the most comprehensive collection of Computer Science publications metadata (something that could be assessed in future analysis).
Along the same lines, the index of open citations used, i.e. COCI, does not contain all the citations defined in CS publications, but only DOI-to-DOI citations as retrievable from Crossref data. Although Crossref is the biggest DOI provider and it is used by the majority of the big publishers,citations defined in publications with a non-Crossref DOI (e.g. DataCite) and those not having any DOI assigned (e.g. the papers published in CEUR Workshop Proceedings) are not included in COCI and, consequently, have not been used in the analysis. However, OpenCitations plans to extend its data coverage adding more sources in the next years. Thus, it would be interesting to replicate the same analysis in the future to see if and how much the coverage increase, at least in the context of Computer Science publications.
Still about coverage, currently (i.e. 31 December 2021) the only publisher of those included in Table 1 which is not providing open references through Crossref is IEEE. Indeed, while COCI includes several citations involving IEEE publications as citing entities, there is no availability of such citation after October 2018, when IEEE decided not to allow anymore Crossref Metadata Plus users to access these reference data.
Finally, analysing the preliminary results of this study, it would be interesting to understand which are the main subject categories of non-DBLP publications included in the 61% of citations shown in Figure 1 (e.g. by using the Scimago Journal and Country Rank database to retrieve their subject categories) to understand what are the citation dynamics between Informatics and other disciplines. However, I will leave the answer to this question to future analysis.
A final remark on reproducibility
Since several of the suggestions provided above start from the idea of either replicating or extending this study with additional materials and insights, it is important that all data and software used to perform the analysis are available online to permit its reproducibility. To this end, I have published both the software and the data retrieved online with open licenses to enable anyone to reuse it freely for any purpose.