An Experiment of the Feasibility of Providing Large Scale Professional Digital Library Services to Clinical Users

Ana D Cleveland, Ph.D., University of North Texas
Steven L MacCall, Ph.D., University of Alabama
Irvine D Prather, D.O., University of North Texas HSC
Jerry T McKnight, M.D., University of Alabama
Ian E Gibson, M.S., University of Alabama
Larry Johnson, L.S.W., University of North Texas HSC
Jay Morrow, D.V.M., University of North Texas HSC

Introduction

This paper describes the Clinical Digital Libraries Project (CDLP) and its efforts to create and evaluate a working model for health sciences libraries' provision of large-scale professional digital library services to clinical users. The model, which is based on a classical digital libraries model (CDLM), enables the provision of focussed repositories of links that are deliverable over networks to clinical environments (MacCall, Cleveland & Gibson, 1999). The central goal of the CDLP is that health sciences libraries work together in this effort by coordinating core collections in order to enable collection development efforts to concentrate on value-adding to the common core. The CDLP has created a core collection of links that combine quality web resources and subscription resources that when taken together provide a consistent interface to varied networked resources for clinical users. The result is that the CDLP core collection contains resources of multiple publishers, such as MD Consult, Harrison's Online and the Merck Manual, integrated into a series of digital libraries, which can fit a "just in time" model for delivering information to clinical environments (Chueh & Barnett, 1997).

For purposes of this paper, a clinical digital library is defined as a collection of collections of electronic knowledge resources developed and maintained in order to meet the totality of information needs for a given clinical user population. By large-scale professional digital library services, we mean those services that are provided by health information professionals to networked users. Large scale is meant to imply that the number of items (links) in a digital library is the number and breadth that is associated with professionally managed physical libraries. Large scale is also meant to imply that these services are provided in a distributed way, i.e., by local professionals for local clinical users. Feasibility is measured from the perspective of the enduser. Is a large-scale digital library useful within the context of clinical practice? Though precise feasibility measures are not currently available, we estimate whether clinical endusers are successfully using the clinical digital libraries of the CDLP through web server log file analysis and user survey data, as described below.

The Clinical Digital Libraries Project

The Clinical Digital Libraries Project (CDLP) was established in 1995 with the goal of delivering clinical information at the point of care for family practice residents. A partnership was developed between the Department of Family Medicine at the University of North Texas Health Science Center at Fort Worth and the Medical Informatics Program at the School of Library and Information Sciences at the University of North Texas. In 1997, the CDLP was expanded to the School of Library and Information Studies and the College of Community Health Sciences at the University of Alabama. The research, teaching and service initiative are currently led by both Schools of Library and Information Sciences.

From the beginning the design of the CDLP was driven by the urgency to deliver reliable, quality information for healthcare providers in a timely manner. It was understood that the CDLP would develop clinical digital libraries (CDLs) that would provide an environment that was conducive to enduser browsing for resources relevant to their information needs.

There are two categories of CDLs based on their content. WWW CDLs only contain resources that are freely available on the WWW. Intranets CDLs also contain subscription-based resources that are licensed by participating CDLP libraries. Our role has been to organize these resources and to integrate them with WWW resources for easier combined access. Each library provides their clientele with appropriate controlled access to their CDL's resources. Access control can be managed by station restriction based on IP domain and/or by user passwords.

The Classical Digital Library Model

The CDLM derived from traditional practices of library and information science (LIS) professionals, which include the expert critical selection and organization of the networked information resources for local populations of users and the integration of advanced information retrieval tools, such as databases and Internet search engines, into these networked collections (MacCall, Cleveland & Gibson, 1999). The strength of the CDLM is that it enables LIS professionals to extend their practices to the digital library environment where networked delivery of digital content does not require library users to be physically present in a library.

Under the CDLM, users do not have to browse, navigate or (most importantly) search large and diverse information spaces without the assistance of LIS professionals who are able to pre-position selected quality resources and organize them with their needs in mind. The key point is that the information resource space, i.e., the digital library, has content that is suited to the needs of users as determined by LIS professionals with specific local knowledge concerning their users. The CDLM is intended to capture the extension of professional services to the desktop of users through the analysis of the working environment within which those desktops are located. This implies that, in theory, a library could manage as many digital libraries as it has identifiable user populations.

The problem of managing a large number of digital libraries is addressed by important aspects of professional LIS practice: the use of core collections and selection aids. Ready access to suitable core collections prevents the need for an LIS professional to build new digital libraries from scratch.  Selection efforts can thus be concentrated at the margin of collections where development activity can be optimized according to the needs of local users. As for selection aids, LIS professionals should use each other’s digital libraries, as well as other published sources, as aids to their own collection building. The CDLM suggests that both approaches, the use of core collections and the use of collection aids, should be invoked to extend the collaborative collection development practices of physical libraries to the digital library environment.

The CDLM as a Foundation for the CDLP

The CDLM is based on the premise that digital libraries can be founded on basic principles of information management. These principles include collection development and organization of networked information resources that are delivered to the point of need. The extension of the CDLM to the CDLP posits that, for example, an academic health sciences library could provide separate digital libraries to the desktops of biomedical researchers and to clinicians. Even further, individual digital libraries can be customized for different clinical environments and user preferences.

The CDLP has accumulated a large-scale collection of some 3,000 web pages with approximately 75,000 links to and within networked resources (example of the web page for Brain Neoplasm: WWW CDL and Intranet CDL). Deep linking occurs to provide immediate access to relevant parts of networked resources rather than simply linking to the homepage. The core collection was assembled using criteria developed by the CDLP management team, including clinicians, nurses, and library and information science faculty and students. Information seeking behavior of clinical personnel was taken into consideration in the development of a detailed clinically-oriented scheme for organizing resources for a wide range of medical topics.

In addition to developing the core collection, the CDLP management team has also worked with health sciences libraries to develop and evaluate a means for presenting the core in the context of each library. The CDLP homepage contains a list of participating libraries and their respective clinical digital libraries. By creating digital libraries for specific clinical environments, the CDLP model allows for the selection of quality resources that are relevant in specific contexts as determined by the demographics and predicted needs, or individual preferences, of clinical staff.

Evaluation of Feasibility

The feasibility of providing large-scale clinical digital library services is evaluated in terms of CDLP enduser usage patterns. Our premise is that patterns of sustained usage indicate a satisfied enduser population. Because digital library evaluation methodologies are not well established, caution is observed when drawing any conclusions from the data.

Usage of CDLP digital libraries was evaluated in two ways. The first was an analysis of CDLP web log files and the second was an analysis of user surveys.

Log File Analysis

All CDLP activity was logged over a three month period (January 2000 to March 2000) and Wusage 6.0 software was used to analyze the log file data. Figures 1 through 3 show user activity on an hourly basis during this period.

Figure 1: CDLP Log of Web Pages Served during January 2000 by Hour (U.S. CST)

Figure 2: CDLP Log of Web Pages Served during February 2000 by Hour (U.S. CST)

Figure 3: CDLP Log of Web Pages Served during March 2000 by Hour (U.S. CST)

Figures 1 through 3 indicate a consistent pattern of use during normal working hours in the U.S. (between 7 a.m. and 5 p.m. U.S. CST). Since a large percentage of users are U.S.-based, we therefore infer repeat use during clinical hours.

To further test this inference, we examined day-to-day data to test whether usage occured primarily during the workweek. Figures 4 through 6 show user activity on a daily basis for each of the first three months of 2000.

Figure 4: CDLP Log of Web Pages Served during January 2000 per Day

Figure 5: CDLP Log of Web Pages Served during February 2000 per Day

Figure 6: CDLP Log of Web Pages Served during March 2000 per Day

Examination of data on a per day basis indicates a pattern of lessened activity during weekend periods over a three month period, which gives credence to our inference that users were visiting their clinical digital libraries during the workweek.

Another way that the CDLP web server log file was analyzed was to examine the trails used to navigate through individual clinical digital libraries. We were particularly interested in analyzing those trails taken by clinical users that consistently arrived at digital library web pages containing links to resources external to their clinical digital library. Patterns detected over time can lead to an inference that clinicians were successfully locating resources within a professionally managed large-scale clinical digital library.

To evaluate user trails through clinical digital libraries, we counted trails by known clinical users that ended on web pages containing links to external resources. The trails consisted of a sequence of links followed by users when traversing the clinical ontology leading to digital library web pages containing links to external resources. For this analysis, the clinical digital libraries of two health science centers were examined.

Table 1 shows the number of times that endusers traversed trails that ended at specific destinations. These destinations are taken from the top levels of the general ontology developed for the CDLP. For example, the Patient/Family destination is counted any time an enduser ended on a web page containing links to networked resources under the Topic of Patient/Family Resources.

Please note that the Clinical category is subdivided into DDx (differential diagnosis), MEDLINE, and Practice Guidelines, which is a jump that we provide from our top level directly to the second level of our ontology. This is for the convenience of our endusers and is based on our observation of common types of clinical information needs. There are other clinical needs other than that of DDx, MEDLINE, and Guidelines, so the total number of Clinical trails is more than the sum of each.

Table 1: Number of Trails Begun by Known Clinical Users that Ended on Web Pages containing Links to External Resources
 

NOTE: HSC=Health Science Center.
 

Table 1 shows that aggregate usage was fairly consistent by trail destination across the two sites.  This data suggests that users were finding that their CDLs provided a feasible means for locating networked resources within the context of clinical practice.

Survey Results

To further examine feasibility issues, a survey was made available to clinical digital library users over a 10 day period. The majority of the respondents were physicians (11 of 18). Analysis of the physician respondents showed that 82% were males in clinical practice for eleven years or longer, and 73% have used their clinical digital library for over one year.

We were particularly interested in identifying the specific environments where CDLs were utilized and for what purpose. Table 2 indicates that the office, clinic and home were the locations with the highest reported use. Table 3 indicates that patient care was the highest ranked purpose for CDL use by both physicians and non-physicians. Table 4 indicates that the type of questions that were most frequently posed by both groups were related to treatment planning, diagnosis and patient education.
 

Table 2: Number of respondents reporting location of clinical digital library use

Note: Total responses=18; physician responses=11.

Table 3: Number of respondents reporting purpose of clinical digital library use

Note: Total responses=18; physician responses=11.

Table 4: Number of respondents reporting types of questions addressed using clinical digital library use

Note: Total responses=18; physician responses=11.
 

We were also interested in discovering which top-level CDL categories were found to be most useful by CDLP users. Table 5 indicates that both physician and non-physician respondents favored the following CDLP top-level categories: MEDLINE, Clinical Resources and E-Journals. In addition, physicians also favored the Internet Searching and Drug Reference top-level categories.

Table 5: Reported use and non-use of clinical digital libraries by top-level category

* Physician reponses=11; Non-physician responses=7; NR = no response.

Table 6 shows that most respondents (91% of physicians and 94% of all respondents) found that the information resources linked to from their clinical digital library were reliable and dependable. In addition, the information resources were perceived as high quality by an equally large percentage of respondents (89% of physicians and 91% of all respondents). Available educational and background reading resources were found to be more useful than clinical decision-making resources (91% of physicians and 94% of all respondents versus 82% of physicians and 83% of all respondents). As expected, there is room for improvement with the interface for the clinical digital libraries. Overall, 73% of physicians and 83% of all respondents reported that it was easy to locate information using their clinical digital library, and 64% of physicians and 72% of all respondents reported that their clinical digital library use reduced the time it took to retrieve relevant information when compared to general Internet use.
 

Table 6:

Reported Satisfaction with CDL Use

* NR = no response.

Summary and Conclusion

The objective of this research project was to ascertain the feasibility of providing large- scale professional digital library services to clinical users. Two evaluation tactics were employed: Log-trails of user accesses and a survey of users. The log-trails indicated a pattern of logically sequenced actions, indicative of successful searches leading to needed resource sites. The survey indicated that users found information resources of quality, and that the CDL is reliable, dependable, and useful.

Due to the size in the number of the responses, it is obvious that more comprehensive evaluation needs to be done. The positive results from these evaluation tactics suggest that it is feasible to provide services to clinical users.

Bibliography

Chueh H, Barnett GO. (1997). "Just-in-time" clinical information. Academic Medicine 72:512-7. [PubMed MEDLINE]

MacCall SL, Cleveland AD, Gibson IE (1999). Outline and preliminary evaluation of the Classical Digital Library Model.” Proceedings of the American Society for Information Science 1999 Annual Meeting, pp. 33-46.

Page last edited: 23/06/00