OCLC/ALISE Project – Design Help Features
In previous research, the research team identified blind users’ unique help-seeking situations in interacting with Digital Libraries (DL) based on the data collected from multiple methods of 30 blind users. The team identified corresponding help needs for 17 help-seeking situations as well as design principles for the help features to resolve the situations. This proposed project aims to continue this research by 1) developing design principles and designing help features to aid users to overcome the 5 critical help-situations; and 2) testing the usability of the new help features. The final products of this project include: 1) documents that illustrate the design principles for the design of help features to overcome the 5 critical help-seeking situations; 2) help modules of the corresponding new help features that can be implemented into DLs; 3) implementation protocols specifying how to implement help modules into real DLs; and 4) scholarly publications and presentations on the results of the usability study and recommendations for the iterative design of help mechanisms for blind users.
In order to set the foundation for this project, the team conducted research to identify 30 blind users’ unique help-seeking situations in interacting with DLs, implications and corresponding needs for different types of help features. Taxonomies of help-seeking situations were identified, based on the individual help seeking-situations and types of help needs revealed by the search processes of the research subjects. Based on the analysis of the first 15 subjects, Xie et al. (2015) published a paper identifying corresponding help needs for 17 help-seeking situations (see Types of Help Seeking Situations document for the complete list) as well as design implications for the help features to resolve the situations. The blind participants were asked to conduct three search tasks, including known item search, specific information search, and exploratory search, using Library of Congress Digital Collections. Multiple data collection methods were used to investigate blind users’ help uses and needs for help features including pre-questionnaires, pre-search interviews, think-aloud protocols, transaction logs, and post-search interviews.
Seventeen types of help-seeking situations were categorized into three types: 1) Help-seeking situations blind users share with sighted users; 2) Help-seeking situations unique to blind users at the physical level; and 3) Help-seeking situations unique to blind users at the cognitive level. Associated help needs for each type of help-seeking situation were also identified.
Building on the previous user study, the proposed project consists of four stages of work. The team will: (1) create design principles to overcome the 5 critical help-situations, which are applicable to different DL systems across platforms, and develop help modules based on the design principles, (2) create new help features by implementing the developed help modules in an operating DL, (3) test the new help features in a usability study, and (4) write the report and disseminate results. The project schedule starts at the beginning of February 2016 and concludes at the end of January 2017. Figure 1 presents the detailed schedule of completion.
Figure 1: Schedule of Completion.
Findings were analyzed to determine differences between the control group and the experimental group. In an overall comparison of frequency of nine help-seeking situations, key differences were observed between the two groups for confusion about multiple programs and structures and difficulty with help. There was a reduction in situations related to confusion about multiple programs and structure in the experimental group. The control group was more confused about the overall structure of pages, search results, and content layout. Help features that may have reduced these situations for the experimental group included: 1) an introductory task to visit the help page which outlined the overall structure, 2) individual descriptions about the page contents and structure, or 3) context-sensitive help with features. The experimental group experienced more situations related to difficulty with help. It is possible that the added instruction with the search limiters encouraged the experimental group to use the search limiters, but the feature itself was extremely difficult to use and navigate. For this situation, the experimental group also experienced difficulties recognizing the meaning of labels and features that were not modified from the original site.
Mann-Whitney U tests were conducted to compare the time spent for task completion between the control group and the experimental group. There is no significant difference in mean time between the two groups in completing all search tasks. This is likely because time limits were placed on each task. Actual differences in time may be better understood if time was not restricted.
We tested ease of use, perceived satisfaction and perceived helpfulness levels in the post-interview. Questions included a Likert scale range from 1-7, with 1 not at all easy/helpful/satisfied to 7 extremely easy/satisfied or helpful. Differences were not significant for perceive ease of use or perceived satisfaction. However, descriptive data showed that the experimental group found the DL to be easier to use and were more satisfied. Although differences are not significant, users reported what made them satisfied and dissatisfied with using the DL. The main reasons for high satisfaction level in the control group included coverage of the DL along with browse options and navigation. The experimental group reported high satisfaction due to the availability of multiple help features, clear headings, DL coverage, clear labels, and easy navigation. Main reasons for low satisfaction levels in the control group included inaccessible content or no alternative text, unclear labels/description, and a complicated structure, whereas main reasons for low satisfaction level in the experimental group included difficulty accessing information, no or irrelevant results, and multimedia problems. There is a significant difference between the controlled group (M=3.63, SD=1.18) and experimental group (M=4.61, SD=0.69) in the perceived helpfulness levels of system help; t(18)=-2.275, p=0.035. This indicates that the experimental group perceived a higher level of helpfulness of system help than the control group. Specifically, the experimental group reporting higher helpfulness levels for nearly all system features.
The experimental group was asked about the helpfulness of new features according to three categories: navigation, description, and search display options. Users found it easier to skip to content or results, browse, and appreciated the clear headings. Experimental subjects who encountered problems with using new features expressed time limitations or difficulty in understanding how to use features. Subjects who did not use the new features indicated that they were unclear about the features, they found them difficult to use (specifically search limiters), or experienced time limitations and would have explored them if given more time. Users in the experimental group were asked to identify their top features. Browse, Skip buttons or links, and headings were the top features preferred by users.
This study resulted in various implications and recommendations for future work in this area. It reinforced the need for researchers to rethink a sight-centered DL design. While there are several features that benefit sighted users, such as widgets with hyperlinks, or dynamic visual or timeline content, these features may serve as barriers to blind users who cannot use them in an intuitive or effective way. It is important that designers of DLs understand how the overall DL structure affects blind users and how their and navigation strategies may differ from sighted users. By keeping these points in mind, DL designers may create a DL that is easier for blind users to navigate, while keeping sighted users in mind as well. It is also important to be sensitive to the development of features, creating new features that blind users can effectively and intuitively use. Context-sensitive help and instructions may be beneficial, but may also cost blind users time in trying to understand how to use features. This leads to the next recommendation: designers must develop new help features without creating new help-seeking situations. Moreover, to ensure that adding new features doesn’t create more problems, new features must be tested across diverse DL designs. Lastly, there is a need to develop DL design principles and guidelines that ensure accessibility, usability, and utility of DLs for blind users.