Definition
A situation that arises from difficulty utilizing screen readers (text with tooltip) A software program that reads textual information through synthesized speech and offers specialized keyboard commands to operate a computer interface. and voice activated commands (text with tooltip) A feature that allows users to control devices, applications, or systems by speaking instead of using manual input methods. within DLs (text with tooltip) The acronym for digital library (DL) due to incompatibility of DLs with these assistive technologies.
Factors Leading to the Situation
- Incompatibility: incompatible with assistive technologies
Guidelines
- Ensure that the design of DLs is compatible with assistive technologies.
Rationale for Suggesting the above Guidelines
To interact with DLs in the mobile environment, BVI (text with tooltip) The acronym for Blind and Visually Impaired. It refers to BVI users who rely on screen readers to interact with digital libraries (DLs). users have to use screen readers (text with tooltip) A software program that reads textual information through synthesized speech and offers specialized keyboard commands to operate a computer interface. and other types of assistive technologies via their mobile devices. Meanwhile, voice activated commands (text with tooltip) A feature that allows users to control devices, applications, or systems by speaking instead of using manual input methods. can make it easier for BVI users to access and use DLs without having to touch the screen to navigate DLs. However, it will be frustrating for BVI users when screen readers and voice activated commands are incompatible with DLs in the mobile environment. Therefore, the suggested guidelines ensure the compatible voice activated commands and relevant AI technologies are integrated to enhance BVI users’ experiences using DLs.
Techniques and Methods to Comply with a Specific Design Guideline
1.1. Integrate
voice activated commands
(text with tooltip)
A feature that allows users to control devices, applications, or systems by speaking instead of using manual input methods.
to enhance users’ search experiences.
1.2. Integrate AI technologies to enhance users’ search experiences.
Features Suggested for Users
1.1.1. Voice Control / Voice Access
1.2.1. Siri / Apple AI / Google Assistant / Google Gemini
Examples of Best Practice
1.1.1.a1. Voice Control: Open a search feature with “Tap search” command
In USE1 Figure a1, users can activate a search feature in a DL by saying, “Tap or double tap search” command.
USE1 Figure a1. An example of using voice command with Voice Control
1.1.1.a2. Voice Access commands
These commands are examples of voice control interactions that enhance accessibility by enabling hands-free navigation and operation of web pages or applications.
1.1.1.a2.1. Create a search
The command “search” allows users to activate the search function, input search terms using their voice, and execute the search without requiring manual input. In the USE1 Figure a2, Voice Access is utilized to perform a search by saying, “search coronavirus.” The search term is automatically populated in the search box (text with tooltip) A design element that allows users to enter search terms into a blank field. and processed to generate the results.
USE1 Figure a2. An example of creating a search using voice access
1.1.1.a2.2. Navigate within a page
“Swipe down/right”, “scroll down/up” and “move up/down” commands allow users to move between different sections or elements on the page (e.g., headings, links, or form fields) by issuing voice commands.
1.1.1a2.3. Go back previous page
“Go Back” command allows users to return to the previously viewed page using a simple voice command, equivalent to pressing the browser’s back button.
1.2.1.a1. Siri: ChatGPT integration
Siri can tap into ChatGPT to help users with certain requests when it determines ChatGPT can provide a helpful answer. In USE1 Figure a3, when a user requests to open the Library of Congress, ChatGPT provides a link along with brief information about the Digital Library.
USE1 Figure a3. An example of ChatGPT integration with Siri
1.2.1.a2. Google Gemini: Applicable command “Ask about this screen”
Users can ask questions through choosing “ask about this screen” to get the specific information within the page. In USE1 Figure a4, Google Gemini will automatically generate the screenshot of the page once users use “ask about this screen” and then Google Gemini will analyze the generated screenshot.
USE1 Figure a4. An example of Google Gemini command “Ask about this screen”
1.2.1.a2.1. Google Gemini: Applicable command “List all the headings on this webpage”
In USE1 Figure a5, Google Gemini will list all the headings on specific webpage when users ask it to “list all headings on this webpage”.
USE1 Figure a5. An example of Google Gemini command “List all the headings on this webpage”
1.2.1.a2.2. Google Gemini: Applicable command “Summarize this webpage”
In USE1 Figure a6, Google Gemini can summarize the page when users ask it to “summarize this webpage”.
USE1 Figure a6. An example of Google Gemini command “Summarize this webpage”
1.2.1.a2.3. Google Gemini: Applicable command “What are the key sections of this site?”
In USE1 Figure a7, Google Gemini can explain what those sections are about when users would like to know the key sections of the page.
USE1 Figure a7. An example of Google Gemini command “What are the key sections of this site?”
1.2.1.a2.4. Google Gemini: Applicable command “Where is the “Contact us” link?” or “Where can I find the search field?”
In USE1 Figure a8 and USE1 Figure a9, Google Gemini can execute the command for users about the location of link and specific features.
USE1 Figure a8. An example of Google Gemini command “Where is the “Contact us” link?”
USE1 Figure a9. An example of Google Gemini command “Where can I find the search field?”
1.2.1.a2.5. Google Gemini: Applicable command “List all the buttons available.”
In USE1 Figure a10, Google Gemini can list all the buttons within the webpage when users would like to identify quickly what those buttons are.
USE1 Figure a10. An example of Google Gemini command “List all the buttons available”
1.2.1.a2.6. Google Gemini: Applicable command “Describe the images on this page”
In USE1 Figure a11, Google Gemini can analyze the image of the collection item and provide a description of its contents.
USE1 Figure a11. An example of Google Gemini command “Describe the images on this page”
Examples of Poor Practice
1.1.1.b1. Voice Control: Interference with Siri
Siri processed comments/commands from VoiceOver and Voice Control. In USE1 Figure b1, when both Siri and Voice Control are enabled, Siri override the command “Tap advanced search” and displayed the search result instead of interacting with the advanced search.
USE1 Figure b1. An example of Siri takes Voice Control commands “Tap advanced search”
1.1.1.b2. Voice Control: difficulty closing a pop-up
In USE1 Figure b2, when users have difficulty closing a pop-up window, the system suggests, “Go Back.” However, when users say “Go Back,” it takes them to the previous search engine page.
USE1 Figure b2. An example of a confusing suggestion for Voice Control
Resources
- Apple. (2024). Apple Intelligence.
- Apple. (2024). New features available with iOS 18.
- Apple. (2024). Use Voice Control commands to interact with iPhone.
- Apple. (2024). Use Vocal Shortcuts on iPhone.