Overview
Accessibility to information is important! But when vision degrades so does the ability to gather and synthesize information from online. SerpUI is free search engine tool that simplifies the Google search engine interface by minimizing the userspace and reducing the learning curve.
Try it out here
How does it work
The user has two controller options their keyboard or the Nintendo JoyCon controller connected via Bluetooth. The JoyCon controls are easy to learn and provide a comfortable browsing experience. The user starts by recording themselves describing what they would like to search. With the power of Google Cloud Services a real-time speech-to-text algorithm translates that to text where the user can verify their input. The background changes to a vibrant blue to appeal to users with strong visual impairment, a loud DING sound is also produced.
SerpUI also supports YouTube and will play the videos auto-magically!
The Minimized Userspace
Once submitted they are redirected to SerpUI (Search Engine Results Page User Interface). On the results page the content is identical to Google's results page just presented differently, high contrast, focused and with interactive audio feedback. Rows represent result ranking, and columns categorize the result's content (ie. Title, Snippet, Page Content, Link). At any given moment the user can only select one of seven instructions:
- Down Arrow => Next Search Result
- Up Arrow => Previous Search Result
- Right Arrow => Next Result Content
- Left Arrow => Previous Result Content
- SHIFT => Read Current Content
- CTRL => Stop Text to Speech
- ESC => Go To Homepage
This environment is surrounded with virtual borders that DING when the user hits them. This provides information on the location of the user.
Backend
A NodeJS server with Express. This is also where the Google Custom Search API was leveraged to attain accurate search results. The server is setup and maintained by me :)
Frontend
In order to integrate the JoyCon the Gamepad API is used. JQuery is used to handle server request, and finally the ResponsiveVoiceJS API powers the text to speech in the entire app.
Source
This project is the product of my Master's research at McMaster University (see more in my thesis under the Academia tab)
