IMAGE is a project by researchers in McGill University’s Shared Reality Lab, in collaboration with Gateway Navigation and the Canadian Council of the Blind (CCB), a consumer organization of Canadians who are blind, to build a web browser extension that lets you experience graphics on the web using audio and touch. Going beyond just a readout of things that are in a graphic, IMAGE conveys the locations of objects through stereo audio that fools your ears into thinking that sounds are coming from different directions and distances. IMAGE also works with two different touch devices, including the Dot, a pin array that is like a large braille display, and the Haply, which can push your whole hand in different directions on a tabletop.
Our web extension is currently for Google Chrome and compatible browsers, which we expect to include browsers like Firefox, Microsoft Edge, Brave, Chromium, and others. Currently, haptic devices are supported only on Chrome, but support for other browsers may happen in the future. These extensions do not currently work on mobile devices like tablets and phones.
Our main focus right now is on desktop and laptop browsers. However, Apple has announced that web extensions for their desktop, iPad, and iPhone browsers will all be compatible later this year. We are keeping an active eye on these developments, since we know the iPhone and iPad are important devices for the community. If there is enough demand, we may also look into solutions on Android.
IMAGE works fine with just stereo headphones, but yes, for the touch experience, additional hardware is necessary. The Haply is a relatively low-cost device, likely around a few hundred dollars from the manufacturer. This makes it potentially within reach of individuals. The Dot pin array is significantly more expensive, and would thus likely be purchased by a library or community centre, and used there.
We do not represent the companies we work with, but we would encourage you to contact them and let them know this would be of interest to you! http://haply.com and http://dotincorp.com
IMAGE works with graphics that you access through your web browser. However, since the IMAGE server will be open source, others could build additional ways of using the server-side smarts in ways, or in new products.
We have an earlier project, Autour (http://autour.mcgill.ca) that is focused on telling you what is around you while you’re walking around. IMAGE is instead focused on making graphics on the web more richly accessible, and is not designed for real-time mobility scenarios. That said, IMAGE will render graphical map content on websites in richer ways than just indicating that it is a map. In addition, since the IMAGE server will be open source, others could build additional ways of using IMAGE’s rich rendering architecture to address mobility.
IMAGE does not replace any of the tools you currently use, including braille devices, screen magnifiers, and screen readers. Since IMAGE is a web browser plugin, it should work fine alongside (and hopefully complement) any such tools. Since we have limited testing resources, we’ll be accepting bug reports once IMAGE is publicly available, in the event of any issues.
Please watch our website (http://image.a11y.mcgill.ca) for information on upcoming beta releases and opportunities to participate. Right now, we are running a survey to help guide the direction of the project. In addition, if you are a resident of Canada, we are also recruiting ongoing participants in the design and testing process. To participate, please make sure to fill out the survey for Canadian residents, since it gathers the additional information we need to have in order to follow up for ongoing participation. If you are a software developer, we’d love to talk to you about participating, and we’d also be delighted to have you try out our extensibility system for new graphics types and new types of renderings!
We are fortunate to be partnered with Gateway Navigation CCC Ltd and the Canadian Council of the Blind (CCB), and through them, connected with a large number of blind, deaf-blind, and low-vision individuals. We would welcome having a blind student join the project, but at this time, there are no blind McGill students or faculty working on IMAGE. In a past project, called “Autour”, the SRL hosted a blind student from another institution who participated in design and evaluation as part of their degree requirements. If you know of any candidates, whether for design, technical implementation, or other roles, please point them in our direction, as we would welcome the help!
IMAGE is funded by Innovation, Science and Economic Development Canada, via the Accessible Technology Program (https://www.ic.gc.ca/eic/site/118.nsf/eng/h_00000.html).
The IMAGE browser extension will be free to install and use. McGill will run a server that supports the extension, and produces rich renderings of web graphics. In addition, our software will be open sourced, so that other developers can use the code, or run their own servers as well. Some of these servers may use commercial machine learning from companies like Microsoft and Google that have a cost associated with them. It is up to the entities running these additional servers whether to charge for their use, and to set any pricing.
We are partnered with Gateway Navigation CCC Ltd and the Canadian Council of the Blind (CCB). Of course, we are happy to work with any organization that shares the IMAGE project’s goals. Feel free to contact us if you would like to join the effort!
A complete description of how we handle your data can be found at https://image.a11y.mcgill.ca/pages/policies.html. If this brief description differs, the terms at that link take precedence. When you click “Get IMAGE Rendering” on a graphic, that graphic itself, plus information from your settings and the webpage where the graphic is found, are sent to a server for analysis. The graphic itself gets analyzed using several web services including Microsoft Azure. The graphic, along with the data generated from it, is kept on our server for a brief time (typically less than two hours) before it is deleted, unless you provide feedback or request support using the link in the pop-up window. In this case, and only with your explicit permission, we retrieve the graphic and the data so that we can provide you with better quality support. Since someone from the IMAGE project team will be looking at this information, please do not send graphics that may contain personal information; your privacy is important to us!