Project
AI prototyping as a research tool
company
Xero
Category
[Accessibility]
[Design]
Year
2026
A big part of what we do in the accessibility team is conducting research with people who use assistive technologies. We sometimes get questions we can’t answer directly, as we don't have the lived experience of being an assistive technology user. Taking these questions to Fable, where we can test with professional assistive technology users, is the best way for us to provide our design team with the right direction and feedback. A challenge we have been trying to tackle for a while is how to communicate our findings back to product teams when most of it relates to interactions, navigation and assistive technology commands. It relies quite a bit on understanding how different assistive technologies work, and this can be hard when all we have are Figma prototypes. Explaining tab order, focus states, and how assistive technology users perceive a page requires a level of interactivity that traditional design tools don't easily provide. We developed a way to use AI to quickly prototype interactions from research findings to share with teams.
For this project, we focused on a recent user testing session with a VoiceOver user we conducted on Fable. We asked them to show us how they would set up an invoice. They selected the ‘set up invoices' card on the homepage, which opened the side panel. Their keyboard focus was then sent to the panel. They start looking for the “how to create an invoice” link on the top to the panel, but keep getting lost as they accidentally navigate out of the panel and need to navigate through the whole page to get back to it. They share some thoughts on what would have worked best for their technology, once they understand that the content has been placed on a side panel: "You can still keep it on the side panel, but if you treat it like a modal the avarage VoiceOver user won't get lost, as they will be able to complete their task and get back to where they were."
The main thing we wanted to communicate is that the way the panel is built directly affects the user's ability to successfully complete their task: • Placing the side panel at the bottom of the page in code enables screen reader users to accidentally navigate away from the side panel without realising, losing context and having to reset their task. Same happened if it was placed between the nav and the main content. • On the other hand, if we treat the side panel as a modal, focus can be trapped to it. This would give the user the isolated context they need to complete the task, and when ready, close it and get back to where they were.
After trying to communicate the differences through screenshots and arrows, I decided to built 3 AI powered prototypes showing the 3 different structures and tab order, as well as adding the screen reader output. Below is an example of the one where we trapped the focus to the side panel, treating it like a modal:
We also prototyped the side panel between the main navigation and the main content, and placed at the bottom of the DOM. This gave us a clear way to communicate the challenges and benefits users would face navigating each of them: 1. Side panel treated as a modal with focus trap (preferred) Focus goes from ‘set up invoices’ to the panel, and user can only get back to the main page after dismissing the panel. 2. Side panel placed between the main navigation and the main content Focus goes from ‘add an online payment service’ to the first interactive element on the main page, which is the ‘hide tasks’ link (hidden behind the panel). 3. Side panel placed at the bottom of the DOM, after the main content Focus goes from ‘add an online payment service’ back to the nav, as the panel is now placed at the bottom of the DOM. User needs to navigate the whole page to get back to it. The prototypes left the team with a clear understanding of why the focus trap approach was the right one. But the process also raised questions about the accuracy of AI-generated screen reader output, about the difference between simulating disability experience and researching it, and about what happens when a faster tool quietly displaces the slower, more rigorous one it was meant to support.
Pooja Gote - Senior Technical Accessibility Analyst