Using AI prototypes to communicate accessibility research insights

A big part of what we do in the accessibility team is conducting research with people who use assistive technologies. We sometimes get questions we can’t answer directly, as we don't have the lived experience of being an assistive technology user. Taking these questions to Fable, where we can test with professional assistive technology users, is the best way for us to provide our design team with the right direction and feedback.

A big challenge we have been trying to tackle for a while is how to communicate our findings back to product teams when most of it relates to interactions, navigation and assistive technology commands. It relies quite a bit on understanding how different assistive technologies work, and this can be hard when all we have are Figma prototypes.

Explaining tab order, focus states, and how assistive technology users perceive a page requires a level of interactivity that traditional design tools don't easily provide.

Here’s an example

The team working on the Homepage Day 1 project reached out as they wanted to understand the flow from the homepage to the side panel from an assistive technology perspective.

Below are a couple of snippets from our session with John, a VoiceOver tester with Fable. We asked John to show us how he would set up an invoice. He selected the ‘set up invoices' card on the homepage, which opened the side panel. His keyboard focus was then sent to the panel.

We were lucky to have a coded prototypes for this test, and for the first few sessions we asked him to build the side panel at the bottom of the page, after the navigation and the main content.

John is looking for the “how to create an invoice” link on the top to the panel, but he gets lost as he accidentally navigates out of the panel and needs to navigate through the whole page to get back to it. (watch clip)

‍

John’s thoughts on what would have worked best for his technology, once he understands that the content has been placed on a side panel. (watch clip).

What we learnt  

The way the panel is built directly affects the user's ability to successfully complete their task.

• Placing the side panel at the bottom of the page in code enables screen reader users to accidentally navigate away from the side panel without realising, losing context and having to reset their task.
• On the other hand, if we treat the side panel as a modal, focus can be trapped to it. This would give the user the isolated context they need to complete the task, and when ready, close it and get back to where they were.

We then tried to explain this to team, highlighting the 3 different ways we could build the page with the side panel, and the one John preferred.  

It looked a bit like this:

and this:

😅 Not the easiest to understand, is it?

What if we try this again, now using Vercel?

After watching demos of our designers putting together a prototype from a screenshot, we saw an opportunity to do the same to explain some of these findings that rely so heavily on the interactions.

I started with Gemini, using it to help me write a prompt — a choice that was itself revealing. I'm not a developer, and prompt engineering for code generation wasn't something I had practised. Using one AI tool to help me use another felt pragmatic in the moment. It also, in retrospect, added a layer of abstraction between my intent and the output that is worth examining. This is what I wrote:

I'm an Accessibility Lead in Design and often I struggle with clearly communicating to teams the barriers an assistive technology user would face when interacting with their designs. Visualising the path of an assistive technology user relies on understanding the technology and its shortcuts well, which is not the case for most people.

So far, I’ve been taking screenshots and using arrows to try and explain different flows. This takes too long and it’s not always effective, so it would be better if I could have an interactive prototype to show them.

I want to explore using Vercel to upload screenshots of designs and get them prototyped with a visual focus ring that makes it easy for people to see what a keyboard interaction would look like.

Can you guide me on how to write a prompt for me to get this outcome from Vercel?

This is (a summary of) what Gemini sent back:

I need an interactive web prototype to demonstrate keyboard navigation for assistive technology users. The prototype should clearly show how focus moves between interactive elements.

Recreate the attached screenshot as closely as possible and ensure all interactive elements (buttons, links, form fields, etc.) are tabbable.

Implement a highly visible focus ring that is distinctly different from the default browser focus styles. Use a bright, contrasting colour like outline-blue-500 for the focus ring. Make the focus ring 4px solid and ensure it appears outside the element's existing borders. The focus ring should be visible only when an element is focused via keyboard navigation (e.g., using the Tab key). Ensure the focus ring has sufficient contrast against various background colours.

Add a visual at the bottom that shows the screen reader output while tabbing through links.

I then also added the tab order I wanted for each of the options we talked about above:

1. Side panel treated as a modal with focus trap (preferred)
2. Side panel placed between the main navigation and the main content
3. Side panel placed at the bottom of the DOM, after the main content

Results

Here’s the results of our experiment after feeding the Gemini prompt into Vercel with the specific tab order we wanted.

1. Side panel treated as a modal with focus trap (preferred)

(view prototype video)

Focus moved from ‘set up invoices’ to the panel, and user can only get back to the main page after dismissing the panel.

2. Side panel placed between the main navigation and the main content

(view prototype video)

Focus goes from ‘add an online payment service’ to the first interactive element on the main page, which is the ‘hide tasks’ link (hidden behind the panel).

3. Side panel placed at the bottom of the DOM, after the main content

(view prototype video)

Focus goes from ‘add an online payment service’ back to the nav, as the panel is now placed at the bottom of the DOM. User needs to navigate the whole page to get back to it.

Learnings

Well, it worked! And it took around 20 minutes, which is much faster than what we’ve experience before with screenshots and arrows. Also:

• It required a couple of rounds of iterations to make sure the tab order was correct and that the screen reader output was accurate. It’s still not quite there yet, but we’re happy with the results of our first experiment.
• Vercel will summarise the changes applied to the interface in a list format. We used the same language it used when we needed iterations, which seemed very effective.  

‍What the experiment didn't resolve

The prototypes showed keyboard focus moving through the interface in the order I specified, but the screen reader output displayed at the bottom of each prototype was generated by the AI, not produced by an actual screen reader. It was an approximation of what a screen reader might announce, based on the AI's understanding of how screen readers work.

For a team trying to understand the general shape of an accessibility problem, that approximation may be sufficient. But screen readers are not uniform. VoiceOver, JAWS, and NVDA behave differently, announce things differently, and interact with web content differently. An AI-generated simulation flattens that complexity into a single, confident-looking output, which can lead a designer who relies on it to believe they understand the screen reader experience more fully than they do.

There's another thought I keep coming back to, which is that prototyping with AI helped us with what we needed, but it highlighted that the bar for understanding disability experience is whether it can be made frictionless enough for non-disabled designers to engage with. The problem is not that AI-assisted prototypes lower that bar, but that the bar exists at all. If teams had meaningful, ongoing relationships with disabled users throughout their design process, the need to simulate that experience for communication purposes would be significantly reduced.

There is also a risk I haven't seen discussed much in the accessibility community: that AI-generated simulations of disability experience become a substitute for research with disabled people rather than a supplement to it. Prototypes are faster and cheaper than recruiting and running sessions with assistive technology users. In an organisation under delivery pressure, the temptation to reach for the faster tool is real, and when that tool produces output that looks accurate, I worry the pressure to validate it with actual users may quietly disappear.

Where this leaves me

The experiment worked better than I expected. The prototype communicated something that hours of annotation hadn't managed to, and the team left with a clear understanding of why the focus trap approach was the right one.

But the process also raised questions about the accuracy of AI-generated screen reader output, about the difference between simulating disability experience and researching it, and about what happens when a faster tool quietly displaces the slower, more rigorous one it was meant to support.

For now, we're treating this as a promising first experiment rather than an established practice. The next step is to test the prototypes using an actual screen reader with a real user, which will tell us a lot more about where the AI output holds up and where it doesn't. That validation feels important before this becomes part of how we work.

‍