I had this exact issue with the Elgato StreamDeck user interface, which renders the macro buttons as one big canvas only drag-and-drop interacts with.

It's rare that I outright cannot figure out a way to make a UI accessible, but that one managed to break everything I tossed at it. I eventually switched my effforts to an open-source third party client which had a web UI, which is now accessible to screen reader users.

As for repairing HTML, my last few engagements have clearly shown me that while HTML CAN be repaired, many devs these days wouldn't know where to even begin to do that. The amount of hand-holding I've had to do has made me suggest, several times, to just give me git access as it'd be faster if I just fix it myself.

Screen readers almost entirely ignore the visual layer of any UI, and are entirely dependent on the layer that most developers ignore because it's not the visual layer. It's a perfect storm.

It stands to reason that someone who's actually used to using a screen reader should be brought in to verify what you've built actually works well for that target audience.

I'm a fully blind accessibility auditor, remediator and trainer myself, but I wouldn't dare to assume to know how a mobility-impaired user using eyegaze tech fares on a website I've audited.

My eyes don't gaze, so I don't have the context to make those calls.

On that note: I'm looking for work, anyone need me to tell them how their UI is bad for accessibility and fix it for them so they don't get sued later? :P

If you know you're going to add accessibility, which ... we have had WCAG since 2005, not knowing that at this point is negligence imho, just make sure you work with frameworks and libraries that won't require overhauling all the things when the PO or management finally get sued into letting devs actually implement it properly. If that kind of functionality takes a backseat to "stunning" and "beaituful" designs that a bunch of people can't use, we take the user out of user interface.

Through that lens, this all looks a bit performative to me, but again, maybe I'll be pleasantly surprised.

The one thing I'm mildly excited to see is the improvement to Voice Control, as guessing what the programmatic name of a button is or having to constantly use a numbers grid to target elements doesn't sound fun.

To respond to what I see in some of the comments:

- On speech rate: It does take quite a bit of practice to crank up the speech rate and there's a degree of retraining you need to do when you switch voices. A lot of more "human" sounding voices are harder to follow at super high speeds which is why a lot of people prefer more robotic but consistent speech and generally aren't convinced by AI-powered TTS yet; they often fall apart if you raise the speech rate past a certain point. - Re: actually waiting for the target audience's verdict: This is so important. I see more and more companies, individuals etc. talk about accessibility, build accessibility solutions and evangelize AI for accessibility without EVER talking to the people they claim to help. This will almost certainly mean mistakes will be made, up to and including doing more harm than good. If you want to do accessibility right, that includes AI products of any kind, hire people with lived experience or you'll get the equivalent of machine-translated text, hackerproof security in one click or an AI-powered coffee bar that orders thousands of rubber gloves. Coincidental note: I have time for new projects right now :P

I'd say LLMs COULD make it easier to implement accessibility, it also couldn't, always a coinflip with those, but I'd say LLMs actually succeeding is probably unlikely given how much shitty code is probably in its training data :P

This is true for many accessibility projects actually (game mods, third-party UIs for inaccessible services/platforms, etc.). These are generally really meant as short-term patches while the problem gets fixed, except ... the problem often doesn't get fixed because the platforms in question figure it's been solved now and they don't need to care about it anymore.

Accessibility really only works when it's an ongoing, first-class process within an app/platform's design, and we can absolutely do that; the standards and guidelines have existed for decades. People working in cybersecurity, localization, general UX should recognize this song and dance, which is amusing because a lot of the tools of those trades have atrocious accessibility and require all sorts of workarounds, ask me how I know.

People just ... aren't including it in this way, which means people like myself (screen reader user and accessibility professional) essentially have to keep reminding people that we exist and that it's kinda shitty to keep forgetting about that fact or to decide the least amount of effort possible (LLM, unpaid volunteer, send in a PR LMAO) is enough to cater to people who have very real, very annoying and very constant UX issues we either crash into or crash through on literally an hourly basis.

What are you basing that on? Screen readers tend to not pick those up at least on interactive elements by default, you need to do a bit of "wiggling" to get those to be announced. Disclaimer: screen reader user