Alternative to drag-and-drop to associate actions with buttons
I'm a screen reader user and therefore, I rarely if ever really use a mouse. Provided elements are present in the DOM, clicking/right clicking/double clicking is doable enough, but unfortunately drag-and-drop actions are generally a non-starter.
I'm relatively sure that Opendeck, like the official Windows client, uses this to assign an action to a button. The interface is actually quite usable without sight, it's just that one thing that's stopping me from using this tool. Is there some way we can macgyver some kind of more keyboard-friendly way to do this? e.g. hit space on action, tab to button, hit space on button? Or even double click action then double click button? Maybe mark a button for droppage from the context menu and then click an action?
Right now I am unfortunately unable to use either the official client (whom have essentially indicated if they fix it at all it will be a while which is generally corp speak for when the intern has time in 20 years) nor this one, mostly for the same reason. If there's no bandwidth I can try to hack something together and make a PR but I'm not familiar with the codebase at present.
6 Replies
I must say I'm actually not sure what a screen reader really is
I suppose VoiceOver/TalkBack are the mobile equivalents? If so then I get it
I think I could try implementing the hit space on each route
Yep, VoiceOver / Talkback on mobile, Narrator/NVDA/JAWS on Windows.
Basically the issue is two-fold right now, looking at the code:
- The key components are rendered as canvases , which don't appear in the accessibility tree of a web page by default. (https://blog.pope.tech/2023/11/27/how-to-use-chromes-accessibility-tree/)
If it's not in the accessibility tree, a screen reader won't know about it. This should be relatively easy to fix by adding role="button" to the <canvas> elements, and, if that results in buttons with no name, maybe set the button position as the canvas's aria-label attribute.
- The drag and drop bit: We'd probably first need to make sure the canvases can even be reached with the keyboard cursor (tabindex) and then like you said, hitting space on each should work. Extra credit if we can somehow make the button reflect what the current action is (aria-label ?) but I'll take anything at this point
Whitney Lewis
Pope Tech Blog
How to use Chrome's accessibility tree - Pope Tech Blog
Learn what the accessibility tree is, how to access it, and how to use its information.
Reason for all this is because a screen reader doesn't actually "read" the screen at all, it interprets messages from a bunch of APIs at the app and OS level. So, say, a link has a name because it's link text is its name. A button with just an icon on it doesn't because there's no text to parse. Same with a canvas, which is just a ... well ... canvas, it has no semantic meaning and no textual info associated with it so screen readers have no idea what to do with them , role='button' tells the screen reader to interpret it as a button without changing any of the visuals, and aria-label overrides, or sets, a control's name if it doesn't have one, or has a bad one. I hope that makes sense, I know its a little conceptual 🙂
I did a bunch of work over the weekend to make this work and just issued a PR, I'm afraid it's pretty big though, sorry about that 🙁
It does make pretty much the entire thing keyboard and screen reader accessible
@zersiax thank you so much for your work, but there's no way I can review a PR that large in one go 😅
if you don't mind, could you separate it out into different bits? then after one is merged you can open the next and it would be a lot more manageable for me
maybe start with 'Delete on a key clears it, and ctrl+c and ctrl+v should work for copying actions from one key to another.', that one sounds pretty simple
or probably 'Allows for arrow-key-navigation within the key grid as well as the action list', that one sounds like it needs to come before that
Will see what I can do, unfortunatley my holiday's over so work's started up again, not sure when I'll be able to get to it 😅Lot of it is somewhat interlinked so it will be a bit of a thing to split it up, but will see what I can do when I have some downtime