Keyboard accessiblity improvements

[royeden]

Describe the bug

It would be good to contemplate the keyboard accessibility for drag move gestures, as the current implementation is not working the same as other libraries that implement this feature. The current implementation starts the drag gesture as soon as any of the keyboard keys are pressed and it brings the following issues:

1. Dragging from the keyboard is complicated and doesn't feel good because the actions are direction locked where the gesture was initiated. If we wished to switch the directions, we must not release the current direction key, even for a single tick, as doing that would consider the gesture inactive and, therefore, reset it's position.
2. This makes implementing accessibility for complex gestures or even dnd oriented libraries based on use-gesture really complicated, as many drag gestures in those cases would rely on moving elements on more than a single axis.

Relevant examples from other dnd libraries:

• react-dnd-kit implements gesture-like API.
• react-beautiful-dnd is more focused on having specific list containers where the objects end in, so they end up behaving more like a FLIP animation.
• svelte-dnd-action behaves similarly to react-beautiful-dnd.

Possible naive fix:
It could be good togo with the dnd-kit approach, where the gesture has to be activated with Enter or Spacebar before moving when you are using the keyboard (This is also used in other libraries like svelte-dnd-action and react-beautiful-dnd, so it seems like the common / intuitive behavior). This would imply thinking about the following caveats:

1. Which event should fire first when using the Enter / Spacebar on the element? Should a click handler be fired on the element if its natively supported (like with button or a)?
2. Should the user be warned that their element is not keyboard focusable when they use the bound gesture during development? (And be offered solutions like using buttons / tabindex).
3. How to handle blur events on the element? They should end the drag gesture. Should a blur event handler be placed during the binding? Or should it track globally with a system based on event delegation?

I believe that issues 1 and 2 could be achieved with the current engine, by swapping out the requirements for the gesture to be active, but 3 might require a bit more thinking.

Let me know if this issue and overview is useful, I would like to try my hand at implementing this feature because I'm looking for a framework agnostic solution to build some DND actions 😄

A possible relevant discussion regarding this issue on this repo

Sandbox or Video

https://codesandbox.io/s/silly-monad-ff6k54?file=/src/App.js (Try tab focusing and using the arrow keys 🙂)

Code:

import { useDrag } from "@use-gesture/react";
import { useSpring, animated } from "@react-spring/web";

export default function App() {
  const [{ x, y }, api] = useSpring(() => ({ x: 0, y: 0 }));
  const bind = useDrag(({ active, movement: [mx, my] }) => {
    api.start({ x: active ? mx : 0, y: active ? my : 0, immediate: active });
  });

  return (
    <animated.div
      {...bind()}
      tabIndex={0}
      style={{
        alignItems: "center",
        justifyContent: "center",
        backgroundColor: "red",
        display: "flex",
        height: "4rem",
        width: "4rem",
        x,
        y
      }}
    >
      Test
    </animated.div>
  );
}

Information:

• Use Gesture version: 10.2.26
• Device: Any device that uses the gesture driven by a keyboard input
• OS: Any OS
• Browser Any browser

Checklist:

• I've read the documentation.
• If this is an issue with drag, I've tried setting touch-action: none to the draggable element.

[dbismut]

Hey, thank you very much for the detailed proposal. I didn't have the time to read it through yet but will try in the coming days. Thanks for your patience.

[royeden]

No problem!

In the meantime, I've tried to implement some stuff on my own, but that made me stumble into some implementation details where I don't understand enough of the context to follow up with it or take decisions that could break the implementation. I'm leaving this here so in the future they can be clarified and also as possible know-ahead for implementing this feature 😄

I've had to manually invoke reset when the gesture is de-activated when I replaced the keyup flow with a toggle to the active state when pressing specific keys that trigger with the keydown. If I didn't resort to that, the element would go back to the initial position after de-activation and return to its transform on a new activation instead of starting a new gesture. This also happens after dragging with the pointer, if no reset is called it will activate the gesture from the last place that the pointer left it.

I've also had a hard time trying to figure out how event listeners were delegated to add a global blur event to cross-reference with the target element, couldn't figure out specifically where the element is stored internally, etc.

Hope we can find a solution in the future and I'll see if I can progress further ahead with my testing.

[dbismut]

Hi @royeden, finally had the chance to look at this. Thanks for your time on this.

Here's a sandbox that we could iterate on:
https://codesandbox.io/s/loving-stitch-ydrdly?file=/src/App.tsx

Dragging from the keyboard is complicated and doesn't feel good because the actions are direction locked where the gesture was initiated. If we wished to switch the directions, we must not release the current direction key, even for a single tick, as doing that would consider the gesture inactive and, therefore, reset it's position.

Unfortunately there's not a lot we can do about this (and it's specific to the simple example sandbox using movement and resetting the position to 0 as soon as the gesture is inactive). One solution would be to debounce the keyup event so that it's not triggered if a new keydown event occurs within let's say 100ms. I'm not sure the other libs you've mentioned handle this anyway.

Handling focus

I really don't think the lib should be opinionated about handling focus. Focus universally works with the tab key and there's no reason for the lib to handle it differently. Since events are attached the draggable div, if it loses focus I guess the gesture should end, but I haven't tested this yet (it's a good point although a bit of an edge case).

Should the user be warned that their element is not keyboard focusable when they use the bound gesture during development? (And be offered solutions like using buttons / tabindex).

That could be interesting indeed, and relatively easy to implement.

[royeden]

Hi @dbismut , thank you for responding!

I believe that adding an option to debounce the keyup handler and leave it to the user to config the debounce time seems like a good fix for the issue in the meantime.

I would really like to see a fix in the future that standardizes more with other libraries/people using keyboard drag gestures (with the enabled / disabled state), so a drag and drop library can be implemented with this library under the hood 😄

Leaving this post as a reference of what I mean with the enabled / disabled interactions to handle movement: https://medium.com/salesforce-ux/4-major-patterns-for-accessible-drag-and-drop-1d43f64ebf09

[dbismut]

Thanks for the link, interesting read. I think we should discuss this on Discord to make this more fluent. Can you add me there?

[royeden]

Joined the pmndrs discord, I'll send the message on the corresponding channel when I'm available!