Gamepad integration #791

albinekb · 2024-07-14T19:12:29Z

albinekb
Jul 14, 2024

Hello! Amazing library. I got it running on my RGB30 handheld console with little hassle now that it's running the mainline kernel.

I am trying to figure out how I can use the accessibility apis to enable navigation using the gamepad.

I can successfully focus next/prev node using gilrs and use_platform() hook:

gilrs::EventType::ButtonPressed(button, code) => {
  match button {
    Button::DPadLeft => {
       let _ = platform.send(EventMessage::FocusPrevAccessibilityNode);
     }
     Button::DPadRight => {
        let _ = platform.send(EventMessage::FocusNextAccessibilityNode);
      }
}

Now I am trying to figure out how to get the currently focused nodeId so that I can trigger the click handler:

let action = ActionRequest {
    action: accesskit::Action::Default,
    target: nodeId,
    data: None,
};
let window_event =
    accesskit_winit::WindowEvent::ActionRequested(action);
let evt: EventMessage =
    EventMessage::Accessibility(window_event);

let _ = platform.send(evt);

Is there some way I can access the focused_id?

freya/crates/core/src/platform_state.rs

Line 8 in 1cd1d2c

pub focused_id: AccessibilityId,

Would it make sense to add it to use_platform_information or some other way to access the focused_id? Perhaps adding it as a PluginEvent?

Or am I doing this from the wrong direction, is there a smarter way to integrate gamepad/inject custom keyboard/accessability events?

Thanks!

marc2332 · 2024-07-14T19:30:53Z

marc2332
Jul 14, 2024
Maintainer

Hey! Interesting thing you got in your hands!

You can read whether an element is focused or not with the use_focus hook.

Example:

#[allow(non_snake_case)]
fn CoolComp() -> Element {
    let mut focus = use_focus();
    let is_focused = focus_manager.is_focused();

    rsx!(
        rect {
            width: "100",
            height: "60",
            focus_id: focus.attribute(),
            background: "rgb(45, 45, 45)",
            color: "white",
            label {
                "Am I focused? {is_focused}"
            }
        }
    )
}

Not sure what you meant by so that I can trigger the click handler:, how is a click event handler related to this? Maybe if the example I shared still didn't work, you could share a bit of your code to give me some context 😄

Or am I doing this from the wrong direction, is there a smarter way to integrate gamepad/inject custom keyboard/accessability events?
Honestly, it's hard to tell! But I think that if it works, it works! I never attempted anything similar before, so perhaps there are some rough edges that I could make more flexible, like allowing other types of inputs like game pads. Your implementation reminds me of Freya's NativeContainer component which does a similar thing to focus the previous and next elements.

Let me know if that works or if you still need help :)

8 replies

albinekb Jul 15, 2024
Author

Some of the components of Freya can be actioned by pressing Enter when focused. This is how its done in button for example:

freya/crates/components/src/button.rs

Lines 154 to 160 in 1cd1d2c

let onkeydown = move |ev: KeyboardEvent| {

if focus.validate_keydown(&ev) {

if let Some(onpress) = &onpress {

onpress.call(PressEvent::Key(ev))

}

}

};

If there was an API to map a game pad press to -> keybown event with Enter being pressed perhaps it could work. Maybe as a plugin?

Yes! I can configure gptokeyb to emulate keyboard events for the gamepad buttons, will try it. I was thinking to use gilrs to get access to analog inputs.

It would be more flexible if I could emit my own KeyboardEvent somehow 🤔