Support the gamepad UI layout for the integrated screen checks

[stylpe]

First of all, Thanks for a great tool!

I like to run the leaguestart campaign with a gamepad due to the small QoL features like auto gem leveling. But the integrated check for the Gamescreen don't seem completely reliable with the alternate UI art and layout. After calibrating, the test seems to have a 50% chance to be positive or negative. The inventory calibration and check seems ok, but I haven't tested it extensively. The other image based checks are of course fine.

Here's a couple of screenshots to show what it looks like:

[Lailloken]

Yes, I can look into it. Though the gamescreen check is less important when playing on a gamepad. Aside from auto-hiding clone-frames (it used to affect more features), it mainly controls what an omni-key press does. If the script detects that the player is on the "gamescreen" when the omni-key is pressed, it will assume it was pressed to use a skill bound to it and not to access a feature. If the check is negative during a press, it will initiate screen-reading to determine the context of the key-press.

So the check merely minimizes the amount of screen-reading the script does (depending on which key/skill combination is used, the omni-key might be pressed excessively) in order to improve script -- and sometimes system -- performance. Since playing on a gamepad doesn't involve much omni-key usage, it should be fine for now. To make sure you don't get too many false-negatives for that check when you do press the omni-key, you could calibrate the gamescreen check with a rare color-value (setting the camera in a way that e.g. the fireplace in Lioneye's watch fills the top-right corner of the screen).

[stylpe]

Ok, thanks! I have also started noticing the leveling guide behaving weirdly, skipping ahead of for example Tidal Island as soon as I entered it, don't know if it's related to this or because I rerolled to a different class and reset the progress with a new build? I assume that's based on log parsing though so idk.

One more note is that ctrl-f doesn't work as usual since there's a game pad specific button combo instead, but I am able to use the buy gem filter by manually focusing the search field first and then hitting the omni-key. Not sure if that's possible to improve script-wise but you could explain it in the help tooltip as a backup plan.

[Lailloken]

I rerolled to a different class and reset the progress with a new build

As long as you use the "reset progress" in the settings menu after importing a new build, everything should work fine. If this keeps happening, please report back.

ctrl-f doesn't work as usual since there's a game pad specific button combo instead

Right, makes sense. I think you're the first to bring up gamepad support, so it never crossed my mind. I don't think AHK can send gamepad presses but I'd have to do some research to be certain. Otherwise, one would have to make do with this hybrid approach.

[stylpe]

Yep, and I'm almost certainly a minority user here, so my expectations are low and I'll get by just fine with how it already works. It's just for the campaign after all. Already appreciate you just replying and considering it 😁

[stylpe]

Update on the level guide automation: I think it always detects me as being in Lionseye, because it never moves to the next step except if the next step is in Lionseye.

[Lailloken]

Huh, the first thing you could check is if the client.txt in the logs folder of the game is still being written to correctly by the client. If it behaves normally, the script itself could be stuck or the game may deny access to the log-file. Either way, it's the first time I've heard of such a behavior.

[stylpe]

Yeah, here's the last part, and it looks like this is a PoE issue :P

2023/04/08 15:02:08 73203 951bc057 [INFO Client 1820] [STARTUP] Registration in 0.484979 seconds
2023/04/08 15:02:11 75734 47bbea58 [INFO Client 1820] Changing to device "Headset Earphone (11- Arctis 7 Chat)"
2023/04/08 15:02:11 75750 951bc057 [INFO Client 1820] [STARTUP] Audio in 3.02254 seconds
2023/04/08 15:02:11 76281 b51d4f61 [DEBUG Client 1820] [RENDER] SetRendererSettings
2023/04/08 15:02:11 76281 b51d6447 [DEBUG Client 1820] [RENDER] ResizeWindow
2023/04/08 15:02:11 76281 3cd2dd07 [DEBUG Client 1820] [VULKAN] ResizeBuffers
2023/04/08 15:02:11 76281 3cd2cec0 [DEBUG Client 1820] [VULKAN] ReinitSwapChain
2023/04/08 15:02:11 76281 97399bf1 [INFO Client 1820] [VULKAN] SwapChain = 3840x2160
2023/04/08 15:02:11 76328 9d4eec8a [DEBUG Client 1820] [WINDOW] TrigggerDeviceLost
2023/04/08 15:02:11 76359 9d4eeba7 [DEBUG Client 1820] [WINDOW] TrigggerDeviceReset
2023/04/08 15:02:11 76390 b51d614c [DEBUG Client 1820] [RENDER] RestartSwapChain
2023/04/08 15:02:11 76390 9e6a0b0a [INFO Client 1820] Enumerated adapter: NVIDIA GeForce GTX 1070
2023/04/08 15:02:11 76515 9e6a1d2a [INFO Client 1820] Enumerated device for adapter: NVIDIA GeForce GTX 1070. Selected feature level: 45312. Max feature level: 45312
2023/04/08 15:02:11 76515 9e6a1363 [INFO Client 1820] Enumerated output for adapter NVIDIA GeForce GTX 1070 of \\.\DISPLAY1
2023/04/08 15:02:11 76515 9e6a0b0a [INFO Client 1820] Enumerated adapter: Microsoft Basic Render Driver
2023/04/08 15:02:11 76531 9e6a1d2a [INFO Client 1820] Enumerated device for adapter: Microsoft Basic Render Driver. Selected feature level: 45312. Max feature level: 45312
2023/04/08 15:02:11 76531 b51d650d [DEBUG Client 1820] [RENDER] ResizeDevice
2023/04/08 15:02:11 76531 9d4eec8a [DEBUG Client 1820] [WINDOW] TrigggerDeviceLost
2023/04/08 15:02:11 76531 9d4eeba7 [DEBUG Client 1820] [WINDOW] TrigggerDeviceReset
2023/04/08 15:02:11 76578 3275a75e [INFO Client 1820] [SHADER] Disable Async: 1 frames
2023/04/08 15:02:11 76578 b51d610e [INFO Client 1820] [RENDER] Disable Async: 1 frames
2023/04/08 15:02:12 77484 3cd2cec0 [DEBUG Client 1820] [VULKAN] ReinitSwapChain
2023/04/08 15:02:12 77484 97399bf1 [INFO Client 1820] [VULKAN] SwapChain = 3840x2160
2023/04/08 15:02:12 77546 9d4eec8a [DEBUG Client 1820] [WINDOW] TrigggerDeviceLost
2023/04/08 15:02:12 77593 9d4eeba7 [DEBUG Client 1820] [WINDOW] TrigggerDeviceReset
2023/04/08 15:02:15 80187 ca86d7fb [DEBUG Client 1820] Rebuilding UI::Core::Properties due to input method change
2023/04/08 15:02:15 80187 ca86fd3f [DEBUG Client 1820] Rebuilding UI::Core properties. Current input mode = '1'
2023/04/08 15:05:59 303781 ca86d7fb [DEBUG Client 1820] Rebuilding UI::Core::Properties due to input method change
2023/04/08 15:05:59 303781 ca86fd3f [DEBUG Client 1820] Rebuilding UI::Core properties. Current input mode = '2'
2023/04/08 15:05:59 303875 890d9d5f [INFO Client 1820] Async connecting to ams01.login.pathofexile.com:20481
***** LOG FILE ERROR. REOPENING *****

Don't know if the timestamps are UTC or local timezone, but nothing's been logged for at least 3 hours 😅 I'll deal with that on my own.

Good news is that this log also shows an obvious way to detect a gamepad being used, so there's a silver lining!

[stylpe]

Well there's your problem!

[Lailloken]

I did some testing around integrating gamepad support and initially got good results. But then I noticed that AHK was causing unusually high CPU-load when the omni-key was mapped to a gamepad-key (and when there were references to gamepad-keys in general) -- to the extent that the script was generating as much load as the game-client itself. While CPU-load itself is not a big deal, I cannot rule out the possibility that this may lead to stalling or increased latency within the script's code itself.

There's also the problem that AHK is limited to receiving gamepad key-presses. It cannot send key-presses without using external gamepad drivers/emulation, so there's no easy way of sending gamepad-specific versions of the required key-combinations (e.g. CTRL-F).

[stylpe]

Unfortunate. And even if you could send button presses through a virtual controller, I think PoE might lock input to a single device.

The gamescreen detection might still be possible to improve, though.

[LuckyTianTian2]

I also use Xbox 360 gamepad. And I used reWASD to edit what the button mean. And everything was fine for me.

For example,

1. I edit the "omni-key" to be "P" key,
   2.1 in reWASD, "holding gamepad button B" is set to "press button B + Holding P key for 0.5 second".
   2.2 in reWASD, "press gamepad button B" is set to "press button B + press P key".

Everything was solved for me at present. However, I only use iteminfo-check at present. I don't know if any other function work well or not.

[stylpe]

Hey again! Quick update, it seems like ´Ctrl+F´ is working now even when using a gamepad, so that's no longer an issue! But I'm still having problems with unreliable screen checks, even on the syndicate board which is pretty static. This might point to it being unrelated to using a gamepad, so I will do a bit of extra experimenting tomorrow!

And I'm using a Steam controller, which also has a rich ability to rebind buttons. I set the omni key to ´Home´ since it's the least in the way of both normal usage and standard bindings.

[Lailloken]

I see. Now that I think of it, it's probably better to leave the gamescreen pixel-check uncalibrated for gamepad-usage in order to prevent false-positives. You can delete the calibration data by removing the [gamescreen] section in ini\screen checks (XXXXp).ini and then restarting the script. This will lead to omni-key presses always prioritizing omni-key features (as opposed to whatever is normally bound to the key, e.g. middle mouse-button).
The only drawback is that automatic clone-frame toggling would not be available since that also uses the gamescreen pixel-check. If you want to make use of that, let me know -- I'll try to source a gamepad from somewhere then.