Can not see elements inside WebView Element with XCUITest on simulator(with UIAutomator2 it works)

[dragos-panzaru-md]

Do I have the most recent component updates?

• I use the most recent available driver/plugin and server versions

Is the component officially supported by the Appium team?

• I have verified the component repository is present under the Appium organization in GitHub

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When using ignoreUnimportantViews(false) in conjunction with enableMultiWindows(true), in my native iOS app, when I am redirected to a screen that represents a webview (app is native so no webview context available), the only element I see in the page source is an element of type XCUIElementTypeWebView, but can not see the elements inside it, when using simulator
On Android I am able to see all content and elements in the webview element, when using emulator.

Expected Behavior

Expected behavior was to be able to see all elements in the webview on iOS. like I am able on Android

Minimal Reproducible Example

The repository is private

Environment

• Appium server version (output of appium --version): 2.5.1
• Appium driver(s) and their version(s): uiautomator2@3.0.2 (automationName 'UiAutomator2'), xcuitest@7.3.1 (automationName 'XCUITest')
• Appium plugin(s) and their version(s): no plugins
• Node.js version (output of node --version): v21.7.1
• npm version (output of npm --version): 10.5.0
• Last component(s) version which did not exhibit the problem:
• Platform and version under test: iOS 17.2
• Real device or emulator/simulator: simulator (on real device on Saucelabs it works fine)

Link to Appium Logs

No response

Further Information

[
iOS.json
android.json
android.txt
iOS.txt
](url)

[KazuCocoa]

It usually means the XCTest framework (appium relies on) didn't detect elements inside WkWebView.
https://stackoverflow.com/questions/47324568/how-can-i-enable-voiceover-for-webview-content-on-tableview

https://appium.github.io/appium-xcuitest-driver/latest/preparation/#accessibility-settings would help to expose elements as possible.

[dragos-panzaru-md]

I understand what you are saying, but it does not explain why on simulator it does not work and on real device it works fine. My expectation would have been the other way around.

[KazuCocoa]

Only Apple could answer for it. The attached accessibility settings is one possible method to expose more elements afaik based on my past experience. I'm not sure if they help for WebView contents though.

[dragos-panzaru-md]

Thank you for your help. Will try and come back with a conclusion.

[mykola-mokhnach]

You may also verify if you can connect to the web view using Safari remote debugger. Because if the latter does not detect it we won't be able to do that as well.

[dragos-panzaru-md]

@mykola-mokhnach Not sure how I can do that since my app is not hybrid, but rather it is a native app that at one point takes you to a screen that is a webview. Are there any instructions on how to connect to that webview using Safari remote debugger?

[KazuCocoa]

https://webkit.org/web-inspector/enabling-web-inspector/
https://developer.apple.com/documentation/webkit/wkwebview/4111163-isinspectable?darkschemeovr=1

Below implementation in an app is hybrid we usually say.

• SFSafariViewController
• WKWebView
• UIWebView
• JSContext

[dragos-panzaru-md]

@mykola-mokhnach @KazuCocoa I just verified, and it seems for Android the elements in the webview are accessible on emulators, but on real devices, the elements inside are initially available, then if I start scrolling down through the article list in screen, then the elements inside the webview element no longer appear in the page source. Any ideas why this behavior is occurring on Android?

[mykola-mokhnach]

I assume this is how rendering there works. You could try to play with allowInvisibleElements or ignoreUnimportantViews settings, but otherwise I cannot see how the rendering of the accessibility layer could be influenced from the outside

[dragos-panzaru-md]

So for iOS I talked to the devs to add isInspectable=true, but for android I don't understand why for emulator it works (although it generates all content in the webview all at once, even if certain elements are not actually in the viewport) and for real devices I just see the webview element and nothing in it. Does android have something like isInspectable?

[mykola-mokhnach]

So for iOS I talked to the devs to add isInspectable=true, but for android I don't understand why for emulator it works (although it generates all content in the webview all at once, even if certain elements are not actually in the viewport) and for real devices I just see the webview element and nothing in it. Does android have something like isInspectable?

It highly depends on the framework used in your app. Make sure you follow the documentation for it.

[mykola-mokhnach]

Closed as not an issue

[dragos-panzaru-md]

@mykola-mokhnach Could you please reopen this issue? The reason for my request is that when running a manual test session on Saucelabs, the elements inside the webview are rendered correctly in the xml page source, but locally I still can not see anything in the webview element in the xml page source.
Given the situation described above, I believe we still need to sort out what capability (or maybe device setting) leads to the difference in behavior between my local setup and Saucelabs execution.

[KazuCocoa]

I assume asking Sauce support to check their configuration like accessibility configuration on the device would help

[dragos-panzaru-md]

Already asked what capabilities they are using, but not reply yet. They also they gave me a link to this page: : https://medium.com/@atifqamar29/debugging-ios-webviews-on-safari-browser-822def097497. But I already configured my own iphone with those setting from the page, and still no luck

[KazuCocoa]

Based on my experience, simulators and real devices could have different behavior in iOS, thus possibly such diff also could cause this kind of diff.

[dragos-panzaru-md]

But, on both Saucelabs and locally I am running my test on a real device. So no simulators involved now, but still see the difference in how the page source is rendedered, and I can't figure out why.

[KazuCocoa]

I'm not Sauce person, so I don't know of their internal. It sounds like the only thing is asking for their support. Did both devices have the same accessibility config etc? I wondered also if debuggable signed app https://appium.github.io/appium-xcuitest-driver/latest/preparation/#webview-testing could do it