Real device testing with Appium and ruby/cucumber

Testing
1

problem

I am unable to successfully run a good cucumber scenario when running on a real iOS device against a mobile web page. The home page is opened, the global navigation hamburger menu (GNav) is opened, an option on the menu is supposedly tapped, and the menu closes. But the requested page is never displayed and eventually the test times out looking for an element on the expected new page.

I’ve tried everything I can think of to track down the problem, including issues on the appium site – all to no avail.

If someone has been able to run tests on real iOS devices using appium in an environment similar to what I’m running (see below), I’d sure appreciate the help.

##TL;DR

environment

  • appium version : 1.6.4(beta)
  • OS X ElCap, 10.11.6
  • node.js version : 7.5.0
  • iOS versions under test : 9.3.5 and 10.0.2
  • Xcode version : 8.2.1
  • appium-ios-driver@1.16.0 (installed with -g option)
  • appium-xcuitest-driver@2.7.0 (installed with -g option)
  • authorize-ios@1.0.4 (installed with -g option)
  • ios-deploy@1.9.1 (installed with -g option)
  • ios_webkit_debug_proxy@ 1.7.1, Built with libimobiledevice v1.2.0, libplist v1.12
  • libimobiledevice: stable 1.2.0 (bottled), HEAD
    *ideviceinstaller: stable 1.1.0 (bottled), HEAD
    *.xcconfig created with:
    DEVELOPMENT_TEAM = XXXXXXXXXX
    CODE_SIGN_IDENTITY = iPhone Developer
    IOS_INSTALL_PAUSE = 3000
  • WebDriverAgentRunner version 1, com.apple.test.WebDriverAgentRunner-Runner
  • IntegrationApp version 1, com.facebook.IntegrationApp

test flow

  • The test begins running, the WebDriverAgent is built and loaded, and I can see the device / Safari reacting: Starting from Safari, the device slides Safari out of the way, displays a completely black screen, then goes to the home screen, then back to Safari.
  • After a longish pause, the device displays our home page on Safari and the Global Navigation menu opens
  • At this point, the test taps one of the menu items and we should be redirected to the next page
  • However, what happens is that the menu is closed, and the home page just sits there - no transition to the requested page.
  • Cucumber/Selenium emit the following error message:
Element {:css=>"#m-j-access-reg"} was not present in 20 seconds (Selenium::WebDriver::Error::TimeOutError)
      ./lib/utils/extensions/page-object/platforms/selenium_webdriver/element.rb:11:in `when_present'
      ./lib/utils/extensions/page-object/accessors.rb:39:in `block in expected_element'
      ./lib/mobile_website/pages/registry_home_page.rb:23:in `initialize_page'
      ./lib/utils/extensions/page-object/page_object.rb:47:in `block in initialize'
      ./lib/utils/extensions/page-object/page_object.rb:61:in `call'
      ./lib/utils/extensions/page-object/page_object.rb:61:in `block in perform_benchmark'
      ./lib/utils/extensions/page-object/page_object.rb:61:in `perform_benchmark'
      ./lib/utils/extensions/page-object/page_object.rb:39:in `initialize'
      ./features/step_definitions/mobile_website/registry_migration/registry_brand_index_steps.rb:29:in `/^I visit a registry "(.*?)" page via "(.*?)"$/'
      features/website/Selection/MEW/registry/XXXX/registry_navigation.feature:16:in `And I visit a registry "Home" page via "GNav"'

running programs

  • appium servier
appium -g ~/appium_server.log --log-timestamp --local-timezone --debug-log-spacing --tmp ~/tmp --default-capabilities '{"showXcodeLog": true, "automationName": "XCUITest", "cssSelectorsEnabled": true, "nativeWebTap": true, "nativeEvents": true, "autoWebView": true}'
  • ios_webkit_debug_proxy
ios_webkit_debug_proxy -c xxxUDIDxxx:27753 -d running in another terminal window and connected to my device ("JON's iPad")
  • cucumber test
bundle exec cucumber DEVICE=Iphone -p mobile_website_default SITE=MCOM URL=http://m.qa17codemacys.fds.com/ features/website/Selection/MEW/registry/MCOM/registry_navigation.feature:14

appium logs (if you really want to go there :=)

https://gist.github.com/JESii/1bfdfdea3f5378c7942415075b091813

2

Okaay… found and fixed the problem.

Part of the framework provides support for global navigation requests and they handle it like this:

HTML

<li id="NAV-NAME" ...>
<a href="LINK-URL" ...>LINK-NAME</a>
</li>

Navigation Test Code, where current_nav_elements are the li elements from above:

nav_item = current_nav_elements.find { |el| el.text == LINK-NAME }
nav_item.click

To get this to work, I changed nav_item.click to nav_item.link_element.click

We are currently running:
selenium-client (1.2.18)
selenium-webdriver (2.53.4)
which I’m guessing is part of the problem; our selenium-webdriver is currently at 2.53 and the latest is 3.2.0 :=(

3

RESOLVED with a work-around. Tapping a link inside a div works differently for real device as opposed to the simulator. May well be the old selenium-webdriver we’re using.