Testing Mobile Apps with a Mobile Device Cloud

Mar 29, 2018 4:06:35 PM / by Laura Shumaker

 The problem: Apps need to be tested on a massive variety of mobile devices 

  • Market demand for mobile app development services continues to grow, and today’s users expect the highest quality user experience. They demand the most efficient, effective and secure methods for their mobile app development processes, including mobile app testing.
  • To ensure their mobile apps perform reliably for every user, businesses need to test on key variations of the burgeoning variety of mobile devices.
  • Mobile device access management has been plagued with inefficiency and risk, requiring IT teams to purchase an unreasonable number of devices to cover their enterprise’s growing needs.

cloud testing

That’s expensive

In addition to the cost and inefficiencies of having multiple mobile devices, security has become a top concern for today’s enterprises, thanks to the rise in data breaches.

The solution: The Mobile Device Cloud

The Mobile Device Cloud solution allows access to large device clouds or farms featuring hundreds or even thousands of real devices, along with emulators and simulators. Mobile developers don't have to worry about the impractical proposition of rounding up their own devices for testing.

The Mobile Device Cloud helps developers looking to test and debug applications. The solution also helps QA teams performing manual or automated testing facilitating collaboration among these groups and other stakeholders such as marketing and customer support.

Mobile device cloud benefits

A mobile device cloud offers organizations four significant benefits:

  1. Security: On-premises, mobile device clouds access a centrally managed pool of devices and mobile apps and enable testing within an enterprise’s corporate infrastructure and behind the firewall.  This creates a highly secure environment for proprietary apps and sensitive data throughout the testing process. Cloud-based mobile testing improves compliance with regulatory standards by reducing the chances of a data breach or unauthorized user access.
  2. Accessibility: Mobile device clouds meet the enterprise need for global access—developers, testers and mobile engineers can securely access centrally managed mobile devices from locations anywhere across the globe without compromising corporate data security.
  3. Speed: Technology advancements in mobile application testing platforms allow automated, continuous testing simultaneously on multiple devices. This allows mobile device clouds to be even more effective in increasing an enterprise’s app testing speed and quality without compromising security.
  4. Cost: IT teams need affordable options that still maintain efficiency, effectiveness and security. Mobile device clouds often are much more cost-effective than off-premises, public device clouds, while also providing an agile, efficient and effective solution.

3 Mobile Device Clouds Menlo recommends

Mobile app device clouds come in all shapes and sizes, with different pricing models, target audiences, feature sets and use cases. Menlo Technologies studied a variety of mobile device clouds and has summarized some of the popular options for you:

1)  Xamarin Test Cloud

Acquired by Microsoft last year, Xamarin enables cross-platform app development using the C# programming language.

  • Company's Service Description: Find bugs before you ship with automated tests running on the devices your customers use. You can test everything your users can do, from pinching and swiping to double-tapping and scrolling. Find performance problems on any OS automatically with step-by step memory and performance tracking. Beautiful reports make it easy to know when you're ready for launch.
  • Pricing: Plans with varying levels of limitations and capabilities include: Newbie, $99/month; Small Startup, $379/month; Growing Business, $799/month; and Enterprise, contact for a quote. A 30-day trial is available.
  • Available Devices: It currently lists 2,624 devices, running iOS 8 through iOS 10 and many Android versions from Honeycomb to Jelly Bean.
  • Notes: Developers can write tests in C#, Ruby with Cucumber, or Java with Appium. Xamarin Studio and Visual Studio can be used for C# tests, and Xamarin Test Cloud templates are integrated into those IDEs. It can integrate with standard NUnit test runners. Code can be shared for cross-platform tests between iOS and Android. For automated tests, Xamarin engineers can provide hands-on coaching to get projects started. Functionality includes changing GPS locations, pressing physical buttons, activating device cameras and rotating the device between portrait and landscape modes.

2)  Test Object from Sauce Labs

At Sauce Labs, mobile testing is just one of a host of services, including an open source automated testing platform, cross-browser testing, JavaScript unit testing, professional services and more.

  • Company's Service Description: Test your mobile apps across iOS and Android real devices, emulators and simulators on our public cloud or on dedicated devices in a private cloud. Run automated tests to accelerate your releases and perform manual spot checking to isolate issues quickly. Access hundreds of different devices without the hassles of procurement or maintenance.
  • Pricing: Many different models are available for monthly or annual pricing plans. For the monthly plan, options include: Manual, $29/month; Individual, $199/month; Expert, $399/month; Professional, $499/month; Small Team, $699/month; Enterprise, contact company. A 14-day free trial is available.
  • Available Devices: 308 devices are available, featuring Android from version 2.2.2 to 7.1.1 and iOS from 7.1.2 to 10.0.
  • Notes: Both real devices and simulators/emulators are available. With last year's acquisition of TestObject, it supports manual real device testing and secure testing on private clouds. Company claims it's the "only test cloud that lets you automate your native, hybrid and mobile Web tests across emulators, simulators and real devices." Developers can run automated mobile tests across native or hybrid apps with Appium (cross-platform framework for iOS and Android apps), Espresso (Google's framework for testing Android apps) or Robotium (providing a record and playback test case creator for Android apps). All are open source frameworks.

3)  AWS Device Farm
This comes from Amazon Web Services Inc. (AWS), the leading cloud computing platform.

  • Company's Service Description: AWS Device Farm is an app testing service that lets you test and interact with your Android, iOS, and Web apps on many devices at once, or reproduce issues on a device in real time. View video, screenshots, logs, and performance data to pinpoint and fix issues before shipping your app.
  • Pricing: Three levels are offered: pay as you go, $0.17/device minute, with first 1,000 minutes free; unlimited testing, starts at $250/month; private devices (test on dedicated devices deployed exclusively for your account), starts at $200/month.
  • Available Devices: 206 Android and Fire OS devices; 152 iOS devices. An interactive device list is available.
  • Notes: Two primary testing methods are provided -- automated testing on a variety of frameworks and remote access to devices on which developers can load, run and interactively test Android apps in real time. It tests native and hybrid apps, including apps created with PhoneGap, Titanium, Xamarin, Unity and other frameworks. If developers don't have their own tests, a built-in fuzz test is available. Tests are run on real, non-rooted devices, a mixture of OEM and carrier-branded offerings.

A different approach to Mobile Testing: Crowd testing

Crowd testing provider Applause has a global, on-demand network of software experts that manually test your software, and provide you rapid, actionable feedback. Applause gives you the direction to develop the applications that best satisfy consumers; making the whole application development process a success.

  • Company's Service Description: Applause connects your apps, websites, IoT products and in-store experiences to uTest, the world’s largest community of digital experience quality experts.
  • Pricing: Applause requires you to fill out a pice inquiry form or a price estimator form here.
  • Available Devices: A list of devices available on request.
  • Notes: The testing features are immense result drivers with regards to the application development.  The feature of mobile sentiment analysis helps you to gauge the likability of the applications by the users and how well they review the application product on the Applause Score. The comparative analysis and benchmarking are useful for gearing up the process. With security testing, you can know how the applications work or will work on the standards of security. This is another feature that consumers are always looking out for. If the application is secure enough in its dealings and offerings, only then they are expected to use the application.

Do you want to learn more about testing your app?

Let Menlo Technologies take care of you

We’ll help you sort through these testing clouds so that you can choose which one makes the most sense for your business.

Talk to an expert

Topics: Mobile App Testing, mobile device test cloud

Laura Shumaker

Written by Laura Shumaker

Laura Shumaker is on the marketing team of Menlo Technologies. She's been writing about health and tech for over 10 years.

Categories

see all

Subscribe to Email Updates