Skip to content

Last month I covered few topics on how we’re running our real device cloud and in that blog, I sidelined briefly the built-in monitoring that provides us vital element to ensure everything is working as expected and users can get the best customer experience through the service. As there is a myriad of moving components, attributes, that all must be maintained and checked whether everything is up in speed, we have to rely on our internal monitoring implementation to keep everyone up-to-date with the status. Actually, this is not whole lot different thing if you are running your own cloud device farm with Bitbar On-Premise as the tool provides some neat capabilities for monitoring test runs, queues, devices, infrastructure, capacities and so forth.

devops insights into on-premise cloud device farm

In this blog, I’ll be walking you through one example that we do pretty much every day here at DevOps team: adding a new device to our cloud device farm. This is pretty common procedure but I hope it will give some insights on how to do it, what things you should consider if you do it in your on-premise device lab, and what things are specifically under monitoring for each device – 24 hours per day.

Adding a New Device to On-Premise or Cloud Device Farm

That may sound trivial, and it actually is. But still there are few things that must go right – or otherwise, devices can be flaky and highly unstable to its users. As we support Android and iOS devices, working with ADB and Xcode are naturally every-day-business for our DevOps team. In addition, there are different measures – and sometimes hurdles – with the configuration we have to go through to enable new devices in our Bitbar Device Cloud. And the very same thing applies to Bitbar On-Premise as well.

If you are not operating your own cloud device farm you probably don’t get to see this view at all. This is the main admin view snapped from Bitbar Testing that gets us to the core of running the entire real device cloud. From the system status through device configuration to basic communication of the status of real device cloud is something we can access through this view.

As there can be hundreds of test runs initiated just during one minute it’s very important to get an instant understanding of performance capabilities, status of test runs, devices, clusters and other information that gets DevOps folks all possible details of misbehaviors, alerts, and statuses of the system. Furthermore, this is the view where we control each test run and take measures if something doesn’t work as expected.

test run queue on-premise cloud device farm

The above view works for test runs but when we monitor the status of each device we use our Device view. The important information for running and keeping these devices online is the status of power, in which cluster it is connected to, and when the device was last time seen. Furthermore, we can get exact numbers of the battery power/status, WiFi connection (disabled, enabled) and accounts connected to the device. The ‘healthy’ status gives us vital information how has the last test runs finished and if something is failing, we can get exact output logs of runs, performance and other relevant data to fix the device to be online/working again.

Screen Shot 2016-02-23 at 12.51.29 PM

Okay, let’s take a look at the steps of how to add a new Android device in this kind of cloud device farm.

Prerequisites to Prepare Android Device for Tests

Here is an example of a checklist we typically go through when adding a new Android device on our cloud device farm. The very same instructions also work for on-premise device labs if you use Bitbar On-Premise. First, we’ll go through some basic procedures that are related to hardware and make sure that all relevant software is properly installed on that device.

  1. Set WiFi network.
  2. Add Google account to the device. Remember to uncheck auto-sync for the account.
  3. Install / Update Google Play Store to the latest version.
    • Without Google Play Store installed, the device won’t have the required services which could be critical for some of the testing scenarios.
    • Start the Play Store once and check possible notifications. Typically there are few so check them and close the app.
  4. Install Google Play Services
    • Open Google Play Store and search “Google Play Services”
    • NOTE! Google Play Services is updated quite often. Without upgrading your device will get notifications about a need to upgrade Google Play Service
  5. (Optional) Install Google Play Games
    1. This is required if – for example – you are testing games or your app depends on it.
    2. Open Google Play Store, search “Google Play Games” and install it.
  6. Disable security checks for installing applications through a USB connection.
    • Enter “Development options” in Device settings
    • Check “USB debugging”, “Stay awake” and “Allow mock location”
    • Uncheck “Verify apps over USB”
    • Enter “Security” in Device settings
    • Check “Unknown sources”
    • Uncheck “Verify apps” (This is not necessarily available with all versions of Android)
  7. Disable Screen Lock
    • Enter “Security” in Device settings
      • Go to “Screen lock” settings and select “None”
      • If this is not possible, you need to configure the device to be unlocked from Device settings on the Admin pages.
  8. Set correct Date & Time to device
  9. Set all “Location Services” on
  10. Install Google Chrome Browser (Optional)
    • Install Google Chrome Browser, if your tests need it
    • Sign in to Chrome browser to make it work robustly on tests.
  11. Set Screen timeout to a maximum value
    • Enter “Display -> Sleep” in Device settings
    • Select maximum timeout value for sleep.

test run dashboard

Connecting Android Device to Device Server (aka Device Cluster)

We have an ebook available on the physical setup of a device lab so check it out for more information about the different possibilities and features with servers and additional hardware.

  1. Connect the device to Device Server using a USB cable.
    • Check possible “Trust” dialog
      • After first connection to Device Server Android device will show notification dialog about “Trust this computer”
    • The device should be available at Jenkins
      • Check the “Android Devices” page from Device Server Jenkins
      • The connected Android device should be shown as “Online” state.
      • Install Device Service (TDS) application to the device using the Jenkins “install” option
  2. Configure new device on the Admin pages
      • Use the device name to find the correct device from the Admin page
      • Open Device page for the device
        • NOTE! Device item is specific for each physical device
        • Change device name to something you prefer
        • Select device init behavior (SKIP / REBOOT / REBOOT_WITH_UNLOCK / UNLOCK ONLY).
        • Set labels for the device (If you have multiple similar devices, it is a good idea to use a specific identifier on the device name). In the following view, admin can add multiple tags for the device that will be shown in the device view for all users. For example, if we upgrade the Android OS version, we first do it manually on that device, then changing the tag in this view to reflect the correct OS version / API level:

    cloud device farm - sony xperia

    • Open Device Model page by clicking the device model name on the Device page
      • NOTE! Device Model (by default) will include all similar devices with the same Android version.
      • Set image for the device model (This image will be displayed in Bitbar Testing UI for this Device Model)
  3. Run the first test with the new device
    • Create a device group and select the new device to it.
    • Create a new test run with some application & test you know will work
    • Check the results

After this, we typically go through troubleshooting if results do not get automatically shown up in our system. Well, when working with physical real mobile devices, so many things can go wrong in the initiation, running phase or communication and fetching results back to our servers. This is the next topic I’ll be blogging about so stay tuned and please let me know if you have any questions or topic you’d like to know more about.

Happy (Robust) Testing!


Article author