Bitbar customer case study: PayPal

Bitbar Testing Makes Devs Lives Easier

PayPal allows users to make and receive online, mobile and store payments anytime, anywhere and in any way. The service offers users simpler ways to send money without sharing financial information and with the flexibility to pay using their account balances, bank accounts, credit cards or promotional financing. With 128 million active accounts in 193 markets and 25 currencies around the world, PayPal enables global commerce, processing more than 7.6 million payments every day and an expected $20 billion in mobile payments in 2013.

In May 2013, PayPal released the PayPal mobile SDK for Android in the USA, and is planning a wider release later in the year. The SDK is designed to provide a native Android experience that simplifies accepting credit card and PayPal payments in a developer’s mobile app. The library presents a simple user interface, features (owned by PayPal) credit card scanning and uses a proof of payment system that eliminates Payment Card Industry (PCI) compliance worries.


PayPal releases across many different currencies, countries and languages, an ongoing and complex task for its mobile developer teams. “An SDK handling money needs to work all the time, and people need to have confidence in it,” said Tom Whipple, mobile engineer and Android lead developer at PayPal. “It is therefore important for the developer team to test on a wide variety of devices. Android has many different screen sizes and there are numerous versions available. The majority now run Android 4.0 or higher whereas only six months ago this wasn’t the case. Managing all the different combinations is extremely difficult.”

Therefore, the development team needed a testing solution which would enable them to ensure that each release works across a wide variety of devices. As the company releases into an increasing number of non-English speaking markets, this need for testing becomes even more crucial.

Tom Whipple and his developer team were already familiar with Bitbar and its cloud-based Android testing solution—Testdroid Cloud—the first automated, real-time testing tool for mobile application developers. His team had used Testdroid Cloud to test the low level camera access of Initially unaware of Testdroid Cloud, the team received crash reports and did not understand what was happening. Valuable time was spent exchanging emails with users, taking up to a month to fix each issue. Once they began using Testdroid Cloud, this timeframe was shaved down to just a few of hours.

Based on this experience, in early 2013 the team began testing its developer-facing SDK using Testdroid Cloud integrated to their continuous integration system with the Testdroid Jenkins Plugin. This enabled them to run tests on different devices on a continous basis using a ramped-up, layered approach that complemented their own manual and local testing.

PayPal’s developers began with a basic run of testing that involved a combination of manual tests and tests run on a local continuous integration server. “Manual testing is part of the development process, as a developer will want to manually test the part of the app on which he or she is working. However, there are limits. Only a small amount of manual testing is possible since tests are time consuming and difficult to exactly replicate,” added Tom Whipple.

PayPal’s development team followed this initial manual testing with their own set of tests that were run on a local continuous integration server. Once successful, the testing was passed on to Testdroid Cloud, where tests were run on over 200+ phones on an as-needed basis, sometimes several times a day. With every change to the code, automated tests enabled continuous feedback. Within a few short hours, testing was completed and followed by a report that was generated and delivered to the development team detailing issues that were detected during the testing and any fixes that need to be made.

“We have a drawer full of phones, but we are also of the mindset that if tests can be automated, let’s automate them. Testdroid Cloud enables our development team to avoid relying purely on manual testing, which is next to impossible without a large team. In the long run, automated tests allow us to release a more stable and reliable product for our customers, resulting in fewer bug fix updates and an overall better impression about our company,” concluded Tom Whipple.

To learn more about PayPal’s Android SDK, visit or


Tom Whipple is the engineering lead for the PayPal Mobile SDK team, which was formed from PayPal’s acquisition of in July 2012. At he led development of the Android SDK and fraud prevention efforts, as well as contributing to iOS development. Tom joined from a software consultancy he started in Ventura, California.