Somebody might say it’s not possible, but of course it is. It’s all about the stretch! Problems and the solutions are unique that’s the fact. The catch is we just need to find the right places where we need to stretch Bitbar Testing to match your unique use cases. There’s no one tool to address all of the aspects, but the game changes if you have toolset in your hands. For us the toolset is Bitbar Testing. Let me elaborate the steps we take in our stretching process. The first step here as in any successful project is analysis.
The Good Questions
There are several types of problems customers might want to solve. It would be great to have the answers available in the beginning, but in real life, we need to begin from the right questions. The Bitbar team works the same way as the best journalist does for information-gathering. It’s following the 5Ws and 1H principle: Who, What, Where, When, Why and How – and the questions are as follows:
- Who are the users and what kind of roles are present? Developers, QA and managers have all their unique needs and use cases.
- What’s the scope of testing? Unit, functional, non-functional or compatibility?
- Where? Are there some security- or privacy-related issues limiting the deployment options? Are your backend systems available for public?
- When? What phase is your app in the R&D process? Do you need to sync the real device testing with your continuous integration? Or is it just the final step supporting the manual testing process? Or would you like to adjust your product based on Google Play feedback?
- Why? What’s the problem you want to solve? Speed, accuracy, volume or coverage? Do you want to find out UI discrepancies on different HW or OS version? Or maybe you validate and measure your application behavior on a reference setup?
- How? Are there any preferences for testing frameworks? What’s the testing awareness and the target state of your team? How much external help do you need and prefer?
These questions are good to start with. Quite often some of them cannot be answered right away, but they help us all to synchronize the expectations. It’s much easier to start to negotiate about the speed when we know where we’re aiming at. It’s also important to understand we’re setting the target for the beginning, but we need to be prepared for changes in the future. It’s quite common the requirements and the objectives keep changing along the process and with the lifecycle of the application. Like every project, we set the goals and follow up the progress with the milestones.
Bitbar Testing is an execution platform consisting of the best of breed open source test automation frameworks. In practice this means we’re not bound to single truth or a solution. We cherry pick the ones matching the needs the best. This enables us to get started quickly with minimum upfront investment. We use our solution building kit to prepare the proof-of-concept solution in a couple of days. Typically this is a dedicated environment having a bunch of the most relevant devices available without any restrictions. It’s important to keep focus and feedback loop fast to ensure both technical and usage questions are solved promptly. This helps in identifying the essential parts for the value add from the ones having secondary priority.
Functionality is one side of the coin. Availability and the support are on the other side. Customers are able to evaluate the importance of non-functional requirements during the hands-on trial. In this phase, it becomes obvious how much own effort the customer is willing to use for the support and maintenance activities. There is often hidden cost in the slide deck and spreadsheet evaluations.
Finally we have completed the first phase of evaluation. The work starts here. At the best, the first iteration round takes only a couple of days. This is a good starting point to understand the use case and find out the gaps in the solution. It’s also an important learning opportunity for both teams to synchronize the working methods and to find the right pace of working.
The Bitbar team will make the needed changes to the solution. In this phase, it can be a temporary fix for iteration purposes. The final solution will be added to our development backlog for detailed solution analysis and schedule. Customers can also adjust his use case to match better to capabilities of the solution. We’ll update the documentation together and we’re ready to take another iteration round.
Proof-of-concept takes typically a couple of iteration rounds to complete. Once the solution is ready we can proceed to last step to adjust the expectations of the non-functional requirements. This is selecting the correct deployment option and needed support and maintenance model.
In the end we’re all set for the GO-decision. The best thing is we have not only a plan, but a proved to be working a small-scale example of the to-be state. I’ll elaborate on the execution phase in my next post in 4 weeks.
Happy Testing folks!