The evolution of new technologies has facilitated the development of the collaborative economy, and in turn, the emergence of a new concept: Crowdsourcing. The general idea when using the English term “crowd” is to present a concept in which a large group of people participate. In this specific case, Crowdsourcing refers to the fact that the functions that a company used to carry out internally are now externalised, leaving them in the hands of a large group of people.
Recent years have seen a growing interest in concepts such as Crowdfunding, Crowdlending, etc. And now, these concepts are being joined by a new one: Crowdtesting.
What is Crowdtesting?
Crowdtesting is simply letting software evaluation tests be carried out by people outside the project, and the more people who carry them out, the better.
Crowdtesting is carried out on mobile applications, web pages, programs, etc. in general on any software that users can access from their devices, as it is the users themselves, with their physical means, who are responsible for assessing whether the software analysed contains errors, usability problems or any vulnerability that makes it weak.
When Crowdtesting is considered as an evaluation method, it is done with the aim of having the largest possible evaluation group, in order to have a greater critical vision and find all possible errors. In short, to have the intelligence of the group.
How does Crowtesting work?
The way the Crowdtesting is carried out is associated to who will be the users who will carry out the testing.
Usually, the companies that want to carry out this technique create an environment so that the chosen users can have access to the product, and in turn they can have contact with those responsible for transmitting doubts and results.
The choice of users is related to the needs of the project, for example: a specific professional profile, special social characteristics or specific computer skills. Normally, there is a previous base of users who have already carried out tests and are known for their good work.
Once the typical users with whom to carry out the tests have been chosen, the essential points to be evaluated are made explicit. There will be times when these points are very specific, and other times when they are very general, or even non-existent, and you simply let the software be tested, without any indication.
With the chosen users and the key points to be evaluated, the test is carried out, and the details found are reflected in a report, so that the developers can modify everything that is necessary.
What are the benefits of Crowdtesting?
The main advantage that can be obtained when betting on Crowdtesting is that the opportunities to find bugs and errors in the products are maximised by having a large base of testers, who, in turn, if they are specialised in the testing itself, will be able to find more bugs than if the testing were carried out by the developers themselves.
A benefit that goes hand in hand with the previous one is that with the wide base of testers, the devices and browsers where the tests are carried out are expanded, thus making the multiplatform test more efficient. And furthermore, if they are more on the street, the tests are more real, far from a controlled environment such as the company’s own environment.
Another clear benefit is the economic savings involved, or rather, the improvement in cost efficiency, since the way of paying for these tests is associated with the errors found, and in many cases the unique errors that each tester is capable of finding. Therefore, the cost of the test is associated with the quality of the test. And at the same time, it allows, if necessary, to scale up the number of testers in a short period of time.
In addition, Crowdtesting allows for such extensive testing that evaluations of elements that would otherwise be left out of any possible evaluation can be undertaken. A case in point is usability, which, although it would not be left out in any case, would remain in the background, and thus gains weight, by expanding the testers’ base. Another case is that of local testing, with a large base of testers, tests can be extended throughout the world, and results obtained from different geographical points, with their own characteristics and connotations.
The speed of error detection is another benefit obtained by applying Crowdtesting: the larger the testers’ base, the faster problems are detected. There are studies that speak of the fact that, on many occasions, Crowdtesting can discover hundreds of problems in less than two days, provided that a previous internal control has been passed. This is associated to the fact that the diversity of testers makes the path followed to evaluate is very diverse, and therefore multiple paths are explored in less time.
Putting figures to the benefits that are obtained with the Crowdtesting, from the development team can increase a 200% the capacity to carry out tests, can increase a 150% the launches per year, and reduce in a 50% the necessary critical corrections. While, from the business side, planned revenues are accelerated by 30%, customer retention is increased by 10%, and the customer conversion rate is increased by 10%.
What are the disadvantages of Crowdtesting?
The main disadvantage of Crowdtesting is the fact that it finds a basis for correct and effective testing.
In addition, this basis is usually less prepared than specialised testers and interns, so it may be that the evaluation is too superficial, or that, being less familiar with the software, they spend too much time learning how it works and find fewer errors.
In connection with this, the questions asked to the testers must be clear and concise, especially in the case of usability, otherwise there is a risk that the evaluation carried out may be too light.
Another problem associated with Crowdtesting comes with the very idiosyncrasies of the process, in which the tester is paid according to the test performed, and the more unique the errors reported, the higher the payment received. This can lead to demotivation and lack of commitment as the process progresses. If the Pareto principle is applied, 80% of the time will be used to discover 20% of errors, therefore, to discover more errors, a very important amount of time is needed, with the risk of tiredness and abandonment that this entails.
A risk that is also taken with Crowdtesting is that there is a high turnover in the testing team, thus preventing a greater depth in the results. And related to the principle of Pareto that was cited, never reach a great depth in the errors found.
Finally, a clear disadvantage is the loss of confidentiality: the more people know about a product before they go out, the greater the risk that its existence will be leaked, and worse, that some innovative functionality will be leaked.
Crowdtesting in the context of COVID-19
The arrival of COVID-19 has changed the reality of companies. The need to implement teleworking, to bring products closer to customers remotely… has led to a definitive commitment to digital transformation, which, although in many cases was already in the pipeline, this year has accelerated in order to continue working and reaching customers.
This acceleration has resulted in more developments and products to launch, and therefore, to test. Given this reality, the testing teams cannot absorb all the necessary tests, and that is why, this year, there has been a growing interest in Crowdtesting. In addition, the need for social distancing has helped to encourage this form of testing.
A clear example of this accelerated reality is the development of the COVID-19 tracking application carried out by Israel. In view of the need to develop the application in the shortest possible time, the company in charge of carrying it out opted for carrying out its testing through Crowdtesting, thus allowing tests to be carried out on all types of devices, locations and contexts.
But beyond the needs explicitly related to the COVID-19, there are other examples of use: in the automotive sector, retail, travel… Sectors all of which have been negatively impacted by the pandemic, and which, instead, have opted for the development of solutions and have made use of Crowdtesting to speed up processes.
A new boost to Crowdtesting companies
It seems clear that companies have needed a plus in order to be able to take forward their innovative needs; and as we have seen, this plus has been supported, to some extent, by Crowdtesting.
This is why companies specialised in providing Crowdtesting services have benefited. They have seen how the demand for their services has grown: they are companies with knowledge of the necessary processes, with personnel adapted to the demands of the development companies
Conclusions
The arrival of the COVID-19 has accelerated and put into value the Crowdtesting, a trend that until now had gone more unnoticed. Before the COVID-19, companies were already relying on crowdtesting, but the need to develop products urgently meant that more and more companies were opting for it.
Among the benefits, in addition to speeding up processes, are improved efficiency in finding and reporting errors: the greater the number of testers, the greater the chances of finding and correcting errors. On the other hand, among the disadvantages, we can mention the need to define, beforehand and very clearly, the elements to be evaluated and tested.
In short, a trend that until now was gradually gaining ground, and depending on the type of project to be tested, has seen the arrival of the COVID-19 give it the definitive push to move to the mainstream and establish itself as another element in the software product development process.
