How SoundCloud approaches user testing for 175 million users

With over 175 million users worldwide, SoundCloud is the world's largest music and audio distribution platform. Despite this many users and a global footprint, SoundCloud understands that despite having large amounts of quantitative data on user behaviours, qualitative testing is still much needed to create the best possible product. This case study covers how SoundCloud leverages continuous user testing in their product development cycle.

Finding usability issues

Issues related to usability are impossible to find using quantitative analytical tools or automated test cases. That's because quantitative data only shows whether a specific transaction happened or not, but not how the user got there, or whether they got stuck while doing so. By conducting usability testing with users, they were able to identify areas that can be improved to make a user's life better and for them to be more at east and accomplish what they need to do on SoundCloud. 

Testing across devices and contexts of use

SoundCloud runs remote tests live with users across over 22 different countries and over 140+ devices each time. This is to ensure that the feedback that they get from these tests are representative of their global user base, and the diversity in devices that they use. This helps them find and resolve issues that happen on specific devices and not others, and also uncover regional differences such as how the same functionality is represented in different languages. To make this happen, they leveraged remote user testing tools like Lightster,

Finding functional bugs

While SoundCloud has a large team of internal quality assurance professionals who run through thousands of test cases,  found over 150 issues, 11 of which were critical issues. Only through qualitative can they can identify issues that only a real human would find, in a real context of use.

SoundCloud uses live user testing with real users continuously through their product development lifecycle, an activity which supplements their core quality assurance function. Doing so helps them identify usability issues and opportunities, find bugs in the product from real-world usage, and also helps them ensure there are no issues across different devices, form factors, and locations.