So this year the BBC has done something exciting for the 2018 FIFA World Cup: for the first time all the games we broadcast on BBC One have been available in virtual reality.
For the BBC it’s important to innovate and explore new experiences for audiences now and in the future. We saw this as the perfect time and opportunity to test what our audience expects from a virtual reality sport app. There were a number of areas we were keen to learn more about including audience expectations, device preferences, the amount of time people spend using VR, how popular the platform is and what the demographic of a typical VR is.
To take a look into this we have partnered with FIFA and its official VR suppliers in LiveLike and Sony Interactive Entertainment Europe (SIEE) to create stunning immersive environments our s can dive into and watch the action. Partnering with these companies allowed us to learn a lot whilst providing a great new platform for our s and so far nearly 350,000 people have taken advantage of this opportunity.
In this post we are aiming to show the work that has gone into launching this app and what we have learnt along the way.
How we delivered the apps: by Rebecca Heapy — BBC Sport Apps Project Manager
Six months to customise an out of the box solution — easy right? Then as the weeks tick over you realise there is much more to it than a collection of posters and a couple of end-to-end tests. You’re now working with more suppliers based in multiple geographical locations. Internally the number of teams / departments involved has also grown as you’re…
- setting up CDN networks
- settling contracts and agreements
- ensuring security standards meet our levels
- prepping editorial team on the new content interfaces
All this on top of ing and developing the BBC Sport App, one the biggest Sport apps in the market.
So how can you ensure successful delivery when so many elements are out of your control? Without sounding cliché, communication was key.
Initially, direct with the teams building the interfaces was limited and irregular. The first obstacle we had to overcome was establishing governance and routines with all parties, fostering transparency and collaboration. Forging a strong direct relationship with the development team was key and helped us gain a better understanding of our role in this project and get closer to the product itself.
When equipped with more knowledge we were able to efficiently customise the environments, promotional elements and (probably the biggest of all our tasks) test the app. It was paramount that before releasing, before even announcing the apps, that we were 100% confident in the application and the services that feed in to it so that it meets the standards our audience expect.
How we made sure they worked: by Vinny Sebastian — BBC Sport Apps Senior Test Analyst
Whilst we have a lot of experience testing the BBC Sport app, we do not have much experience testing in VR. Testing a VR app from a third party, not only involves looking at the app but also ensuring the mobile and console hardware (cardboards, headsets etc..) each display the app correctly to the and checking the stream and on demand video quality is acceptable and plays. During this experience many lessons were learnt from our testing as the VR app was delivered.
We had to base our testing on specifications, unlike working hand-in-hand with the development team like we are used to. After analysing the spec, we needed some clarification before we were happy to start our testing. We were also given a document with acceptance tests which were based on journeys that we used for our testing.
The end-to-end test sessions lasted for approximately two hours and covered testing the stream, branding, video on demand, data and camera angles in the stadium.
In our first end-to-end session we had eight people independently testing the app across a selection of devices. People were going through the test scenarios at their own pace, some were filling out spreadsheets while others were shouting out the issues they found, which resulted in a disorganised round of testing.
As a result, we decided that we would go through each journey together and with one person scribing the issues that were found.
From then on the test sessions ran more smoothly, however, we took part in a further six tests rather than two tests due to issue found along the way. This extra vigilance ensured we delivered a product of the highest quality.
Learnings
The game’s changed!
Two years ago for the 2016 Rio Olympics we first dipped our toe into immersive / VR technology. In the two years since, not only the headsets but the software / environments have improved dramatically. This is also reflected in the usage with over four times more people ing the World Cup app compared to the Olympic app. The quality of the lounge, which is central to the app and how s navigate, is fantastic across all platforms and something our s have picked up on.
The camera angles available are great, but there could be more…
As the tournament progresses we have been conducting testing and monitoring . The s of the app love the multiple camera angles but some would like even more control. We currently have cameras set up in the stand and behind both goals but s would like to see this expanded along with the ability to have more control over the live feed.
Same app different s
The app has been a great success in of s and one area we have seen significant growth is in smartphone s. However these s have the lowest average session time. When looking at s who invested in headsets like the Oculus Go, PlayStation VR and Samsung Gear VR they spend far more time watching than smartphone s.
When thinking about VR in the future we need to take this into e.g. should a smartphone VR experience be based around a second screen or highlights and headset s around live games?
Same app different s
People don’t want to be left out, ever
Whilst working with our partners we ed all devices available to us so we could learn as much as possible. Unfortunately not every VR device or headset was ed as this was very much a trail, and the findings taken from this will feed into the decisions we make around device in the future.
What next…
At the moment the experiment continues until the World Cup finishes on July 15th. After that we will take the learnings from this already successful experiment and factor those into whatever might be next. But we know, there is certainly an appetite for VR and Sport to work together.
If you haven't had the change to make the most of the World Cup experience yet, you can the app from the below locations:
