During the insanity of COVID-19, live streaming seems to be growing in popularity at a faster rate than ever before. Last night, my girlfriend, who works as a content developer for Twitch, helped orchestrate a live stream concert for popular DJ, Diplo. Throughout the live stream, there were consistently 15,000 unique viewers tuning in. The concert took place in Diplo’s living room, where he had a very basic set up but nonetheless managed to entertain thousands of people remotely. I genuinely had fun watching the stream, and almost felt like I was at a concert.
The event got me optimistic for the future of live XR concerts and gave me inspiration about exactly what that would look like.
There are two methods of creating a hologram for a musical artist that immediately come to mind. The less expensive way to do it would be to have the artist perform in a motion capture suit and translate their movements to a pre-rigged model of the artist. The more expensive (but better) method would be to have the artist perform in a volumetric capture booth. In both cases, the audio would be recorded and output directly into the stream. Both the audio and hologram would be fed into a real-time engine. I should also note that video could also be used pretty powerfully for VR, but wouldn’t work for AR.
The artist could perform in a digital venue of their choosing. Users enter the venue as avatars and a variety of viewing experiences could easily be implemented.
- Users could view the hologram privately or with friends
- Users could view the hologram with the avatars of all other users at the virtual concert. This brings about the question of where the user will watch the concert from.
- Would more expensive tickets be sold for spots closer to the hologram?
- Would spots be determined on a first come first serve basis like at traditional music festivals?
- Would the user be able to navigate the venue in a way that does not obstruct the view of other avatars (e.g. hovering transparency mode)?
A digital artist would likely be necessary to implement additional generative animated effects to spice up the performance. There is truly no limit to the potential visual spectacles that could take place within such venues.
Through AR, users could put the hologram in their living rooms or any other real-life venue. This would mean that as long as participants have AR compatible devices (ideally glasses), they could view the hologram in the same location as the other viewers in that location. This would mean that the performer could potentially perform at infinite different venues at a single time. With a single hologram stream, they could simultaneously perform at a music festival, club, living room, bedside, etc.
The popularity of live streaming has already proven that musical artists to do not need to be in the same location as viewers to provide compelling live performances. Through VR and AR, such performances could continue to grow enormously in entertainment value. Both have the potential to provide concert experiences to audiences from the comfort of their own homes, which has never been more important than it is today.
Travis Scott Concert in Fortnite
One of the most innovative examples of a concert taking place in a 360 virtual environment is the Travis Scott concert that recently took place in the video game, Fortnite (late April 2020). This concert experience has been very powerful for a lot of people because it has been able to create visuals never seen by viewers before. Because the experience was created in Unreal Engine, it could have just as easily been made in virtual reality. Because of the popularity of this event, I imagine that similar concert experiences using game engines will be very popular in the near future and will become commonplace within no time. These concerts could possibly benefit even more from the performances being live (through motion capture), actually taking place in VR, and larger viewing areas capable of hosting larger groups of viewers.