When digital is blended with reality

Welcome back, Cybercultural subscribers! This week’s theme is how the virtual world is increasingly crossing over into the physical world. From James Dean being digitally reincarnated for a new film role, to the “Touch Video” technology of game developer Flavourworks, to the rise of immersive news, to a newly announced Pokemon GO developer platform, in 2019 digital technologies are mixing with the real world in strange new ways.

  1. James Dean Reborn in CGI for Vietnam War Action-Drama 🎥

In recent years we’ve seen dead music legends Roy Orbison and Tupac Shakur play “live” concerts, in the form of holograms. That was bizarre enough, but news that James Dean - who died in 1955 aged 24 - has been “cast” as a major character in a Hollywood movie has upped the ante on digital reincarnation. Hollywood Reporter stated:

Directed by Anton Ernst and Tati Golykh, the project comes from the filmmakers' own recently launched production house Magic City Films, which obtained the rights to use Dean’s image from his family. Canadian VFX banner Imagine Engine will be working alongside South African VFX company MOI Worldwide to re-create what the filmmakers describe as “a realistic version of James Dean.”

[…] While Finding Jack will be live action, The Hollywood Reporter understands that Dean’s performance will be constructed via "full body" CGI using actual footage and photos. Another actor will voice him.

After a backlash on social media, the directors defended themselves in a follow-up Hollywood Reporter story:

When discussing whether resurrecting Dean digitally crosses a line with regards to posthumous casting, Ernst explained, "Anyone that is brought back to life — you have to respect them." He noted [Carrie] Fisher's posthumous appearances in the Star Wars franchise, saying that if the actress had expressed never wanting to be in a film after her death, or if her legacy or that of the franchise could be "tarnished" because of her casting, "then that should be a line."

My take: My main issue with the director’s defence is that James Dean could not possibly have predicted he would be “brought back to life” over 60 years later for a new movie, so I think Anton Ernst has crossed a line here. Also, in my view it’s disingenuous to say that James Dean is “starring” in your movie. It’s not James Dean, it’s a digital facsimile. The reason why Dean became such an iconic movie star is because of his highly individualistic character - what was inside of him. This digital version will be an empty vessel, so it feels like a marketing stunt to say it’s James Dean.

I’m not completely opposed to virtual versions of deceased famous people, provided it was either consented to or (if that wasn’t possible) it allows something unfinished to be completed respectfully. An example of the latter is when Brandon Lee tragically died aged 28 while filming The Crow in 1993. Lee was accidentally shot on set, near the end of filming. As described in Wikipedia:

The incomplete scenes which were to form the beginning of the film had to be rewritten. The scenes were completed using computer graphics and manipulating scenes already filmed of Lee.

It was a horrible tragedy, but it felt respectful to Lee and his family to use CGI technology to finish the movie. Particularly as Lee’s performance in The Crow was incredible and deserved to be seen.

While James Dean also died young in tragic circumstances, this new movie is so far removed from his life and era that it feels exploitative to use his image in this way. The use of digital technology that supposedly brings people back to life must be used very carefully, either with consent or in a fully respectful, non-exploitative way. (I don’t think that was done with both Orbison and Shakur either, by the way.)

  1. Erica developer Flavourworks has secured funding to expand its cross-platform Touch Video technology 🎮

Gamasutra reports:

Flavourworks rose to prominence earlier this year with the launch of Erica, a full motion video (FMV) game that combines live-action footage with tactile choices to deliver a movie-like gameplay experience.

The UK studio describes its Touch Video tech as "a combination of behind-the-scenes innovations, novel interaction methods, and the endless possibilities of an intuitive visual editor [that] allow us to deliver playable filmed worlds to wide audiences like never before."

My take: I chose this news story because it’s an example of gaming that uses filmed footage of the physical world. Or is it the other way round? A press release states that Flavourworks is “bringing interactive stories to filmed content.” Either way, it’s a merger of game and video. The company’s first product, Erica, is available on the PlayStation 4 now, “with mobile launches following soon.” Erica has a run-time of a feature length film.

So why is the technology called ‘Touch Video’? Because the game “allows the player to touch the world using the touchpad on Sony’s Dualshock 4 or a mobile device screen.” One example given is wiping a tear off a character’s face.

Ultimately, Erica will be judged on the quality of its story and movie-making. So in that respect, it seems more like a movie than a game. I think the first truly unique game/movie mashup will eventually be done in VR, which will allow a fully immersive experience. It’ll likely be something completely different from a movie and a game - something entirely new. Erica doesn’t feel like that, but regardless I like that Flavourworks is experimenting with bringing game-like features to movies. Check out the video below if you’re interested:

  1. Yahoo News Kicks off Immersive News Initiative With Camp Fire AR Story 🗞️

Variety reports:

Yahoo News published an augmented reality (AR) feature dubbed “Rebuilding Paradise” as part of a new partner program for immersive news content Thursday. The feature takes viewers inside the reconstruction process for a home destroyed in last year’s deadly Camp Fire in Paradise, Calif.

“We want to put the user in the middle of the story,” explained Verizon Media general manager of news Alex Wallace in an interview with Variety. The video features a look at the home as it is being rebuilt, family photos, and audio from an interview with the homeowners about their decision to stay in Paradise.

My take: Alex Wallace also told Variety that “the future of journalism is going to be XR,” the industry term for augmented and virtual reality storytelling. Certainly this technology helps viewers be more empathetic towards the victims of the fires. Given the amount of bad news in most general news shows and websites these days, enabling more empathy seems like a useful feature. The drawbacks are that XR is a time intensive and (at this point) expensive way to produce a news story. But for insightful feature stories after the fact - as ‘Rebuilding Paradise’ appears to be - or for in-depth documentaries about a news story, XR looks like a highly promising technology for the media business.

  1. Pokemon GO maker announces ‘The Niantic Real World Platform’ & $10M creator fund 📱

Niantic, the company behind AR game sensation Pokemon Go, has just launched a platform for third-party developers. In addition, the company announced a $10 million fund for developers. From Niantic’s blog:

Being able to share the Real World Platform with more developers is the realization of a dream we’ve had since the earliest days of development - to make Earth an AR Gameboard that allows developers to bring their creative visions to it. We believe AR will be the medium of tomorrow; transforming how people interact with the world, and each other.

This promo video gives a taste of what might be built:

My take: I think we’ll be seeing a ton of awesome new AR apps come onto the market over 2020, as 5G networks ramp up globally. I’m personally more interested in AR apps that enhance real-world cultural content - like museums, art galleries and historical tourist attractions. Being able to interact with culture and find out more about (for example) an art work or a historical artefact is a promising use case for AR. Hopefully that’s something that can be done using Niantic’s platform; although that platform seems most suited to gamified AR. Regardless, this is great news for AR developers and (eventually) for we consumers.

Data Points 📊

  • CISAC (the International Confederation of Societies of Authors and Composers): Worldwide royalty collections for creators of music, audiovisual, visual arts, drama and literature reached a record €9.65 billion in 2018 ($10.74 billion). 💰

  • Meanwhile, in gaming… NPD Group: So far, game spending is over $27 billion for 2019 in the US. 💰💰💰

  • Statista Digital Market Outlook: mobile will account for 50 per cent of ad spending by 2021 and finally surpass desktop in 2022. 📱

  • Common Sense Media: Online video use among tweens and teens surges. 📹

  • World Press Trends 2019 report: digital news subscriber numbers worldwide have increased 208% over five years to 2018. 🗞️

Tweet of the week 🐦

Brent Spiner, famous for playing an android (Lieutenant Commander Data) in the television series Star Trek: The Next Generation, weighs in on CGI James Dean:

That’s a wrap for this week’s subscriber newsletter. Ping me any feedback you have, or suggestions for themes in upcoming editions. Thanks as always for your support, it’s much appreciated. 🙏