Sony Is Working on the Future of Game Graphics for Everyone

Now that the PS5 Pro is here, we know that it does deliver slightly better gaming performance, especially whenever a ‘performance mode’ is enabled. This is largely thanks to the RDNA 2 GPU simply having more compute units and faster memory – but Sony also worked in PSSR, or PlayStation Spectral Super Resolution, to bring AI upscaling to consoles for the first time.

This is exciting technology, to be sure, but the way Sony got here is fascinating, and could have repercussions for the entire gaming industry, thanks to an open approach to who gets to use the underlying technology behind these new features. Because unlike an Nvidia graphics card, the PS5 Pro does not have a discrete ‘Tensor’ core to accelerate its AI algorithm – which potentially means anyone can create a similar neural network, even Microsoft with its next piece of hardware.

No Dedicated AI Hardware?

We recently got a chance to sit down with Mark Cerny and talk about the PS5 Pro’s GPU on a more granular level, and he confirmed that the new console is still using an RDNA 2 GPU, just like the base PS5, but with small alterations to each of the cores. These alterations were implemented by Sony to enable the neural network behind PSSR to run natively on the shader core itself, rather than, as Cerny put it, “adding a Tensor unit or something to it."

I asked Mark Cerny how Sony did this and he said “rather than use a discrete unit for machine learning, the roughly 300 TOPs of processing power was added directly to the WGP (Sony’s streaming multiprocessor)”. Further, he stated that there’s “not much benefit to having a separate unit process the neural network – the rendering pipeline would just stall waiting for the results.”

It’s not a mystery that GPUs are uniquely gifted at powering machine learning technology, but Sony is essentially using the same hardware to render the game, and also to upscale it. I’m not sure what specific improvements Sony has made to the RDNA 2 GPU, but it does mean that PSSR fundamentally can not add quite as much performance as something like DLSS, which does rely on a discrete Tensor unit inside of each Nvidia graphics card.

This is likely because adding a discrete Tensor unit or AI accelerator to the GPU would cause issues for developers who ultimately need to code their games to run on both the PS5 and PS5 Pro. In fact, that’s the whole reason Sony is sticking with RDNA 2 and adding select features from RDNA 3, instead of just implementing an RDNA 3 GPU entirely.

As Cerny explains: “Drivers in the console world tend to be very, very thin. That's viewed as one of the benefits of consoles – that you can take full advantage of all of the hardware features.”

Trying to have a console generation with two disparate generations of GPUs would complicate game development too much. In fact, you can just look at the problems Larian faced when trying to port Baldur’s Gate 3 to both the Xbox Series S and Series X, and those consoles are both using the same generation of graphics – one is just more powerful than the other.

It makes sense, then, why Sony would opt for further customizing RDNA 2 instead of just making a generational jump with RDNA 3. Because while RDNA 3 already has AI accelerators – regardless of whether FSR uses them or not – a new generation of GPU would necessitate a new console generation. And Sony isn’t ready for that move quite yet.

Instead, Sony made some new tweaks, including adding a hidden cache of DDR5 to run the operating system, to let the new neural network have greater access to the 16GB of GDDR6 memory.

"Open For Anyone That Wants to Use It"

One of the best things about AMD graphics cards is they use a lot of software that is open to anyone, regardless of what hardware they’re running. For instance, FSR or FidelityFX Super Resolution will work on any graphics card, be it from AMD, Nvidia or even Intel. So, even if it isn’t quite as powerful as DLSS or XeSS from Intel, developers can implement it and know that their entire install base can use it.

This has made upscaling tech the default for PC games over the last few years, making it much easier to play games at higher resolutions and with more pretty effects. And now that’s coming to consoles. While PSSR is only in the PS5 Pro – it has PlayStation in the name and everything – the underlying technology isn’t exclusive to Sony. Instead, anyone that wants to implement a neural network for upscaling can build off of what Sony has done with the PS5 Pro.

In fact, in his latest deep dive into the tech behind PS5 Pro, Cerny revealed that Sony and AMD are entering into a strategic partnership, called Project Amethyst, with the goal of developing and advancing the future of gaming graphics – for everyone.

“This is not about creating proprietary technology or hardware for PlayStation,” Cerny told IGN. “It can be used by anyone who wants to use it.”

Given that we still haven’t seen a mid-generation refresh of the Xbox Series X, it means Microsoft could theoretically take advantage of this technology, or at least part of it.

What This Means For Xbox And PC

On PC, we already have access to machine learning-accelerated upscaling through Nvidia’s DLSS and Intel’s XeSS, however both of those are locked to their respective graphics cards. FSR is still the only upscaling technology that can be enabled on any graphics card on the market, and if the foundations of PSSR are successful, it could mean gamers get access to a much better upscaling technology.

I’ve been waiting for AMD to announce a version of FSR that works with a neural network to boost its accuracy and quality, and because Team Red worked so closely with Sony on PSSR, it’s not that much of a jump to assume that FSR 4.0 will take a lot of inspiration from the PS5 Pro’s upscaling tech.

This would obviously be huge for high-end gaming PCs running AMD graphics cards, but the biggest benefactor would be handheld gaming PCs like the Asus ROG Ally X or Steam Deck. With those, the amount of horsepower is ultimately limited by the small form factor of the device, plus the fact that it needs to perform under battery power. Right now, the only real upscaling tech you can use with one of those handhelds is FSR, which has limits, especially at 1080p.

Likewise, there have been rumors for a while that Microsoft is working on some kind of Xbox handheld. That’s nowhere near being confirmed, but having a more reliable upscaling tech would certainly eliminate some of the hurdles of creating a handheld able to handle current-generation Xbox games without too much of a visual downgrade.

Is PSSR The Future?

With how ubiquitous FSR, DLSS and XeSS have become, it’s hard to argue against the importance of AI upscaling in gaming. Really, it was only a matter of time before either Microsoft or Sony implemented it into their consoles. What’s amazing, though, is that Sony is approaching it in a way that will facilitate the growth of the technology across the industry.

Because PSSR isn’t being run on discrete hardware, it will never have quite as much of a performance uplift – at least with the same quality level – as something like DLSS that does run on a discrete Tensor core. Cerny confirms this performance cost, stating that there’s “a tradeoff – render at a slightly higher resolution using less expensive non-ML upscaling, or render at a lower resolution and use PSSR. Of course, rendering resolution should not be the primary concern, the ultimate goal is the highest possible image quality!”

No matter how you slice it, rendering a game at 1080p, then upscaling that to 4K is always going to be easier on your hardware than rendering natively at 4K, even if you add an AI upscaling algorithm to the mix.

Hopefully, other hardware manufacturers can take what Sony has started and iterate on it, in order to make gaming at higher resolutions more accessible for everyone, no matter the level of hardware they have access to. We don’t have any idea what the next generation of consoles are going to look like, but I am now almost certain every next-generation console is going to be using some sort of machine learning to make the games look better than they otherwise would.

Jackie Thomas is the Hardware and Buying Guides Editor at IGN and the PC components queen. You can follow her @Jackiecobra