AMD’s director of game engineering explains how their temporal upscaling tech was developed.
This interview was conducted online and recorded on video, but a written transcript is available below.
Note that both questions and answers have been edited lightly for readability.
Let’s start by getting into the motivations behind FSR 2.0.
AMD released FSR 1.0 almost a year ago as an image space scaling solution.
Nicolas Thibieroz: That’s a great question.
So let’s spend a bit of time talking about FSR 1.0, right?
So it’s interesting because we could have chosen to build a super resolution tech for higher-end GPUs only.
And in a way it would have been easier, right?
But it did not quite fit with what our developer partners and our customers wanted.
Again: cross-platform, ease of integration, and open source.
So you chose to use temporal supersampling essentially for FSR 2.0, temporal reconstruction.
How does FSR 2.0 approach that to try and eliminate ghosting as much as possible?
Nicolas Thibieroz: Yeah, good question as well.
Because we’ve seen, as you pointed out, that TAA is now pretty ubiquitous among games today.
That’s what we call disocclusion.
The secret sauce, if you will.
So we’ve got a bunch of technical details on this that I would recommend anyone to read.
And FSR 2.0 will get better and better.
So what was AMD’s approach here, to ensure the upscaling stayed high quality even for screen elements?
Nicolas Thibieroz: So I think your question hits the nail on the head.
Games are very complicated beasts, right?
They are made up of lots of different types of rendering.
And that means for high quality upscaling, FSR 2.0 has to be great at everything.
But I think it’s fair to say that transparency in particular can be challenging with temporal upscalers.
And that’s because those translucent pixels typically don’t have motion vectors or even depth information.
So you have to rely purely on colour information to decide how to upscale those.
Now FSR 2.0 is equipped to handle a large variety of the more difficult cases out of the box.
We do have that still to take care of these situations.
However, we are all about giving developers maximum control.
And to that end, we expose a mask.
I called this jittering ‘disocclusion fizzle’ in the video I made on FSR 2.0 in God of War.
Is this expected behavior based upon FSR 2.0 working principles?
Is it by design or is it stylistic?
Is this something that is going to be changeable in future iterations of FSR 2.0?
Nicolas Thibieroz: I don’t think I’ve got a yes or no answer for you, unfortunately.
What I would say though is upscaling is fundamentally a complex problem.
You’re trying to reconstruct a high quality signal from essentially a very limited set of inputs.
Our number one priority here is the player experience with FSR 2.0.
And that means we’ll optimise the algorithm based on typical playing conditions.
But again, I expect FSR 2.0 to be better, to get better and better.
yo enable JavaScript to use our comparison tools.
Nicolas Thibieroz: I think what we are seeing here is honestly the power of open source.
And I think we are seeing this first hand with all the projects you mentioned.
So yeah, there’s a rich history of modding in this industry, right?
And you know, we are very happy to support the wider development communities via all GPU Open initiatives.
But of course… official game integration of FSR 2.0 will always be preferable to a mod.
Not only from a pure quality perspective, but also in terms of ease of use for all players.
Because obviously the option is directly in the game as opposed to having to fiddle with .DLLs and whatnot.
FSR 2.0 mods forCyberpunk 2077and Dying Light 2 have real promise… and rough edges.
And same for AMD’s FSR 2.0, hopefully.
I was just curious whether AMD wants to support such an initiative in the future?
Nicolas Thibieroz: I’m going to be direct with you here.
We don’t plan to support Streamline at this time.
So you’re talking about having an open source framework that plugs into a closed technology, right?
You have different paths, essentially, for different GPUs.
Nicolas Thibieroz: So that’s a very yes or no answer.
So we think this is specifically optimised for each platform and that includes optimising for the Xbox.
Nicolas Thibieroz: Yes, yes.
So these are the throw in of examples that basically would warrant, potentially, a different code path.
So FSR 2.0 has been out for a while.
It is going to proliferate.
Developers are using it.
It’s already been announced in upcoming titles.
Are you just interested in that millisecond runtime getting lower and FSR 2.0 becoming faster?
Nicolas Thibieroz: I think it’s a good question.
Yeah, I would say we’ve got a pretty vast roadmap for upscaling research and development.
That’s very important because we want to elevate the experience for everybody.
So again, FSR 2.0 will get better and better.
I think that is something that is close to our heart.
So without pre-announcing anything, I would say that this is something that we’d want to maintain.
Okay, so it’ll get better.
It’ll be faster.
Is AMD interested at all in advancing FSR 2.0 in the direction of making RT also look better?
That mask is different from the one I mentioned earlier.
This one is called the transparency and composition mask.
Now, we’ve just talked about a bit where FSR 2.0 might be advancing the future.
Machine learning is not a part of FSR 2.0 right now.
Nicolas Thibieroz: I think there is no doubt that machine learning is a great tool.
In practice, that is great, right?
And it can be applied to a range of problems.
It was not an option for us.
So FSR 2.0 will definitely be evolving.
As to whether a future ML based upscaling solution may be released by AMD.
You know what’s coming, I’m afraid I can’t comment on that.
Because the success is there… what does this mean to you?
And FSR 2.0 is certainly a major piece in that puzzle.
And believe me, it was hard.
We had significantly difficult gates in terms of quality and performance and other metrics that we had to pass.
It really took a concerted efforts to get to where we are.
So it’s genuinely wonderful to get to work with those people day in, day out.
And I’m really, really proud of what we achieved together as a team.
That’s wonderful to hear.
It’s not just about tech.
It’s sometimes about the experience of loving tech and loving the technology behind everything… Do you remember any kind of breakthrough moment or ‘aha!’
moment in the FSR 2.0 project, regarding one of the technical aspects of FSR?
Nicolas Thibieroz: Actually, there are a few of those, certainly from a quality perspective.
So hence all of the effort we put into optimising for different platforms.
Definitely these kind of moments really made it worthwhile.
Yeah, of course.
Thank you so much.
Thanks for Nicolas for his time and AMD’s PR team for arranging the interview.
