- Solving Blendshapes for ARKit As you have probably seen, ARKit has become quite popular for facial animation. It has obvious appeal for mocap on a budget, but it’s also being used onprofessional productions as well. So I decided to do a test, and here it is:
- Visibility TAA and Upsampling with Subsample History Adventures in Visibility RenderingPart 1: Visibility Buffer Rendering with Material GraphsPart 2: Decoupled Visibility MultisamplingPart 3: Software VRS with Visibility Buffer RenderingPart 4: Visibility TAA and Upsampling with Subsample History
- Software VRS with Visibility Buffer Rendering Introduction
- Decoupled Visibility Multisampling Adventures in Visibility RenderingPart 1: Visibility Buffer Rendering with Material GraphsPart 2: Decoupled Visibility MultisamplingPart 3: Software VRS with Visibility Buffer RenderingPart 4: Visibility TAA and Upsampling with Subsample History
- Visibility Buffer Rendering with Material Graphs
- Face Scannning Services with Standard Deviation In partnership with Standard Deviation (sdeviation.com), we are very excited to announce new face scanning services in Los Angeles! These services are:3D Photometric Stereo Face Scanning4D Face CaptureHeadcam Face Capture
- Command-Line Photogrammetry with AliceVision (Tutorial/Guide) Do you need to automate a huge number of photogrammetry scans? Then I have some good news for you.
- Minimal Color Grading Tools As you have probably noticed, there are a lot of color correction algorithms out there. It can be a bit daunting to figure out the best combination of tools to use on a given project. I’ve personally gone through many iterations of trying different things in different orders and over time I’ve settled on a few key features.
- Filmic Tonemapping with Piecewise Power Curves It’s been a long while since the original posts on Filmic Tonemapping, and it is time for an update. For newcomers, the basic premise of Filmic Tonemapping is to simulate the tone curve of film in our images, with a shoulder and a toe. In the image above, the left side is a pure linear tonemap and the right side uses a filmic curve.
- Cheating Physically Based Light Intensity (IRL) One question that I struggle with in PBR is: How physically accurate should we be? In general, I’m a PBR beliver in that we should try to model CG surfaces to behave like real surfaces and we should try to model lights to act like real lights. But what about the times when an artist comes to you asking to do something that is not physically realistic in order to make it look artistically better? As luck would have it, I got a chance to look at film set for a big budget TV show recently. Or rather, the film set came to me.
- A new year, a new server. Happy new year to everyone. As you might have noticed, I’ve gotten an early start on my new year’s resolution of overhauling my web page.
- A perfect display needs 4x the resolution of Visual Acuity (maybe) In discussing what a “perfect” display would be, the common view could be said like this:
- Visual Acuity is not "What the eye can see" What resolution does a display need to have before you can not perceive aliasing? The discussion usually goes something like this:
- The Uncanny Valley as FFA Rejection Why does the Uncanny Valley exist? Why is it so difficult to cross? Why is it that all the tricks we use for making CG things like rocks, houses, and cars do not work for faces? The usual answer is something along the lines of “Faces are hard” or “As human being we are experts at understanding faces”. That just tells us the symptom, not the cause. What is the real, physiological reason why the Uncanny Valley exists?
- Combined Approximation of Fresnel/Visibility One of the problems that has been bothering me for several years is that it seems like there should be a way to speed up the Fresnel and Visibility functions by combining them together. In particular, Schlick-Fresnel and the Schlick-Smith Visibility term have a similar shape so I’ve done some experiments to combine them together. The results are above with the reference version on the left and the approximation on the right.
- The Uncanny Valley should be respected, not feared. The Uncanny Valley is something that we have all dealt with and/or thought about in computer graphics. It has become a sort of bogeyman used to scare CG artists and graphics programmers in the same way that monsters under the bed scare little children. “Don’t do that or the Uncanny Valley will get you!!!!” Like all things that we are irrationally scared of, we can put that fear to ease by really analyzing what it is, where we are, and what we can do about it. The TL/DR version of this post: Solving the Uncanny Valley will not be easy, but we shouldn’t be scared of trying.
- Specular GI Reference Recently, I was sitting in an office and I noticed some really cool light shafts that seemed to be dancing on the ceiling. Here’s a picture.
- Simple and Fast Spherical Harmonic Rotation Spherical harmonics rotation is one of those problems that you will occasionally run into as a graphics programmer. There has been some recent work, most notably Sparse Zonal Harmonic Factorization for Efficient SH Rotation (Project , PDF) which was presented at Siggraph 2012. In games we usually care about low order SH, especially 3rd order. According to Table 2, it takes 112 multiplications to rotate 3rd order SH by the zxzxz method, and 90 by the improved Zonal Harmonics method. I’ll show you a simple way to do it in 57, and source is provided at the end of the post.
- Occlusion that needs Forward Shading As mentioned in the last post, many materials out there need some kind of custom shading model to look correct. If you wanted to do all those materials with a fat G buffer, your G buffer would quickly become way too fat. So to get that kind of shader generality on todays GPUs you probably need forward shading.
- Materials that need Forward Shading The new standard shader that I’m seeing everywhere is GGX distribution plus Schlick Fresnel plus Smith Visibility. By switching to this model we’re getting much better shading than in the past.
- Optimizing GGX: Update Thanks for everyone who took a look at the proposed optimization of GGX that I put together in the last post. There were a few interesting discussions that came out of it with some thoughts worth post.
- Optimizing GGX Shaders with dot(L,H) The GGX distribution is quickly becoming the dominant lighting model games. But the obvious downside over the previous models is shader cost. So I’ve been looking at ways of optimizing it and the image above shows the before and after.
- Animated Diffuse Maps in Facial Animation If there was one specific takeaway that I wanted the audience to get from my GDC talk, it’s that we need to start animating our diffuse maps on faces.
- Physically Based does not mean Correct One important consideration with shading models is that physically based models don’t necessarily look like the real world. They might, but usually do not.
- Physically Based Specular for Artists Another day, another blog post about physically based rendering. Physically based rendering is becoming very common in games, but that doesn’t help you very much if the artists don’t know how to use it properly. So consider this post a quick intro for artists.
- Visual Acuity, Vernier Acuity, Anti-Aliasing, and You Note: This is a lightly edited post written in 2012. My core argument at the time is that FXAA and other AA algorithms are “Good Enough”, so there isn’t much place for MSAA or other AA techniques that impose a significant burden on the rendering (like a MSAA depth pass). Since then, I’ve changed my mind, which will be part of another post in the future. But for now, you have my future predictions from 2012 which didn’t go as I had expeted.
- Future of AA Techniques Hopefully you all had a good time at Siggraph. For game programmers, the one course that you absolutely must check out is Filtering Approaches for Real-Time Anti-Aliasing. Make sure to check out the course webpage which will include slides and sample images. Here’s my general thoughts on it.
- Is it really a "Retina Display?" Update: This post was originally written several years ago, as a discussion of the iPhone 4 and the “Retina Display” marketing term. The original post was quite outdated but the discussion of LCDs versus physical printing is still relevant.
- How Hard is Your Face?
- Gamma Correction and Gamma Correction Hi everybody. It’s been a while since my last post, mainly because I’ve left Naughty Dog and started my own company. Suffice it to say, I’ve been very busy and feverishly coding. Don’t have anything to say about it right now other than it’s going to be cool and it’s not going to be a game. Btw, U3 is going to be crazy. And yes, I need to fix that image on the right.
- Everything has Fresnel You can sorta think of this post as part 2 of my “Everything is shiny rant”. While standard specular lighting is pretty common in games, one effect that we rarely see in games is proper fresnel.
- Everything is Shiny Every once in a while I’ll see a tutorial online or a paper that talks about specular. And it will have a line like “Some materials such as cloth and cardboard are pure diffuse surfaces with no specular”. This is a lie. Every object in the real world has specular, even good old cloth and cardboard. I took these with my homebrew polarized lighting setup. Since I was holding the polarizer manually, the separation isn’t perfect, but seems to work well enough.
- Cross Bilateral Filters for Skin Shading Hope you all don’t mind a plug for a cool skin paper written by Morten Mikkelsen. Morten is a programmer for the ICE team at Naughty Dog, and he has put together a technique getting faster skin shading using a full-screen buffer.
- Gamma and Mipmapping For the next few weeks, my plan is to try and talk about the last few gamma things that no one tells you about and then I can stop talking about it. Today’s topic is gamma and mipmapping.
- Linear-Space Lighting (i.e. Gamma) Linear-space lighting is the single most important thing for graphics programmers and technical artists to know. It’s not that hard, but for whatever reason, no one really teaches it. Personally, I got a BS and MS at Georgia Tech, took basically every graphics class they had, and didn’t hear about it until I learned about it from George Borshukov. This post is essentially the short version from my talk at GDC this year. You can check out the slides for more details.
- How To Split Specular And Diffuse In Real Images File this under things that people think is harder than it actually is. One thing I’ve got set up at home is the ability to use polarized light to split out the diffuse and specular components from real images. Artists spend lots of time looking at photo reference, but splitting the specular and diffuse really helps you understand a material. In the title image, the left is the diffuse light only, the middle is diffuse and specular light, and the right is specular light only. Btw, I’m not the expert here, but this is how I understand it.
- Why a Filmic Curve Saturates Your Blacks In the previous post, I talked about how a Reinhard curve, when applied to each channel individually, will desaturate your blacks. In fact, any curve that is concave downwards will do the same thing. But if we use a curve that is concave upwards, will it do the opposite, and give us more saturation in our blacks?
- Why Reinhard Desaturates Your Blacks One thing I’ve mentioned is that if you use the simplest Reinhard tonemapping operator, then you will get desaturated blacks. In this image, the top row is using Reinhard, the middle is straight linear, and the bottom is using Filmic Tonemapping by HP Duiker. Also, there are many different forms of Reinhard, so the one I’m using is x/(x+1) for each color channel.
- Filmic Tonemapping Operators The most common questions that I get about my GDC talk have to do with the tonemapping operators. In particular, I’ve always found that when I read through presentations for code snippets that I always miss something. Those 2.2s can be tricky! So this post is a quick reference for various operators that I talked about. Also, I copied and pasted this code from my RenderMonkey scene so there may be typos.
- What is Gamma? So what is this gamma thing that I’m always talking about? Here’s a quick question for you: What is half way between 0 and 255 on your monitor? Check out this image. The left and right sides are alternating lines of 0 and 255. Looking at the center 2 squares, the bottom square should about match the left and right whereas the top square should be much darker. What color is the bottom square?
