In partnership with Standard Deviation (sdeviation.com), we are very excited to announce new face scanning services in Los Angeles! These services are:
- 3D Photometric Stereo Face Scanning
- 4D Face Capture
- Headcam Face Capture
1. 3D Photometric Stereo Face Scanning
The first service that we are offering is a continuation of the face scanning projects that I’ve been doing for the last few years. We’ve put together a portable rig with computer vision cameras so that we can do face capture on-location. We can scan your talent, wrap the head to your topology, and deliver all necessary textures. In other words, we are a scanning/modeling vendor.
The scanner is extremely solid, but actually portable. There are 3 camera columns (with 3 cameras each) and 4 light columns (with 4 lights each). So altogether that is 9 cameras and 16 lights.
Each column of 3 cameras has a server inside that controls the cameras and lights, and encodes the images onto a removable SSD in the back. Since the cameras are all pre-wired and pre-aimed, the setup for shooting is minimal. And when shooting is complete data transfer is as simple as removing the SSD.
The really exciting feature is that we can fire each light individually. So instead of standard Passive Stereo scanning, we can do 3D Photometric Stereo scans. That means that we can sync lighting to capture the talent from many different lighting directions.
Why do we shoot with different lighting conditions? So that we can calculate raw normals that look like this:
For each pixel that a camera sees, it knows which direction the light is coming from. By analyzing the difference in light intensities, each camera is able to calculate a world-space normal map.
Note that this map is the raw normal map for a single camera, so it has artifacts around areas like eyelashes. We can then merge multiple views together and fix up the problem areas to get a single high-res normal map. This approach gives us much more accurate normals and pore details than extracting them from the final color map.
Finally, we have an automated solution for wrapping each pose to the base topology. We can handle the remaining fixes like the interior of the mouth and making sure the edge loops around the eyes line up with the eyeball geometry. Then we deliver a cleaned mesh, diffuse map, and normal map for every pose.
2. 4D Face Capture
The second service we provide is 4D Face Capture, which is currently in beta. We are still sorting through a few issues but we should be ready to go live relatively soon.
Here is a video of some test footage that we can scrub inside Maya.
4D Face Capture means we are capturing a mesh per frame, instead of stills. Originally we had planned to use the same cameras for both the static scans and 4D scans. The problem we ran into is actually lenses.
For static scans, the talent is trying to keep their head still, so we focus high-res cameras as tightly as possible. Different people have different opinions, but our view is that actors need to be able to move their neck while performing.
As an example, here is a clip from Jeff Berg’s monologue during our test shoot. Even without audio, we can clearly see how much expression comes from overall fluidity of the motion. If we had to direct Jeff to keep his head rigid, it would not be the same performance.
Allowing neck motion has obvious downsides. It means we that have less effective resolution, and the data is much more challenging to track. But ultimately making sure the actor can give their best performance is more important than maximizing the accuracy of the data.
Thus, we came to the conclusion that we needed separate cameras for the 4D Capture and static 3D Scanning. However, we could use the same lights with a different strobing pattern.
That led us to this design where the upper camera is for 4D Capture and the lower one is for high-res 3D Scanning. So instead of needing to book two separate shoots, we can do high-res 3D Scanning and 4D Capture with the same talent, on the same day, in the same chair, with the same lighting.
Additionally, the high power strobing LEDs have benefits for 4D Captures as well. During the 4D Captures, we are firing the LEDs at 60fps with a 100 microsecond exposure time. That gives us evenly lit images with virtually no motion blur or ambient lighting, all without causing eyestrain.
This setup gives us flexibility on pricing. In particular, if you are already doing a FACS shoot for high-res scans, we can add on 4D Capture for a significantly lower price than doing a completely separate shoot.
3. Headcam Face Capture
For motion capture performers, we have designed and built a custom Stereo Headcam (HMC).
Standard Deviation has been making headcams for over a decade, and they have a highly refined headcam design. It is strong, rigid, and extremely light. In fact, you can actually do a handstand wearing it.
The specs for the HMC are:
- 2 Cameras, 2MP each
- Global Shutter
- 60fps
- 150-200 Microsecond Exposure
- Strobing Light
- IR and Visible Light Options Available
- Encode directly to JPEG
- Passive Cooling (No fans!!)
- 10W Power Consumption
In particular, we are most proud of the small size and low power. Writing to JPEG keeps the data size manageable if you are capturing large volumes of data. The challenge was encoding that much data in a small enough package so that we could use passive cooling. Eventually we figured it out so we can capture clean audio without fans.
We were also able to get the cameras to work with very short exposure times because the lighting strobes in sync with the cameras. To the talent, the lights appear quite dim, but since the entire pulse is condensed into under 200 microseconds we get a well exposed image. Additionally, this short exposure time makes the camera virtually immune to ambient light and motion blur.
Next Steps
At GDC, Standard Deviation is going to have a booth where you can see the hardware in person. So if you are at the expo make sure to stop by booth P1768. It was a great experience working with Standard Deviation to put this hardware/software pipeline together and I can’t wait to see what content creators do with it. And if you have any questions feel free to ping us directly.
PS: Big thanks to Jeff Berg (theofficialjeffberg.com, @jeff_berg) for being our first test case and giving a great performance. If you are looking for a talented actor in LA he is highly recommended.
comments powered by Disqus