[Zhiyuan Xin's introduction] Just now, Linkou Chang Gung Memorial Hospital, which treated Jimmy Lin, announced his facial surgery plan, and one of the 3D face reconstruction technologies may save his immortal face value.On July 22nd, Jimmy Lin had a car accident while driving Tesla ModelX. It was reported that the accident caused Jimmy Lin's facial fracture and comminuted humeral fracture ...He once had an immortal face, but now he may face the risk of "disfigurement". Many netizens feel that what time can't take away has been taken away by fate.However, the emergence of a 3D face reconstruction technology this year may enable Asian Cyclone to regain its pre-injury charm.

It is reported that Lin Zhiying's facial reconstruction plan has been released, which is simply titanium alloy implantation and facial reconstruction.

Because of the comminuted fracture in his face, a larger portion of the bone was implanted through titanium alloy.

After all, bones are broken, and there must be a scaffold to hold the rest of the skeleton together.

The other, more delicate part of the bone will be reconstructed by 3D printing. When the reconstructed bone comes into contact with the body, bone cells can attach and grow into it, thus integrating the implant into the bone.

Does that sound a little oozing.

But the technique is well developed abroad, and according to doctors, the facial reconstruction can be done in just 10 hours.

In recent years, with the continuous development of deep learning and image processing units, 3D face reconstruction technology has gradually matured.

It can effectively restore the 3D information of the face through the 2D image of the face, including face geometry reconstruction and texture reconstruction.

In the past five years, there has also been a growing number of papers and scholars in the deep learning field focusing on 3D face reconstruction.

So, what exactly is this face-changing trick?

The mystery will be solved soon

How to 3D face reconstruction

Excellent technology and valuable results, nature is inseparable from the support of high-performance hardware!

The good progress of 3D face reconstruction mainly depends on the availability of multi-core CPU, GPU, AWS and other cloud applications.

Typically, 3D data is represented as voxels, point clouds, or 3D grids that can be processed by the GPU.

Let's take a brief look at the different categories of 3D face reconstruction.

3DMM

3DMM is a basic model for generating the appearance and shape of a human face. All faces to be generated are dense point-to-point correspondences. Its core idea is that faces can be matched one by one in three dimensional space.

3D reconstruction technology based on deep learning

3DGAN and 3DCNN are 3D face reconstruction techniques based on deep learning.

The main advantage of these methods is high fidelity and good performance in precision. However, training Gans still takes a lot of time and is difficult to perform, so it is difficult to apply to real-time 3D face solutions.

Of course, 3D face reconstruction is not only these two categories, Xiaobian here will not expand.

Then in the process of face reconstruction, the most important performance indicators are MAE, MSE, NME, RMSE and adversarial loss.

application

Facial puppets, video dubbing and virtual makeup are just some of the popular applications of 3D face reconstruction technology.

The game and film industries use facial puppets in video-based facial animation. The specific operation is to transmit the user's expression and emotion to the target person through the video stream. In addition, when an artist voices an animated character in a movie, 3D facial reconstruction can help the artist convey the expression to the character.

Another example is virtual makeup, which is commonly used on platforms such as web conferencing and video chatting.

It includes digital image changes such as using the right color of lipstick, masks and so on.

Here's an example of a synthetic virtual tattoo that can be adjusted to reflect facial expressions.

There are many applications of 3D face reconstruction, such as face replacement, face aging and so on. These technologies have greatly enriched our daily life.

Lin Zhiying's accident has made countless people anxious, but fortunately, 3D reconstruction of the face can be carried out. Could this technology bring the Asian Cyclone back to its pre-injury glory? Let's be patient.