The digital transformation of traditional art
Digital Mask utilizes AR filters to transform traditional theatrical mask art into digital art. Transform these on-stage masks into digital masks that everyone can wear. I'm thrilled to use multiple techniques to convey traditional mask art more broadly.
Lens Studio, Adobe PS, AI
Traditional opera art is fading from the public's view.
Beijing Opera is a synthesis of stylized action, singing, dialogue, mime, acrobatic fighting and dancing to represent a story or depict different characters and their feelings of gladness, anger, sorrow, happiness, surprise, fear and sadness. The characters may be loyal or treacherous, beautiful or ugly, good or bad. Their images are always vividly manifested in bright costumes that show the styles of ancient China.
The masks of Beijing opera incorporates the personality, role and background of the characters. Different mask styles and colors represent different historical figures.
How can design bring traditional art into digital era?
Customized digital AR masks
In the Peking Opera makeup, artists use exaggerated and abstract lines to express the characters’ face, use different colors to express the personality of the characters.
Digital masks digitize styles and colors of Peking Opera mask and use expression recognition to customize each mask.
Scan in snapchat
Understand the essential of opera mask
weigh main elements
Ensure technical path
choose the right tools
Test the prototype
improve the performance
The relationship between color and personality in Beijing opera masks.
In Peking Opera masks, people use exaggerated and abstract lines to express the characters’face, use different colors to express the personality of the characters. Different color schemes help viewers understand thepersonality of the roles. For example, red often represents loyalty, blue stands for straight and unruly.
Styles + Color shifting
Link emotions with mask colors.
expression recognition supported by deep learning network
Color can affect mood, and emotion can be expressed in different colors.
In order to preserve the relationship of color and personality in the digital mask. I use emotions to replace personality. Using facial expression recognition technology, emotions can be quickly identified. This can serve as the basis for the real time changing of digitized masks.
A simple neural network can distinguish A from B.
A more complex neural network can further distinguish same color shape into circle and square.
Facial expression recognition.
The principle of deep learning networks is similar to neurons in the human brain. It can greatly improve the intelligence of the computer and make it possible to deal with problems that was previously only humans could do, for example, art creation.
The neural network I use, called convolutional neural networks, can classify and identify different emotions.
Successfully recognized the expression in the picture
Design digital opera masks as a snapchat fliter
I want people to experience traditional mask art in a fun and easy way. Snapchat has powerful filters in it. I use Lens Studio to create a filter and publish it in snapcht.
Through this snap fliter, people can get their own Peking Opera masks and interact with AR masks. They are also able to post these in social networks.
Technical validation of customized AR masks
Two parts of technical issues need to be tested and validated.
-The first was a custom mask. I decided to use self-generated art instead of designing a lot of mock-ups and letting users choose their own. Each user's mask should be unique. Just like the masks for characters in Peking Opera are exclusive.
-The next step is to make filters in Lens Studio. I want it to be able to accomplish the effect of changing faces. By turning back once or using gesture to change the AR mask, just like the face changing magic in the traditional theater stage.
Develop customized masks
Step1 Build database
Collect Peking Opera masks and extract the elements to create a database for deep learning network learning and training.
Step2 Recognition + classification
Using the recognition program to detect and classify the face, and then the tester's facial expression is recognized. Then pick the appropriate parts in the image library and recombine them.
According to the appearance of a person, the program selects the appropriate materials in the content library to combine, and then selects different color combinations according to the facial expressions.
Work flow of self-generated digital mask
Test the basic function — generate 2D masks
Fliter development in Lens Studio
Face Mask Function
Hand Gesture Function
Convert 2D mask images into 3D mesh that fits the face
Lens Studio development interface
Hand Gesture makes it easy for users to change their masks (They can have several generated masks based on their six expressions )