Javanese-Indonesian Female Facial Expression ( JIFFE-3 D ) Based on 3 D Marker

Facial expression is a topic which is frequently used in several areas of research, such as security, psychology, and entertainment. All of these researches demand a high quality source data of facial expression. This research builds JavaneseIndonesian Female Facial Expression Database in three dimensional data called JIFFE-3D Database. JIFFE-3D Database contains facial expression which expresses Javanese-Indonesian female universal emotions, such as happiness, disgust, fear, angry, surprise, and sadness. JIFFE-3D Database is made by using motion capture system to capture changes in facial expression which is characterized by a shift in markers on face. The facial expression is captured by multi camera and saved in 3D data. JIFFE-3D Database could be a high quality source data if all data use only marker which has influence the shifting in facial expression. Keywords Facial Expression, Javanese-Indonesian, 3D Marker, Facial Motion Capture.


I. INTRODUCTION 1
uman-computer interface is topic which frequently used in several areas of research.The researches have a purpose to make friendly interface for user.One of those researches is avatar-human making.The avatar resembles all of human characteristics, such as body and face shape, voice, and facial expression.The purpose of avatar-human making is to make user more comfortable when using computer technology.
Facial expression is the important part of human characteristics which is imitated to make natural avatar.It is because facial expression has the main part in human social interaction.Facial expression contribute 55% to make a message can be delivered in human social interaction, and the other component, such as verbal and vocal component contribute 7% and 38% [1].Thus, as a concerned to make advance research, then needed a based research about facial expression, such as arrangement database of facial expression.
Topic of this research is about arrangement database of facial expression which is called as Javanese-Indonesian Female Facial Expression in three dimensional data called JIFFE-3D Database.JIFFE-3D Database use Javanese-Indonesia female as model and three dimensional markers are used as the data.
There are many facial expression databases which are used in researches, such as 2D pictures like JAFFE (Jappanese Female Facial Expression) used in [2,3,4] or 3Dimensional data like BU3DFE (Burmingham University 3D Facial Expression) used in [5,6,7].However, all of those facial expression databases never used Javanese-Indonesian female as model.Thus, it is needed to build a database consist of facial expression which is used Javanese-Indonesian female as model.
JIFFE-3D Database uses three-dimensional (3D) data because 3D data have many benefits for facial expression capture.3D data could provide data at three dimensions.In addition, 3D data could capture head movement which support at facial expression capture [8].
JIFFE-3D is made by motion capture and multicamera system.Motion capture was used to capture changes in facial expression which is characterized by shift in markers on face.The facial expression is captured by multi camera and saved in 3Dimensional data.

II. JIFFE-3D DATABASE
JIFFE-3D Database is a group of three dimensional facial expression data which is presented human universal emotion.There are neutral expression and six human universal emotions, such as angry, sad, happy, surprise, disgust, and fear.Those emotions are divided as human universal emotion by Ekman,Freisen,and Tomkins [9].Data JIFFE-3D is the coordinate position of facial marker in three dimensions.The marker position data was captured during one frame of time.
JIFFE-3D Database consists of ten female Javanese-Indonesian models.Table 1 describes identity of the models.

III. RESEARCH PHASE
JIFFE-3D Database research phase is divide at four phase, that is preparation phase, facial expression capture phase, reparation phase, and arrangement phase.The following explanation is about the research phase.

A. Preparation Phase
In the preparation phase, all of equipment which is needed at facial expression was prepared, such as camera and marker.1) Camera Setup Facial expression was captured by multi camera.Multi-camera is motion capture technic to use more than one camera at same time.The cameras are put at specific position that could catch model movement from vary point of view.Meanwhile, all of cameras must have timing synchronization and similarity perception about object location, in order to get the perfect capture.This research uses six OptitrackTM cameras.Three cameras are placed in a semicircle above head and three others are placed in a semicircle at chest level.All six cameras must be pointed at the model's nose, about 60 cm from model's face.See Figure 1 for camera setup illustration.

H 2) Camera Calibration
Camera must be calibrated in order to have similarity perception about object location.Calibration process divided in to four steps: a.To make sure about cameras position.b. Camera setting adjustment.c.Wanding process has function to synchronize cameras perception about marker location.d.Ground plane process has function to adjust coordinate at program, in order to appropriate with world coordinate.

3) Camera Calibration
Marker was used to identify object's movement area.At this research, the object was face of Javanese-Indonesian female model and the marker is 4 mm set reflective marker.There are 37 markers.The markers should be covered object's movement area, such as head, eye, nose, mouth, and cheek area.Every marker should be capture by minimal three cameras in order to be detected by motion capture system.We put all markers on the model's face with a composition as described in [4].See Figure 2 and Table 2 for face markers location.

4) Facial Template Configuration
Facial template configuration step is used to make sure that facial marker have similarity with facial template at the facial motion capture system software.Face template configuration could be configured when all of marker could be capture completely.Facial template configuration was performed before facial expression capture phase, therefore, every model has different facial template.

B. Facial Expression Capture Phase
At this phase, model's face was captured by using motion capture system.Motion capture system was able to record any movement of facial marker at real world and translates it to 3D movement.
At this phase, facial expression was captured during 10 second or 1000 frame.At the first three second, model show neutral expression, continued for the next seven second model show one of the universal facial expression, such as angry, sad, happy, surprise, disgust, fear, or neutral.Each expression was captured by four times.If there was data mistake or problem, the data must be recaptured.See Figure 3 for JIFFE-3D Database's facial capture examples.
Capture data result was shown as two-dimensional marker data from each camera.All of data was synchronized in order to get completed data in threedimensional coordinate which similar to world coordinate.Data consist of three-dimensional marker location was written to the file with extension .pt2.

C. Valve Component
Marker data in 3-dimensional coordinates need to be examined in advance to avoid any marker detection error by motion capture system.Marker movement was visualized by a trajectory which could be repaired in motion capture software.There were three types of reparation data, namely smoothing, fill gap, and fix swap.See Figure 4 for reparation data illustration.a. Smoothing was a process to smoother capture result.
Smoothing process was applied when there was a ripple at trajectory.The ripple was the result of the fault marker movement, such as marker location at real world was not same as at the motion capture software.b.Fill Gap was a process to connect two tracks which were cut off because there was gap.The fill gap process was applied when there was a marker which couldn't be captured by camera system.c.Fix Swap was a process to switch two tracks.The switch process was applied when there were two markers get accidently exchanged.

D. Arrangement Phase
At this phase, three-dimensional market data which already repaired would be selected and labeled.Arrangement phase was divide in two steps, there were selection step and labeling step.At selection step, threedimensional data would be selected by using survey method.Only the best expression would be represented as JIFFE-3D Database data.At labeling step, each was labeled for the purpose how to differ an expression into another.The following explanation is about arrangement phase at the research.

1) Selection Step
In the selection step, we choose the best expression from all of capture data.It is needed because each model would not only show the best expression at the capturing process, sometimes it would be bad data.The selection method was employed by survey using specific questionnaire.Correspondence choose the best pose from each pose.The pose which has the most votes would represent JIFFE-3D Database data.
At this step, there are 840 pose which is used as survey data.Each expression was represented by three pose.The three pose was considered to be representatively universal facial emotion.The reference to choose the pose was Tekalp's and Ostermann's in [11], see Table 3.

2) Labeling Step
The chosen data is labeled based on data identity, such as model name, expression name, and sequence capture number.The labeling rule is the first three digits is model name, the middle three digits is expression name, and the last two digits is sequence capture number.

IV. RESULTS AND FUTURE WORK
The outcome of this research wa a facial expression database which is called as JIFFE-3D Database (Javanese-Indonesian Female Facial Expression-3D Database).JIFFE-3D Database is made by using motion capture and multi-camera methods.During our work to collect the data and build this database, we got some conclusion : a.The best quality of JIFFE-3D Database capture process is done in close-room at any time and in open-room at night.Our motion capture system employed infra-red camera, this camera capture infra-red light reflected by facial marker.Some infra-red light from another sources e.g from the sun will cause interference resulting in the determination of fault location of the marker.b.Position of the camera must leads to the center of the model's face, as well as the head of the model must be in the middle of the whole camera.It must be done because the whole camera must be able to capture the entire marker on the model's face.
In the next phase, we will use the data from this database to classify the facial expression of Javanese-Indonesian female.[3] No. Expression Description

Happy
The eyebrows are relaxed.The mouth is open and the mout corners pulled back towards the ears.

Sad
The inner eyebrows are bent upward.The eyes are slightly closed.The mouth is relaxed.

Angry
The inner eyebrows are pulled downward and together.The eyes are wide open.The lips pressed against each other or opened to expose teeth.

Fear
The eyebrows are raised and pulle dtogether.The inner eyebrows are bent upward.The eyes are tense and alert.

Disgust
The eyebrows and eyelids are relaxed.The upper lip is raised and curled, often asymmetrically.

Surprise
The eyebrows are raise.The upper eyelids are wide open, the lower relaxed.The jaw is opened.

Figure 4 .
Figure 4. Reparation data illustration (a) Ripple and smoothing; (b) Gap and fill gap; (c) Marker were accidently exchanged and fix swap

TABLE 3 .
FACIAL EXPRESSION PARAMETER BY