Unity SDK – Scripts and Components

The main part of the MoodMe SDK plugin is the scripts and the prefabs.

• EmotionsManager.cs is the main script that will handle the Emotion Engine and anything you will need to test the engine or to derive from your own script. EmotionsManager.cs is the main component of the prefab MoodMeEmotionsManager that is mainly an empty Game Object with this script attached.
• CameraManager.cs handles the video source. You can derive from it your own script to handle specific kinds of input.
• EmotionInterface.cs is the bridge between Unity logic and engine logic. You don’t need to edit this file.
• MoodMeEmotions.cs is a basic class. Keep this file as-is.
• FaceDetector.cs handles the neural network that detects faces from an image.
• FaceDetectorPostProcessing.cs handles the output of the neural network that detects faces from an image and returns a list of face coordinates and probabilities.
• Preprocessing.cs is a class that handles some common tasks like rotations, scale, image formats conversion.
• ManageEmotionNetwork.cs handles the neural network that detects emotions from an image.
• GetEmotionValue.cs is a simple script used in the demo scene that fetches a value from the EmotionsManager and writes it to the value property of a UI Slider. Use it to learn how to read emotions’ value.
• MorphController.cs is a simple script that can drive a SkinnedMeshRenderer blendshapes using emotion values.

Script Properties #

The script presents the following properties:

• Input is the section where you will refer the Face Detector and Emotions Detector components in the scene.
• Process Every N Frame is a parameter to help save CPU/GPU time with emotion detection, which is a very intensive task. Default is to process 1 of every 15 frames, which is a good trade-off.
  • A smaller value will result in more frequent detections at the cost of more CPU time.
  • A bigger value will spare CPU time but the detection will be less sensitive to quick variations.
• Filter All Zeros is a flag that tells the script to ignore when the engine outputs all zeros in case of the wrong detection.
• Smoothing is the magnitude of the smoothing weight. The range is from 0 to 29.
  • 0 means no smoothing and a lot of noise
  • 29 means almost no noise but very damped variations
• Test Mode is a flag that, if enabled, will detach the engine from the final output and will use the values below instead. This is a helper in the development phase when you can manually change the output to test your implementation.
• emotions sliders at the bottom will show the values from the engine or can be used to input the data directly if the “Test Mode” flag is set. They have a debug functionality and nothing else.

Prefab MoodMeCameraManager #

The prefab MoodMeCameraManager that is mainly an empty Game Object with this script attached that handles camera input and Video input.

The list at the top is a list of the available video sources (webcams). The number of the selected source must be entered in the Device Index property.

• Webcam Plane refers to the quad that is in front of the camera that shows the video stream. This is helper script: feel free to change if you want a different approach.
• Webcam Texture refers to the texture that will be filled with the video data.
• Video Texture (optional) can be referred to as a Render Texture where a video player will render a video file. This way you can use a video in the development phase instead of a webcam source. If this property is empty, the webcam will be used.
• Network Agent is the prefab that handles the 2 neural networks that are needed to make this plugin to work.

The components are already configured in all their parts but if you want, you can play with the threshold in the face detector component. The lower the value, the more sensitive and prone to errors the detection will be.

You can also deselect the preview parts if you don’t need that service.