A Unity mobile game that challenges players to mimic facial emotions and scores them in real time with an on-device convolutional neural network. Built as a final-year project at FAST NUCES by a team of three.
About this fork. This is a fork of the team repository. My contributions (26 commits — see the contributor graph) center on the facial emotion recognition pipeline: training the CNN, exporting it to ONNX for in-engine inference, the in-game emotion-matching logic, and the front-camera capture fix. Teammates owned game UI, authentication, and level design.
The player is shown a target emotion and must mimic it. The front camera captures the player's face; a CNN classifies the expression across seven emotions (angry, disgust, fear, happy, neutral, sad, surprise) and scores how closely the mimicry matches the target. Content is organized into levels of increasing difficulty.
- Emotion recognition model (
EmotionRecognitionModel/) — a Sequential CNN (four convolutional blocks) trained on FER2013 and exported to ONNX (sequential.onnx) so it can run on-device inside Unity.model_training.ipynbis the training notebook;webcam_implementation.pyis a desktop OpenCV test harness for the same model. - In-engine inference — the ONNX model runs against live front-camera frames each round, returning the predicted emotion and a confidence score used to grade the player.
- Game (
FYP/) — Unity project: 44 scenes, ~80 C# scripts, per-emotion art assets, packaged to Android (v1.apkincluded).
- Game engine: Unity (C#)
- ML: Keras / TensorFlow (training), ONNX (deployment), OpenCV (desktop testing)
- Target: Android
Final-year project (2024), archived as submitted. The complete, integrated team version lives on main; a built Android APK is included.