top of page

Avataris

In the Avataris project, Abhimanyu Mehta contributed to the development of an AI-based system for generating realistic 3D avatars that analyzes human emotions and facial expressions from images and transfers them in real time into mixed-reality environments. The application was created as part of a research project funded by the Bavarian State Government and was designed as a cross-platform solution for web, mobile devices, and XR headsets.

During his 17-month involvement, Abhimanyu worked in a core team of four people. He took on software development tasks, trained and integrated neural networks, and was responsible for planning, implementing, and testing new features. In addition, he led the agile development process as Scrum Master and served as Feature Owner for key components of the application.

The project also included the integration of real-time video and image analysis with direct transmission into XR environments. Special focus was placed on the secure processing of sensitive image data in accordance with data protection regulations, as well as the use of monitoring and logging tools such as AWS CloudWatch for system oversight.

​

Technologies
C#, AWS Cognito, DynamoDB, AWS Lambda, AWS Elastic Container Registry (ECR), Serverless, AWS API Gateway, AWS IAM, AWS CloudFormation, .NET, Meta Quest (2 & 3), VR, Git, Google Mediapipe, AI, Convolutional Neural Networks, PyTorch, TensorFlow, Python, Docker, Unity, AI training, Hugging Face, LLaMA3, Ollama, Convai (text-to-speech & speech-to-text), OpenCV, AI-based classification

​

Responsibilities

  • Contributing to the design and implementation of an AI-based system for avatar creation

  • Implementing real-time data and video streams for facial expression and emotion transfer into XR environments

  • Ensuring data protection and access security through AWS IAM and AWS Cognito

  • Setting up monitoring and logging mechanisms with AWS CloudWatch to analyze performance and stability

  • Serving as Feature Owner for the facial analysis and emotion transmission modules

  • Training and integrating neural networks for real-time applications

  • Leading agile processes in the role of Scrum Master

  • Planning, testing, and deploying new features within the overall system

bottom of page