Aircraft Crew 3D - Lip Sync Animation Platform

A cutting-edge Angular-based 3D lip sync animation platform designed for aircraft crew training and operational scenarios. This application combines speech recognition, 3D avatar rendering, and audio synthesis to create an interactive crew instruction system.

Features

🎤 Voice Recognition: Real-time speech recognition for hands-free operation
🗣️ Speech Synthesis: Natural-sounding crew announcements
👤 3D Avatar: Interactive crew avatar with lip synchronization
📊 Audio Visualization: Real-time audio frequency analysis
📱 Responsive Design: Works across desktop and mobile devices
⚡ Real-time Animation: Smooth 3D animations powered by Three.js

Technology Stack

• Framework: Angular 17+ with signals
• 3D Rendering: Three.js (via CDN)
• Speech: Web Speech API (Recognition & Synthesis)
• Audio: Web Audio API for analysis and visualization
• Styling: CSS3 with modern features (Grid, Flexbox, backdrop-filter)
• Build System: Angular CLI with TypeScript

Pre-defined Voice Commands

The system recognizes the following voice commands and plays corresponding crew instructions:

Voice Command	Instruction
"turbulence"	Turbulence warning and safety instructions
"seatbelt"	Seatbelt sign activation announcement
"landing"	Prepare for landing instructions
"safety"	Safety demonstration announcement

Getting Started

Prerequisites

• Node.js 18.x or higher
• npm or yarn package manager
• Modern web browser with Web Speech API support
• Microphone access for voice recognition

Installation

1. Clone the repository:

git clone <repository-url>
cd aircraft-crew-3d

2. Install dependencies:

npm install

3. Start the development server:

ng serve

4. Open your browser and navigate to http://localhost:4200

Building for Production

ng build --prod

The build artifacts will be stored in the dist/ directory.

Usage

Voice Recognition

1. Click the "Start Voice Recognition" button
2. Say one of the pre-defined commands (e.g., "turbulence")
3. The system will recognize the command and play the corresponding instruction

Manual Commands

• Use the quick command buttons for immediate instruction playback
• Each button corresponds to a specific crew scenario

3D Avatar

• The avatar automatically animates with lip sync during audio playback
• Audio visualization shows real-time frequency analysis
• Use "Reset Avatar" to return to default position

Activity Log

• Monitor all system activities in the log section
• View timestamps and action descriptions
• Automatically maintains the last 10 activities

Browser Compatibility

Fully Supported

• Chrome 80+
• Edge 80+
• Firefox 70+
• Safari 14+

Features by Browser

Feature	Chrome	Firefox	Safari	Edge
Speech Recognition	✅	❌	❌	✅
Speech Synthesis	✅	✅	✅	✅
Web Audio API	✅	✅	✅	✅
Three.js	✅	✅	✅	✅

Note: Speech recognition is currently limited to Chromium-based browsers

Project Structure

src/
├── app/
│   ├── app.ts              # Main application component
│   ├── app.html            # Application template
│   ├── app.css             # Application styles
│   └── app.config.ts       # Application configuration
├── assets/
│   └── audio/              # Audio files (placeholder)
├── index.html              # Main HTML file
└── styles.css              # Global styles

Key Components

Speech Recognition Service

• Handles voice input detection
• Processes natural language commands
• Manages recognition lifecycle

3D Avatar System

• Three.js scene management
• Avatar mesh creation and animation
• Lip sync animation based on audio analysis

Audio Management

• Speech synthesis for crew announcements
• Real-time audio analysis for lip sync
• Visual frequency display

Development

Code Style

• Follow Angular style guide
• Use TypeScript for type safety
• Implement reactive patterns with signals
• Write unit tests for critical functions

Adding New Commands

1. Update the availableCommands array in app.ts:

{
  keyword: 'newcommand',
  label: 'New Command Label',
  instruction: 'Instruction text to be spoken',
  audioFile: 'assets/audio/newcommand.mp3'
}

2. The system will automatically recognize the new command

Customizing the Avatar

Modify the createAvatar() method in app.ts to:

• Change avatar appearance
• Add more detailed geometry
• Implement advanced animations
• Load external 3D models

Troubleshooting

Common Issues

Speech recognition not working:

• Ensure microphone permissions are granted
• Check browser compatibility (Chrome/Edge recommended)
• Verify HTTPS connection for production

3D avatar not displaying:

• Check browser console for Three.js errors
• Ensure WebGL is enabled in browser
• Verify CDN links are accessible

Audio not playing:

• Check browser audio permissions
• Verify speakers/headphones are connected
• Test with browser developer tools

Debug Mode

Enable debug logging by opening browser developer tools and checking the console for detailed information about:

• Speech recognition events
• Audio processing status
• 3D rendering performance
• Component lifecycle

Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Three.js community for 3D rendering capabilities
• Angular team for the reactive framework
• Web Speech API for voice recognition
• Modern browser vendors for advanced web APIs

Support

For support and questions:

• Create an issue in the GitHub repository
• Check the troubleshooting section
• Review browser compatibility requirements

---

Built with ❤️ for aircraft crew training and safety