Hans-Den/Nexora-Encoder

1# Distributed Media Encoder Service2 3A distributed video encoding service with a modern dark-themed UI, supporting multiple encoding instances for high-performance video processing.4 5## Features6 7- Modern dark-themed web interface with drag-and-drop8- Distributed video encoding across multiple workers9- Multiple quality outputs (480p, 720p, 1080p)10- Real-time encoding progress tracking11- Single-port access through Python proxy12- Temporary directory support for restricted environments13 14## System Architecture15 16- **Web Interface**: Modern dark theme UI for video upload and monitoring17- **Task Queue**: Redis-based distributed task queue18- **Workers**: Multiple Celery workers for distributed encoding19- **Storage**: Temporary directory based file storage20- **Proxy**: Python-based reverse proxy for single-port access21 22## Prerequisites23 24- Python 3.8+25- FFmpeg26- Redis27 28## Quick Start29 301. Install dependencies:31```bash32pip install -r requirements.txt33```34 352. Start the service:36```bash37python3 start_service.py --port 5000 --tmp-dir /tmp/encoder38```39 40The service will be available at `http://localhost:5000`41 42## Distributed Setup43 44### Main Instance45```bash46python3 start_service.py --port 5000 --tmp-dir /tmp/encoder47```48 49### Additional Worker Instances50```bash51# Worker 152python3 start_service.py --worker-name encoder_worker_1 --tmp-dir /tmp/encoder_153 54# Worker 255python3 start_service.py --worker-name encoder_worker_2 --tmp-dir /tmp/encoder_256 57# Add more workers as needed58```59 60## Command Line Options61 62- `--port`: Port number for the web interface (default: 5000)63- `--tmp-dir`: Temporary directory for storage (default: /tmp/encoder)64- `--worker-name`: Custom name for worker instance65 66## Directory Structure67 68```69/tmp/encoder/70├── uploads/           # Uploaded video files71├── encoded/          # Encoded video files72├── redis.conf        # Redis configuration73├── redis.rdb         # Redis database74└── redis.log         # Redis logs75```76 77## Monitoring78 79### View Service Logs80```bash81tail -f /tmp/encoder/redis.log  # Redis logs82```83 84### Check Worker Status85```python86from app.services.encoder_service import celery87celery.control.inspect().active()  # View active tasks88celery.control.inspect().registered()  # View registered workers89```90 91## Scaling92 93The service can be scaled by adding more worker instances. Each worker should:941. Have FFmpeg installed952. Connect to the same Redis instance963. Have access to the shared storage directory97 98### Example Scaling Setup99 100```bash101# Main instance (handles web UI and coordination)102python3 start_service.py --port 5000 --tmp-dir /shared/tmp/encoder103 104# Worker instances (handle encoding tasks)105python3 start_service.py --worker-name encoder_worker_1 --tmp-dir /shared/tmp/encoder106python3 start_service.py --worker-name encoder_worker_2 --tmp-dir /shared/tmp/encoder107```108 109## Performance Considerations110 1111. **Storage Performance**:112   - Use fast storage for /tmp directory113   - Consider local SSD for better I/O performance114 1152. **Network Bandwidth**:116   - Ensure sufficient bandwidth between workers117   - Consider network topology when distributing workers118 1193. **Resource Allocation**:120   - Monitor CPU usage across workers121   - Balance memory usage for concurrent encoding122 123## Troubleshooting124 1251. **Service Won't Start**:126   ```bash127   # Check if Redis is already running128   ps aux | grep redis129   # Check port availability130   netstat -tulpn | grep 5000131   ```132 1332. **Encoding Fails**:134   - Verify FFmpeg installation135   - Check temporary directory permissions136   - Verify Redis connectivity137 1383. **Worker Connection Issues**:139   - Check Redis connection string140   - Verify network connectivity141   - Check firewall settings142 143## Security Notes144 1451. **Temporary Directory**:146   - Regularly clean up old files147   - Set appropriate permissions148   ```bash149   chmod 700 /tmp/encoder150   ```151 1522. **Redis Security**:153   - Use strong passwords in production154   - Configure proper network restrictions155 1563. **File Validation**:157   - Service validates video file types158   - Implements size restrictions159   - Sanitizes filenames160 161## API Endpoints162 163- `POST /api/upload`: Upload video file164- `GET /api/status/<job_id>`: Get encoding status165- `GET /api/video/<job_id>/<quality>`: Stream encoded video166- `GET /api/jobs`: List all encoding jobs167 168## Contributing169 170[Contribution Guidelines]171 172## License173 174[Your License Here]175