Tasks:
- * Analyze failure cases and inform model and product iteration
- * Assess video generation quality across industry scenarios
- * Benchmark competitor models
- * Build and iterate video model evaluation systems
- * Calibrate scoring and audit evaluation consistency
- * Create test cases, prompts, and evaluation datasets
- * Design edge cases for motion, interaction, and temporal consistency
- * Maintain issue attribution taxonomy
- * Manage outsourced evaluation quality
- * Write reproducible evaluation reports
Perks/Benefits:
Skills/Tech stack required:
[Arena] [Audio/Video] [Audio/video synchronization] [Audio-video synchronization evaluation] [Automated Evaluation] [Automated evaluation scripting] [Case design] [Cinematography] [Competitive benchmarking] [Composition analysis] [Consistency evaluation] [Data Analysis] [Dataset development] [Data Visualization] [Evaluation Framework] [Evaluation framework design] [Excel] [Failure Attribution] [Framework design] [Instruction following] [Instruction following evaluation] [Model Evaluation] [MOS] [Motion quality assessment] [Prompt engineering] [Python] [Quality Assessment] [Scoring Calibration] [SQL] [Storyboarding] [Temporal consistency] [Temporal consistency evaluation] [Test Case] [Test Case Design] [VBench] [Video Editing] [Video Generation] [Video generation model evaluation] [Video quality] [Video quality assessment] [Visual composition] [Visual composition analysis]
Educational requirements:
[Bachelor's Degree]
Role(s):
[AI Model Evaluation Specialist] [Evaluation Specialist] [Evaluator] [Model Evaluation Specialist] [Specialist] [Video Model Evaluator]