Tasks:
- * Analyze evaluation results and identify issues
- * Build LLM-as-a-Judge and LLM-as-a-Verifier pipelines
- * Build scientific data and model benchmarks
- * Design data quality and model capability metrics
- * Evaluate chemical databases and scientific corpora
- * Plan project milestones and deliverables
- * Recommend algorithm, data, and product improvements
- * Track project progress and manage acceptance testing
- * Train and fine-tune evaluation models
Perks/Benefits:
Skills/Tech stack required:
[Agent Pipelines] [Batch Processing] [Benchmark design] [Chain-of-Thought] [Chain-of-Thought prompting] [Chemical databases] [Classifier Training] [Controlled decoding] [Data Analysis] [Data cleaning] [Data evaluation] [Dataset Construction] [Data Visualization] [Embedding Models] [Evaluation Dataset] [Evaluation Dataset Construction] [Fine Tuning] [Gold Standard] [Gold-standard annotation calibration] [Hugging Face] [Knowledge graphs] [LLM Evaluation] [Metric Design] [Multi-Agent] [Multi-agent pipelines] [Pandas] [Prompt engineering] [Python] [PyTorch] [Sampling strategies] [Scientific Data] [Scientific Data Evaluation] [Scoring Models] [Significance Testing] [SQL] [Statistical significance] [Statistical significance testing]
Educational requirements:
[Master's Degree] [PhD]
Role(s):
[Data & Evaluation Engineer] [Engineer] [Evaluation Engineer] [Learning Engineer] [LLM Evaluation Engineer] [Machine Learning Engineer] [Manager] [Project Manager] [Research Data Evaluation Engineer] [Research Project Manager]