Tasks:
- * Build multidimensional agent evaluation metrics
- * Design test datasets for typical, edge, and adversarial scenarios
- * Develop prompt templates and tool-use best practices
- * Gather user needs and structure use cases
- * Track agent failures and inform product iterations
Perks/Benefits:
Skills/Tech stack required:
[AI alignment] [AI Evaluation] [Business Intelligence] [Business intelligence BI] [Case design] [Dataset design] [Evaluation methods] [Language Models (LLMs)] [Large Language Models] [Large Language Models (LLMs)] [LLM Evaluation] [LLM Evaluation Methods] [Product Metrics] [Prompt engineering] [Python] [Retrieval-Augmented Generation] [Retrieval-Augmented Generation RAG] [SQL] [Test Dataset] [Test Dataset Design] [Tool use] [Use Case] [Use-case design]
Educational requirements:
[Bachelor's Degree]
Role(s):
[AI Product Manager] [AI Product Manager Intern] [Intern] [Product Manager] [Product Manager Intern] [Project] [Project Manager] [Project Manager Intern]