Marathi-English AI Safety Red Team Evaluator

Specialised part-time consulting opportunity for Marathi-English bilingual professionals experienced in AI safety evaluation, red team testing, adversarial review, vulnerability classification, and structured feedback on sensitive text-based AI outputs.

Location: Remote

Salary: $20–$30 per hour

Responsibilities

• Review English and Marathi AI outputs for safety, reliability, bias, misinformation, and harmful-behavior risks
• Stress-test conversational AI models and agents using structured adversarial scenarios
• Evaluate model behavior across multi-turn conversations, sensitive topics, and edge-case prompts
• Identify vulnerabilities that require stronger safety controls, clearer refusals, or improved response quality
• Annotate failures, classify vulnerabilities, and flag recurring safety patterns
• Apply taxonomies, benchmarks, and project-specific playbooks to keep testing consistent
• Assess misuse cases, bias exploitation, prompt-injection scenarios, and socio-technical risk patterns at a high level
• Generate high-quality human evaluation data through careful review and structured judgment
• Produce clear reports, datasets, test cases, and written summaries that support model improvement
• Document findings reproducibly so results can be reviewed, compared, and acted upon
• Explain risks clearly for both technical and non-technical audiences
• Maintain accuracy, consistency, and strong attention to detail across submitted evaluations

Requirements

• Native-level fluency in both English and Marathi
• Prior experience in AI red teaming, adversarial testing, cybersecurity, trust and safety, socio-technical risk review, or conversational AI evaluation
• Ability to think adversarially while staying structured, careful, and methodical
• Experience using frameworks, benchmarks, or rubrics rather than unstructured testing alone
• Strong written communication skills and ability to explain safety findings clearly
• Comfort reviewing text-based content involving sensitive topics under clear guidelines
• Adaptability across project types, safety categories, and evaluation workflows

Benefits

• Remote structure with competitive hourly compensation
• Flexible scheduling
• Opportunity to contribute to safer, more reliable AI systems
• Build experience in human data-driven AI safety evaluation and bilingual risk review

Additional Information

• Independent contractor role
• Eligible professionals may be based in approved project locations depending on project needs
• Work is text-based and may involve sensitive topics such as bias, misinformation, harassment, or harmful-behavior risks
• Topic areas will be communicated before exposure to content, and participation in higher-sensitivity projects may depend on candidate comfort and project fit
• Part-time commitment depending on project availability
• Weekly payments via Stripe or Wise
• Projects may be extended, shortened, or adjusted depending on scope and performance
• Work will not involve access to confidential or proprietary information from any employer, client, or institution