Language Model Evaluator – Fully Remote | Up To $20/hr

Mercor connects elite creative and technical talent with leading AI research labs. The company is based in San Francisco and has notable investors including Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Location: Remote

Salary: $15–$20/hour

Responsibilities

• Conduct fact-checking using trusted public sources and external tools.
• Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies.
• Assess reasoning quality, clarity, tone, and completeness of responses.
• Ensure model responses align with expected conversational behavior and system guidelines.
• Work independently and asynchronously to meet deadlines while improving AI model performance.

Requirements

• Bachelor's degree
• Native speaker in Bengali
• Significant experience using large language models (LLMs)
• Excellent writing skills in English
• Strong attention to detail
• Background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)

Benefits

• Application process involves uploading resume, AI interview, and submitting form.
• Resources and support available at https://talent.docs.mercor.com/welcome.
• Support contact: support@mercor.com.

Additional Information

• Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.