DevOps Engineer - AI Model Evaluator

Mercor connects elite creative and technical talent with leading AI research labs. The company is headquartered in San Francisco and has notable investors including Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Location: Remote

Salary: $85/hour

Responsibilities

• Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
• Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, and infrastructure automation.
• Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
• Compare outputs from multiple frontier models to assess strengths and weaknesses.
• Apply professional engineering judgment to realistic infrastructure engineering scenarios.

Requirements

• 2+ years of professional DevOps, SRE, or Cloud Engineering experience.
• Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling.
• Regular use of AI coding agents like Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
• Ability to evaluate model-generated infrastructure and reliability engineering solutions.

Benefits

• $400 per accepted task.

Additional Information

• Application process involves uploading a resume, completing an AI interview based on the resume, and submitting a form.
• The process takes 20–30 minutes.
• Applications are reviewed daily.