HiringWiring
AI Labs

Data Science Evaluator (contract)

Experience
2–3 yrs
Location
United States, Canada, United Kingdom
Work mode
Remote
Job type
Contract
Duration
12 Weeks
Weekly
10+ hours
Salary
$50–$125 / hour
Posted
13 days ago

About the role

AI Labs is seeking a Data Science Evaluator for a 12-week independent contractor engagement supporting the evaluation of AI systems performing real-world data science work. The role involves creating grading criteria, reviewing analyses and technical deliverables, documenting evidence-based scores, and applying consistent evaluation judgment. Candidates must have 2–3 years of professional data science experience, strong Python, SQL, statistics, machine learning, experimentation, and causal inference skills, plus excellent written English and the ability to work independently through ambiguity.

Key Responsibilities

  • Design task-specific grading criteria for exploratory analyses, statistical models, ML pipelines, A/B test write-ups, and technical notebooks.
  • Evaluate AI-generated and human-created data science work against established grading criteria.
  • Write clear, evidence-based justifications for every evaluation score.
  • Apply consistent evaluation judgment across varied technical deliverables and use cases.
  • Reason through ambiguity when assessing the quality, correctness, and completeness of submitted work.
  • Spot subtle errors in polished data science analyses and technical documentation.
  • Clearly separate established knowledge from inferences when documenting evaluation conclusions.
  • Iterate on evaluation work based on feedback from senior reviewers.
  • Follow complex instructions accurately while maintaining consistency across assigned evaluations.
  • Learn new tools quickly and work independently throughout the contract engagement.

Required Qualifications

  • 2–3 years of professional data science experience.
  • Strong skills in Python, SQL, statistics, machine learning, experimentation, and causal inference.
  • Excellent written English.
  • Ability to follow complex instructions, reason through ambiguity, and spot subtle errors in polished work.
  • Ability to maintain a clear separation between what you know and what you are inferring.
  • Ability to work independently, learn new tools quickly, and take feedback well.
  • Based in and working from the US, Canada, or UK.
  • Available for 10+ hours per week for the full 12-week engagement.

Benefits and Perks

  • Fully remote contract engagement.
  • Competitive hourly rate of $50–$125.
  • Flexible scheduling within the required 10+ weekly hours.
  • Weekly payment via Stripe or Wise.
  • Potential for projects to be extended based on business needs and performance.
  • Opportunity to contribute to the evaluation of real-world AI data science capabilities for leading AI research organizations.

Skills

  • Data Science
  • Statistics
  • Machine Learning
  • Experimentation
  • Causal Inference
  • Exploratory Analysis
  • Statistical Modeling
  • ML Pipelines
  • A/B Testing
  • Technical Notebooks
  • Evaluation Judgment
  • Grading Criteria
  • AI Output Evaluation
  • Technical Work Review
  • Python
  • SQL

Education

Preferred

  • Master's degree or higher in a quantitative field

Selection process

  1. Step 1

    Submitted

    Someone who fits the role and has told you they are interested, with their resume.

  2. Step 2

    HiringWiring Assessment

  3. Step 3

    Client Assessment

  4. Step 4

    Selected

    If they qualify and are selected, your payout is due.