Remote AI Safety Data Scientist (Contract)

Skill; New York (NY); On-site; USD 105,000 - 113,000; Part time

Benefits offered by this job

W2 benefits

401k matching

Job summary

Skill is seeking an experienced Data Scientist to measure and monitor risks in conversational and agentic AI products. You will investigate safety risks in production and turn them into scalable measurement, evaluation, and monitoring systems that inform policy, model, and product improvements.

You will design safety metrics, inspect Python data analysis scripts and SQL pipelines, and calibrate LLM-based evaluation workflows.

Qualifications

• Delivered safety evaluations, risk metrics, or mitigations for an AI or machine-learning product.
• Strong Python and SQL data analysis skills with the ability to inspect, critique, and debug pipeline code and queries.
• Experience designing evaluation datasets, rubrics, and measurement metrics.
• Able to define success criteria, methodologies, and tradeoffs with stakeholders in ambiguous problems.
• Experience working cross-functionally across research, engineering, product, policy, or Trust & Safety.
• Clear written communication and ability to translate analytical findings into product or policy actions.
• Minimum 2 years AI safety data science or safety evaluation experience, or 3–5+ years broader Data Science if safety is project-based.

Responsibilities

• Develop and scale risk monitoring and safety measurement systems across AI features.
• Design safety metrics and evaluation frameworks for production systems.
• Inspect and debug Python data analysis scripts and SQL pipelines; leverage LLM tools to accelerate workflows.
• Calibrate LLM-evaluations, label datasets, and build dashboards for leadership and product teams.
• Collaborate with Product, Engineering, and Trust & Safety to translate insights into mitigations and policies.
• Communicate analytical findings clearly to technical and non-technical stakeholders.

Skills

Python

SQL

Data analysis

AI safety

Risk metrics

Cross-functional collaboration

Communication

Evaluation design

Tools

Claude

Codex

Job description

Skill is seeking an experienced Data Scientist to measure and monitor risks in conversational and agentic AI products. You will investigate safety risks in production and turn them into scalable measurement, evaluation, and monitoring systems that inform policy, model, and product improvements.

You will design safety metrics, inspect Python data analysis scripts and SQL pipelines, and calibrate LLM-based evaluation workflows.