Data Scientist: AI/ML Model Quality & Data Integrity

Socket.dev; Austin (TX); On-site; USD 120,000 - 180,000; Full time

Job summary

Socket.dev in Austin, TX seeks a Data Scientist, AI/ML Model Quality to build validation frameworks, observability metrics, and telemetry for GenAI features across Wallet, Payments, and Commerce. You ensure data quality drives trusted models reaching hundreds of millions of users.

The ideal candidate has strong statistics, Python and SQL skills, experience with ML observability, and familiarity with GenAI/LLMs.

Qualifications

• Bachelor's degree with hands-on ML/AI model quality or related research; MS/PhD preferred.
• 3+ years in data science with focus on data quality, model evaluation, or ML observability.
• Proficiency in Python (Pandas, NumPy, Scikit-learn) and SQL for data analysis.
• Experience with distributed computing (PySpark, Spark, or distributed SQL).
• Strong statistics knowledge: hypothesis testing, data drift, SPC.
• Experience defining and tracking ML health metrics in production.
• Familiarity with GenAI/LLM systems and telemetry instrumentation.
• Excellent communication to translate findings to engineers and stakeholders.

Responsibilities

• Own the health of data for ML and GenAI features across Wallet, Payments, and Commerce.
• Build validation frameworks and observability metrics.
• Lead telemetry analysis to surface actionable insights.
• Collaborate with ML Eng, Data Eng, Privacy, and Legal teams.
• Ensure models are trained, evaluated, and deployed on trusted data.

Skills

Python (Pandas)

SQL

PySpark

Statistics

ML observability

GenAI/LLM

Data quality

Communication

Education

Bachelor's degree in quantitative field

MS/PhD in ML/CS/DS/Statistics

Tools

Tableau

Apache Superset

Databricks

MLflow

Weights & Biases

Job description

Socket.dev in Austin, TX seeks a Data Scientist, AI/ML Model Quality to build validation frameworks, observability metrics, and telemetry for GenAI features across Wallet, Payments, and Commerce. You ensure data quality drives trusted models reaching hundreds of millions of users.

The ideal candidate has strong statistics, Python and SQL skills, experience with ML observability, and familiarity with GenAI/LLMs.