Data Scientist - Human-in-the-Loop Quality Analytics

Socket.dev; Shelton (CT); On-site; USD 90,000 - 130,000; Full time

Job summary

Sperry is building a data science team to retrospectively score analyst decisions, understand where the pipeline errs, and improve training, tooling, and thresholds. You will work closely with risk analytics and production engineering in a hands-on, impact-driven role with real data and tight collaboration with operations.

The ideal candidate thrives on challenging problems, communicates clearly with stakeholders, and can lead workstreams while delivering robust, well-documented code and

Qualifications

• Strong proficiency in Python (NumPy, Pandas, Scikit-learn, or similar) and SQL.
• Applied statistics including classification metrics, sampling, inter-rater agreement, bias, and experimental design.
• Supervised machine learning on imbalanced and imperfectly labeled data.
• Experience evaluating decisions against late, partial, or no ground truth.
• Ability to explain methods and uncertainty to operational audiences.
• Familiarity with Git, CI/CD pipelines, and agile practices.

Responsibilities

• Build retrospective scoring of analyst dispositions using available data.
• Quantify agreement and variation across analysts, shifts, territories, and test conditions.
• Separate analyst-attributable outcomes from detection-threshold and data-quality effects.
• Identify factors behind missed defects and express them as a usable taxonomy.
• Turn findings into feedback loops: training content, guidance, tooling changes, threshold recommendations.
• Define and maintain measures of decision quality over time.

Skills

Python

SQL

Applied statistics

Machine learning

Ground truth evaluation

Communication

Git

CI/CD

Agile development

Problem solving

Self-direction

Leadership (potential)

Education

Bachelor's degree in statistics, CS, or engineering

Tools

Version control

Job description

Sperry is building a data science team to retrospectively score analyst decisions, understand where the pipeline errs, and improve training, tooling, and thresholds. You will work closely with risk analytics and production engineering in a hands-on, impact-driven role with real data and tight collaboration with operations.

The ideal candidate thrives on challenging problems, communicates clearly with stakeholders, and can lead workstreams while delivering robust, well-documented code and