Founding ML Engineer: Responsible AI & Compliance

SourceIn; New York (NY); On-site; USD 200,000 - 230,000; Full time

Job summary

SourceIn in the San Francisco Bay Area or New York City Area is seeking a Staff or Principal ML engineer to build the intelligence and compliance layer for an early-stage AI startup. As the founding ML hire, you will own production ML systems, evaluation pipelines, and the strategy for accuracy, latency, and accountability, collaborating closely with the CTO to shape product and technology from day one.

This role offers high ownership, equity, and the chance to establish technical standards,

Qualifications

• Staff or Principal‑level applied ML expertise.
• Ownership of production ML/NLP/LLM systems.

Responsibilities

• Translate customer requirements and regulatory frameworks into measurable detectors for compliance, grounding, hallucinations, risk, escalation, and interaction quality.
• Choose the right combination of rules, classifiers, LLM-based judges, and fallback mechanisms.
• Establish confidence thresholds and promotion criteria based on explicit accuracy, latency, cost, and risk trade-offs.
• Own annotation guidelines, ground-truth datasets, adjudication processes, and offline and online evaluation.
• Build regression gates that prevent ungrounded outputs and unmeasured accuracy from reaching production.
• Turn QA reviews, customer disputes, and human-versus-model disagreements into high-quality training data.

Job description

SourceIn in the San Francisco Bay Area or New York City Area is seeking a Staff or Principal ML engineer to build the intelligence and compliance layer for an early-stage AI startup. As the founding ML hire, you will own production ML systems, evaluation pipelines, and the strategy for accuracy, latency, and accountability, collaborating closely with the CTO to shape product and technology from day one.

This role offers high ownership, equity, and the chance to establish technical standards,