Software Engineer (AI Tooling and Evaluation)

ScOp Venture Capital LLC.; New York (NY); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Downtown Brooklyn office

Occasional remote work

Job summary

Pangram Labs is seeking a Software Engineer to join the AI Tooling and Evaluation team in Brooklyn. You will build internal tools, web apps, and data pipelines to support the model development lifecycle from data collection to evaluation and deployment monitoring.

You will work closely with AI researchers and engineers, applying ML concepts while delivering robust software solutions and dashboards for model assessments and usage tracking.

Qualifications

• BS/MS in CS or related field.
• 3+ years as a software engineer with full‑stack skills.
• Proficiency in Python and at least one modern frontend framework.
• Experience building internal tools, admin panels, dashboards.
• Comfortable with databases, REST APIs, and basic DevOps.
• Familiarity with ML concepts.

Responsibilities

• Build evaluation infrastructure for standardized model assessments and dashboards.
• Develop research support systems to monitor platform usage.
• Improve synthetic data generation workflows and toolchains.

Skills

Python

Full-stack

REST APIs

Databases

ML concepts

Frontend framework

DevOps

Data visualization

Education

B.S./M.S. in Computer Science

Tools

Weights & Biases

Label Studio

Data visualization libraries

Job description

Software Engineer (AI Tooling and Evaluation)

Pangram Labs is hiring an AI Tooling and Evaluation Engineer to join our AI research team. You'll work directly with AI research scientists and engineers to build internal tools, platforms, and systems that power the model development lifecycle: from data collection and annotation through evaluation and deployment monitoring.

This is a full-stack software engineering role with a research support focus. You will spend most of your time building web applications, APIs, and data pipelines, but you'll need enough ML fluency to understand what the research team needs and why.

Roles and Responsibilities:

• Build evaluation infrastructure for standardized model assessments and sign-off, including dashboards and reporting tools to track performance over time
• Build research support systems to monitor customer usage of the platform
• Improve upon and support synthetic data generation workflows

Requirements

• B.S. / M.S. equivalent in Computer Science or a related domain
• 3+ years of experience as a software engineer, with strong full-stack skills
• Proficiency in Python and at least 1 modern frontend framework
• Experience building internal tools, admin panels, and dashboards
• Comfortable with databases, REST APIs, and basic DevOps
• Familiarity with ML concepts
• Ability to work autonomously and translate ambiguous research needs into concrete software solutions

Nice to have

• Experience with ML experiment tracking tools such as Weights & Biases
• Experience with Label Studio or other annotation platforms
• Background in AI/NLP research
• Experience with data visualization libraries
• Experience with evaluation and/or QA of large scale ML systems

What we offer

• In-person role in our Downtown Brooklyn office, occasional remote work is okay