Applied ML Engineer

Macroscope Inc.; San Francisco (CA); On-site; USD 170,000 - 280,000; Full time

Job summary

Macroscope Inc. is seeking a Senior Applied ML Engineer to advance our AI capabilities. You will own large parts of the model development lifecycle—from evaluation datasets and experiments to training and fine-tuning models, including RL, and analyzing results to guide product decisions.

You will collaborate with founders and engineering to deploy models into production, improve system reliability, and stay at the frontier of RL training techniques.

Qualifications

• 3+ years of experience in applied machine learning, AI, or related engineering roles.
• Experience building, training, fine-tuning, or evaluating modern ML models in production or research environments.
• Experience with reinforcement learning or reinforcement learning for LLMs (RLHF, RLAIF, PPO, DPO, or similar).
• Strong skills in dataset creation, curation, and evaluation, including benchmarks and labeling strategies.
• Experience designing and running rigorous experiments and using data to drive model improvements.
• Familiarity with LLMs and the open-source model ecosystem.
• Strong software engineering skills and building reliable ML pipelines and tooling.

Responsibilities

• Own large parts of the model development lifecycle: build and maintain evaluation datasets, design experiments, train and fine-tune models including RL where appropriate.
• Analyze results to drive product decisions and improve models.
• Research and stay up to date on RL training techniques.
• Collaborate with product and backend teams to integrate models into production.

Skills

Applied ML

Model training

Reinforcement learning

LLMs familiarity

Dataset creation

Experiment design

Software engineering

Golang

Tools

Golang

Job description

About Macroscope

Macroscope aims to be the source of truth of what's happening for any company that builds software. Our mission is to give leaders clarity and engineers time.

We help leaders understand how their products and codebases are evolving—what’s changing, who’s working on what, and where progress is happening—grounded in the ultimate source of truth: the code.

Macroscope is founded by former entrepreneurs who have started and sold multiple companies, and operated as product/engineering executives at public tech companies. We're fortunate to be supported by the best VC firms and angels in the business, including Lightspeed Venture Partners, Thrive Capital, Google Ventures, and Adverb.

About the role

We're looking for an Applied ML Engineer to help build and improve the machine learning systems that power Macroscope's core AI capabilities. Your primary focus will be on improving model quality through high-quality evaluation datasets, rigorous experimentation, and model training. You'll work closely with our founders and engineering team to determine which models perform best, why they perform the way they do, and how we can continuously improve them.

This is a highly collaborative role where you'll own large parts of the model development lifecycle—from building and maintaining evaluation datasets, to designing experiments, training and fine-tuning models (including reinforcement learning where appropriate), and analyzing results to drive product decisions. You'll also be expected to participate in the research that can push the boundaries of what we are able to do and stay up to date on the latest RL training techniques.

You'll also partner closely with our product and backend engineering teams to integrate new models into production and help shape the future of AI-powered software engineering.

Our technology stack: Typescript/React (front-end), Golang (backend), Temporal, Google Cloud (GCP), Postgres, Terraform, custom-built AST \"code walkers\" in various languages (Golang, Typescript, Swift, Python, Rust)

Qualifications

• 3+ years of experience in applied machine learning, AI, or related engineering roles.




• Experience building, training, fine-tuning, or evaluating modern ML models in production or research environments.




• Experience with reinforcement learning or reinforcement learning for LLMs (RLHF, RLAIF, GRPO, PPO, DPO, or similar techniques). Any practical experience is valuable.




• Strong skills in dataset creation, curation, and evaluation, including designing benchmarks, labeling strategies, and evaluation methodologies.




• Experience designing and running rigorous experiments, analyzing results, and using data to drive model improvements.




• Familiarity with LLMs, reasoning models, and the rapidly evolving open-source model ecosystem.




• Strong software engineering skills and experience building reliable ML pipelines and tooling.




• Comfortable working in a fast-paced, high-agency startup environment where priorities evolve quickly and everyone helps define what to build next.




• Experience in Golang (the primary language of our backend systems) is a plus, but not required.




• Bonus: Experience with large-scale distributed training, preference optimization, synthetic data generation, evaluation frameworks, GCP infrastructure, Temporal, or building internal ML tooling.

The base pay for the Senior Applied ML Engineer position ranges from $170k/year to $280k/year. Pay is based on several factors and may vary depending on job-related knowledge, skills, and experience.

About you

• You are extremely high agency. We are a small startup and we intend to keep an extremely flat organizational structure for as long as possible. Instead of relying on people managers, product managers and heavy processes, we rely on exceptionally talented individuals with high agency to be self-motivated towards contributing to our mission.




• You want to work at an early stage start-up. The default state of any startup is failure. The only way to overcome the daunting odds of making a startup venture successful is for a densely packed group of insanely hard working and talented people to work together to building something useful to and loved by customers. If you're not willing to work extremely hard on something high risk, this startup isn't for you.




• You act like an owner. You put immense care and craft into what you build because you take responsibility for all parts of the product. You don't walk past broken windows.




• You care about what we're building. Life's too short to work on something you're not passionate about. We are a small group of ambitious people who want to build something insanely great that we want to use, and that we think every company will want to use. If our mission and product doesn't resonate with you, we understand and would encourage you to find something that does.