AI / ML Engineer

New Street Tech; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

New Street Tech in Bengaluru is seeking an Experienced ML Engineer to build and maintain the model pipelines behind our product, including fine-tuning, retrieval, evaluation datasets, and the scoring harness that decides ship readiness.

You will work under the AI/ML Lead with AI fellows, focusing on production-ready experiments, cost and latency awareness, and documenting experiments. This is applied engineering, not research.

Qualifications

• 3–5 years of hands-on experience in applied machine learning or a relevant AI fellowship with production impact.
• Strong Python skills with daily PyTorch and Hugging Face usage.
• You can run a fine-tune and explain the metric movement.
• Experience building and tuning retrieval pipelines in production.

Responsibilities

• Build and maintain model pipelines, including fine-tuning, retrieval components, and evaluation harness.
• Own retrieval quality: chunking, embedding, reranking, and measuring impact.
• Write evaluation datasets and regression tests to quantify model quality.
• Perform error analysis to identify causes and improve systems, not prompts.
• Monitor and optimize inference cost and latency.
• Collaborate with AI fellows to move experiments toward production standards.

Skills

Python

Applied ML

Experiment design

Reading papers

Production ML

Tools

PyTorch

Hugging Face

Job description

Build and maintain the model pipelines behind our product — fine-tuning, retrieval, evaluation and the harness that decides what ships.

Experience

3–5 years

Location

Level

Experienced

About the Role

Our model layer is the product, not a feature of it. This role builds and maintains the pipelines behind it — fine-tuning runs, retrieval components, evaluation datasets and the scoring harness that decides whether a model version ships.

You work under the AI/ML Lead alongside our AI fellows, which means part of the job is setting the standard for how experiments are run and recorded. This is applied engineering, not research: the measure is whether it works in production for a client.

Key Responsibilities

Build model pipelines

Data preparation, fine-tuning runs, checkpoint management and deployment.

Own retrieval quality

Chunking, embedding, reranking — and measuring whether each change actually helped.

Write the evaluations

Build golden datasets and regression suites so model quality is a number, not an opinion.

Do the error analysis

When output is wrong, find out why and fix the cause rather than the prompt.

Watch cost and latency

Track inference cost per transaction and keep it inside the envelope.

Work with the fellows

Review their experiments and help them reach production standard.

What You Bring

• 3–5 years in applied machine learning, or an AI fellowship with demonstrated production contribution.
• Strong Python; PyTorch and the Hugging Face ecosystem as daily tools.
• You have run a fine-tune and can explain why it did or did not move the metric.
• Retrieval pipelines built and tuned in production — and measured, not assumed.
• You write evaluation datasets rather than judging model output by eye.
• You can read a paper and tell whether it is worth implementing.

What Would Be Great to Have

• OCR and document extraction pipelines.
• Speech and audio processing.
• Quantisation and inference optimisation.
• Model governance and documentation under ISO 42001.