Lead ML Engineer

Persistent Systems; Bengaluru Urban; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Persistent Systems is seeking an experienced ML engineer to own and deliver ML-powered systems end-to-end, from problem framing to production monitoring. You will build rankers and recommender-style models, integrate LLM components, and deploy robust data pipelines in a cloud-first environment.

The role emphasizes production-grade code, AB tests, and strong collaboration with data, product, and software teams to move from prototypes to scalable products.

Qualifications

• 6+ years of experience in applied machine learning or a closely related engineering discipline.
• Track record of shipping ML to production with real traffic and clients.
• End-to-end ownership of ML-powered systems or products.
• Proficiency in Python and the standard ML/data stack (LLM/agent orchestration, RAG, vector databases, AWS).
• Strong software engineering fundamentals to own a service end-to-end.
• Comfort in ambiguity with bias toward iterative delivery.

Responsibilities

• Build, validate, and ship applied ML models and rankers in production.
• Integrate ML models and LLM-based components into asynchronous pipelines and multi-agent graphs.
• Stand up evaluation, monitoring, and data-quality discipline for production models.
• Build fast, reproducible data pipelines in the cloud using managed services.
• Decide when not to use ML and justify simpler baselines with evidence.
• Own ML-powered systems end to end from problem framing to operation.

Job description

• Build, validate, and ship applied ML models that require real ML judgment, including channel rankers using learning-to-rank and gradient-boosted approaches, peer-network embeddings and similarity, statistical normalization of noisy LLM outputs, and confidence calibration; deploy them into production behind clean, maintainable contracts
• Integrate ML models and LLM-based components into decoupled, asynchronous pipelines and multi-agent graphs, ensuring they are reliable, observable, and safe under real concurrent production load
• Stand up evaluation, monitoring, and data-quality discipline that confirms whether a model is actually working in production, including offline metrics that predict online behavior, drift detection, and reconciliation of projected vs. real impact
• Build fast, reproducible data pipelines in the cloud by reflex, reaching for managed cloud services over one-off scripts, automating data movement, cleaning, and preparation so that the data layer never becomes the bottleneck to ML delivery velocity
• Decide when not to use ML and defend the simpler answer; reach for the simplest approach that will work best—a rules-based scorer before a trained ranker, a clean baseline before a complex model—and earn the right to add complexity with evidence
• Ship thin first versions into real environments fast, measure against reality, and iterate in tight loops; prioritize putting something narrow in front of real data or real users early rather than waiting for a complete spec
• Own ML-powered systems and products end to end—from problem framing through production and operation—including when models drift, when data is dirty, and when latency spikes
• Design for failure before success: identify silent-failure modes, build guardrails, and treat the risk register as your responsibility, including prompt-injection paths on live data and bad recommendations before clients see them
• Set technical direction and raise the delivery bar for others on the team; make the people around you better through mentorship, code quality, and sound architectural judgment
• Iterate with the broader organization as the team moves from prototypes to products, building delivery momentum through frequent shipping and learning from real usage

Qualifications:

Minimum Qualifications

• 6+ years of experience in applied machine learning or a closely related engineering discipline
• Demonstrated, production-proven ML depth: able to reason from a commercial problem to an appropriate ML formulation, choose and justify the method, and explain how you would know it is working, including the failure modes such as leakage, drift, and offline/online metric divergence
• Track record of shipping ML to production—models that served real traffic and real clients, not only notebooks, papers, or proofs of concept
• End-to-end ownership of at least one meaningful ML-powered system or product, from problem framing through production and ongoing operation
• Proficiency in Python and the standard ML/data stack, including LLM/agent orchestration and RAG, vector databases, and AWS cloud infrastructure
• Demonstrated ability to build fast, reproducible data pipelines using managed cloud services, with a consistent pattern of automating data movement and preparation rather than hand-rolling one-off scripts
• Strong software-engineering fundamentals sufficient to write production-quality code and own a service end to end, not only throwaway scripts or exploratory notebooks
• Comfort starting in ambiguity without a detailed spec, and a visible bias toward iterative delivery over big-bang completion
• Experience using AI tools (ChatGPT, Copilot, Claude, etc.) to accelerate and elevate your outcomes; including but not limited to communication drafting, data analysis, prompt engineering, and/or documentation

Preferred Qualifications

• Bachelor’s, Master’s, or PhD in Computer Science, Machine Learning, Statistics, or a related field
• Experience shipping a recommender, ranker, forecasting, or similar applied-ML product that demonstrably moved a business metric
• Retail, e-commerce, or marketplace domain experience
• Experience making LLM/agentic systems reliable and safe in production, including guardrails, evaluation suites, and monitoring, beyond prototyping
• AWS-native, event-driven or serverless architecture experience (the platform these products run against is AWS, TypeScript, and streaming/event-driven)
• Experience founding, co-founding, or joining a product company early, demonstrating ownership of outcomes rather than only models
• Strong communication skills—able to explain ML trade-offs, failure modes, and design decisions clearly to technical and non-technical stakeholders
• Low ego and high agency, you ask for help fast rather than spinning, makes colleagues better, and treats data-quality work and pipeline plumbing as core to doing ML well