Senior / Lead ML Engineer (Databricks MLOps)

Anblicks; Hyderabad; On-site; INR 2,000,000 - 3,000,000; Full time

Job summary

A leading technology consulting firm is seeking a Senior ML Engineer (Databricks MLOps) to build and operationalize large-scale machine learning pipelines. The role involves collaborating with cross-functional teams to design and implement AI/ML solutions that drive innovation and operational excellence. Candidates should have over 8 years of experience in AI/ML integration and strong programming skills. This position offers the opportunity to work on transformative projects in a cutting-edge environment.

Qualifications

• 8+ years of experience in AI/ML integration and transformation projects.
• 6+ years of professional software development experience.
• Knowledge of modern programming languages such as Python or Java.

Responsibilities

• Build and operationalize machine learning pipelines.
• Develop and maintain enterprise architecture for AI/ML initiatives.
• Lead transformational initiatives to bridge AI/ML capabilities.

Skills

AI/ML integration

MLOps orchestration

Programming skills (Python, Java, etc.)

Data analysis and statistical skills

Education

Bachelor’s Degree in Computer Science

Tools

Databricks

Apache Spark

MLflow

Airflow

Job description

Location

Hyderabad

Experience

8+ Years

Job Description Summary

ML Engineer within the Data Science and Machine Learning team leverages and third‑party software to create solutions to business problems defined by specific business requirements. In this position, you will draw upon technical, AI/ML engineering, data and MLOps experience to solve complex marketing analytics problems on very large volumes of data.

Role

Senior ML Engineer (Databricks MLOps)

Responsibilities

• Build and operationalize machine learning pipelines involving terabytes of data.
• Define requirements, create software designs, implement code to specifications.
• Develop and maintain a comprehensive enterprise architecture for AI/ML/GenAI initiatives, ensuring alignment with overall business strategy and technology roadmap.
• Architect hyperscale MLOps solutions and pipelines.
• Work with Applied Scientists, Data Scientists, Product owners, ML Engineers and Software Engineers to design and deliver ML solutions in production at scale.
• Develop automated AI and ML workflows and end-to-end pipelines for data preparation, training, deployment, monitoring, and ensure quality of architecture and design of ML systems and data infrastructure.
• Leverage AI to develop GenAI powered solutions to complement data science and product build capabilities.
• Assess current state AI/ML/GenAI capabilities across various business domains, identify gaps, and design target state architectures to drive innovation, revenue growth and operational excellence.
• Lead transformational initiatives to bridge the gap between current and desired AI/ML capabilities, collaborating with cross‑functional teams to ensure successful implementation.
• Establish governance frameworks and decision criteria for AI/ML and GenAI projects, ensuring adherence to industry standards, regulatory requirements, Responsible AI and architectural principles.
• Create and maintain reference architectures, patterns and best practices for AI/ML lifecycle and integration within enterprise ecosystem.
• Lead technology evaluation and process improvements to drive experimentation, model development and ML‑Ops at scale.
• Lead and drive standardization of LLM onboarding process, RAG pipelines and application development.
• Conduct architecture reviews and risk assessments for proposed AI/ML solutions, ensuring they meet security, scalability and interoperability requirements.
• Utilize advanced data science techniques (e.g., NLP, clustering, predictive analytics, regression analyses, survival analysis, segmentation, and experimentation) to propose enhancements and innovations to business processes.
• Conduct sophisticated statistical analyses and maintain high reliability of machine learning pipelines in production environments, ensuring minimal downtime and optimal performance.
• Collaborate with business leaders and product to identify opportunities for AI/ML‑driven innovation and guide the development of use cases that deliver tangible business value.
• Foster a culture of continuous learning and innovation in AI/ML practices across the enterprise architecture team and broader organization.

Basic Qualifications

• 8+ years of experience in enterprise architecture, focusing on AI/ML integration and transformation projects.
• 6+ years of professional software development experience.
• Bachelor’s Degree in Computer Science (or Associate Degree with 3+ years of development experience) or equivalent.
• Fundamentals in object‑oriented design, data structures, algorithm design, problem solving and complexity analysis.
• Knowledge of at least one modern programming language such as Python, Java, C++, C, or Perl.

Preferred Qualifications

• 8+ years of experience architecting scalable ML infrastructure and big data systems.
• Databricks Architect Certification required.
• 6+ years of architecting solutions using Databricks, with strong experience using Mosaic AI, Unity Catalogue, mlflow, workflow orchestration and other Databricks native MLOps capabilities.
• At least 1+ year experience in GenAI (technical familiarity with 2 or more of: OpenAI API, Bedrock API, Vertex API, LangGraph, other agentic frameworks).
• High attention to detail and proven ability to manage multiple, competing priorities simultaneously.
• Experience with MLOps orchestration tools such as Airflow, Kubeflow, DAGster, Optuna, Mlflow or similar.
• Experience with operationalizing and migrating ML models into production at scale.
• Develop large‑scale model inference solutions using parallel execution frameworks with Spark, EMR, Databricks.
• Experience with complex orchestration and MLOps pipelines stitching large volumes of data for training and scoring.
• Experience with Large Language Models, fine‑tuning and deployment frameworks such as HuggingFace or cloud provider solutions (e.g., Amazon Bedrock, Vertex AI model garden).
• Familiarity with vector databases such as Pinecone, ChromaDB or similar.
• Experience in CI/CD/DevOps, deployment and automation tools – Jenkins, Terraform, CloudFormation or similar.
• Proficiency with Apache Spark, EMR/DataProc and cloud‑based tools (Snowflake, Redshift, Glue, Step Functions, Lambda, AWS Batch, etc.).
• Experience with ML libraries like H2O, scikit‑learn and deep learning frameworks (PyTorch, TensorFlow, etc.).
• Experience with end‑to‑end software development and lifecycle of ML solutions.
• Excellent technical communication with scientists and engineers.
• At least 2 years of database (SQL) experience and Linux.
• At least 6+ years of AWS infrastructure experience (or equivalent GCP).

What Will Set You Apart

• Databricks Certification.
• LangGraph, Databricks MLflow experience, Docker experience, Kubernetes experience.
• Knowledge of LLM observability platforms.
• Strong communication skills – clear and effective ideas to team and client at multiple levels.
• Analytic problem‑solving skills with ability to think outside the box.
• Analytical mindset excelling at analyzing and understanding data to answer questions.
• Excellent understanding of data concepts, data architecture, data manipulation/engineering and data engineering design.
• Passion for considering how projects fit into the wider business picture.
• Adaptable – able to adapt to diverse technical challenges and systems.
• Self‑starter – able to work independently with little guidance.
• Ability to formulate and present insights with gathered data to both technical and non‑technical peers, leaders and clients.