Technical Lead

KPI Partners; Bengaluru; Presencial; INR 4.200.000 - 7.000.000; Jornada completa

Descripción de la vacante

KPI Partners is seeking a Lead Databricks Engineer to join our data-focused team and drive scalable solutions using Databricks, PySpark, and MLOps. The role emphasizes building robust data pipelines and collaborating with data scientists to translate requirements into technical outcomes.

The candidate will establish CI/CD practices, mentor engineers, and optimize data workflows in a fast-paced environment across multiple business units.

Formación

• Bachelor’s degree in CS/Engineering or related field.
• 5+ years of data engineering experience with at least 2 years in Databricks.
• Proficient in PySpark for data processing.
• Experience implementing CI/CD in data projects.
• Knowledge of MLOps concepts is a plus.

Responsabilidades

• Lead the design and implementation of Databricks-based data solutions.
• Develop, test, and deploy data processing pipelines using PySpark.
• Collaborate with data scientists and analysts to implement MLOps practices for deploying machine learning models.
• Establish and maintain CI/CD pipelines to automate the deployment process and ensure high-quality code delivery.
• Optimize data workflows and improve system performance within the Databricks environment.
• Mentor and guide junior engineers and team members in best practices and emerging technologies.
• Work closely with stakeholders to understand business requirements and translate them into technical specifications.

Conocimientos

Databricks

Pyspark

CI/CD

MLOps

Problem solving

Team collaboration

Educación

Bachelor’s degree in CS/Engineering

Herramientas

Databricks ecosystem

CI/CD tools

Descripción del empleo

KPI Partners is a 5-time Gartner-recognized consulting firm, specializing in Data Analytics and Digital Transformation. With over 19 years of experience, we are a pioneer in AI consulting. Headquartered in Newark, California, we also maintain regional offices across major U.S. cities including Boston, Charlotte, New York, Houston, Cincinnati, Los Angeles, Atlanta, and Washington, D.C. Our global presence extends offshore to India—with offices in Bangalore, Pune, and Hyderabad—and nearshore through our location in Mexico. We proudly serve over 500 clients worldwide across diverse industries such as Retail & CPG, Manufacturing, Life Sciences, Energy & Utilities, Financial Services, and Insurance.

Job Title: Lead Databricks Engineer (Databricks)

Exp: 8 to 12 years

KPI Partners is seeking a skilled and experienced Lead Databricks Engineer to join our dynamic team. The ideal candidate will have a strong background in Databricks, Pyspark, CI/CD practices, and MLOps. This role will be pivotal in developing data solutions that drive business insights and enhance operational efficiency.

Key Responsibilities:

• Lead the design and implementation of Databricks-based data solutions.
• Develop, test, and deploy data processing pipelines using Pyspark.
• Collaborate with data scientists and analysts to implement MLOps practices for deploying machine learning models.
• Establish and maintain CI/CD pipelines to automate the deployment process and ensure high-quality code delivery.
• Optimize data workflows and improve system performance within the Databricks environment.
• Mentor and guide junior engineers and team members in best practices and emerging technologies.
• Work closely with stakeholders to understand business requirements and translate them into technical specifications.

Required Skills:

• Proficient in Databricks and its ecosystem.
• Strong experience with Pyspark for data processing and analysis.
• Solid understanding of CI/CD practices and tools.
• Familiarity with MLOps concepts and their application in real-world scenarios.
• Excellent problem-solving skills and attention to detail.
• Ability to work collaboratively in a fast-paced, team-oriented environment.

Qualifications:

• Bachelor’s degree in computer science, Engineering, or a related field.
• 5+ years of experience in data engineering or a related role, with at least 2 years of experience specifically in Databricks.
• Proven experience in leveraging Pyspark for data processing.
• Experience implementing CI/CD practices in data projects.
• Knowledge of machine learning concepts and MLOps practices is a plus.