Principal Consultant- Databricks Developer

1401 Genpact India Pvt. Ltd.; Hyderabad; On-site; INR 2,000,000 - 2,800,000; Full time

Job summary

Genpact in Hyderabad is seeking a Lead Data Engineer to drive end-to-end data solutions using Databricks, Unity Catalog, Python and Spark. You will design scalable ETL pipelines, manage real-time streams with Kafka, and collaborate with data scientists to deliver impactful insights.

The role requires strong data governance and hands-on experience with large-scale data platforms in a fast-moving AI‑driven environment.

Qualifications

• Experience building scalable data pipelines and platforms.
• Strong hands-on with Databricks, Spark, Unity Catalog and Python.
• Knowledge of data governance, security, and data quality.

Responsibilities

• Develop and maintain scalable ETL pipelines using Databricks.
• Implement data processing frameworks with Apache Spark.
• Integrate real-time data streams using Kafka and Databricks.
• Develop data workflows and orchestrate pipelines.
• Design data governance policies and access controls in Unity Catalog.
• Monitor pipeline performance and optimize for scalability.
• Write Python scripts for data extraction, transformation and loading.
• Collaborate with data scientists and analysts.
• Maintain data documentation including data dictionaries and lineage.

Skills

Data Engineering

Databricks Platform

Apache Spark

Python

Kafka

Unity Catalog

ETL pipelines

Data Governance

Data Modeling

Education

Bachelor's - Business Analytics

Bachelor's - Computer Science

Bachelor's - Statistics

Master's - Data Science

Tools

Databricks Platform

Snowflake

Unity Catalog

Python

Kafka

Job description

Lead Data Engineer

Ready to turn bold ideas into real-world impact?

At Genpact, we don’t just adapt to change, we lead it. AI and digital innovation are transforming the way businesses work, and we’re at the forefront of it. Genpact’s AI Gigafactory , our industry-first accelerator, exemplifies how we scale advanced technology solutions to help global enterprises work smarter, grow faster, and transform at scale. Whether tackling complex challenges through large‑scale models or agentic AI , our breakthrough solutions tackle companies’ most complex challenges.

If you thrive in a fast‑moving, innovation‑driven environment, love building and deploying cutting‑edge AI solutions, and want to push the boundaries of what’s possible, this is your moment.

Get to know us at genpact.com and on LinkedIn , YouTube , X , and Facebook .

Job Description

Inviting applications for the role of Principal Consultant- Databricks Developer with experience in Unity Catalog + Python Spark Kafka for ETL!

In this role the Databricks Developer is responsible for solving the real world cutting edge problem to meet both functional and non‑functional requirements.

Responsibilities

• Develop and maintain scalable ETL pipelines using Databricks with a focus on Unity Catalog for data asset management.
• Implement data processing frameworks using Apache Spark for large‑scale data transformation and aggregation.
• Integrate real‑time data streams using Apache Kafka and Databricks to enable near real‑time data processing.
• Develop data workflows and orchestrate data pipelines using Databricks Workflows or other orchestration tools.
• Design and enforce data governance policies access controls and security protocols within Unity Catalog.
• Monitor data pipeline performance troubleshoot issues and implement optimizations for scalability and efficiency.
• Write efficient Python scripts for data extraction transformation and loading.
• Collaborate with data scientists and analysts to deliver data solutions that meet business requirements.
• Maintain data documentation including data dictionaries data lineage and data governance frameworks.

Qualifications

Bachelors - Business Analytics, Bachelors - Computer Science, Bachelors - Statistics, Masters - Data Science

Certifications

Certified Data Management Professional - RiversandRiversand, Databricks Certified Associate Developer for Apache Spark 3.0 - Databricks AcademyDatabricks Academy, Databricks Certified Associate Developer for Apache Spark - Databricks AcademyDatabricks Academy, Microsoft Certified: Azure Data Engineer Associate - MicrosoftMicrosoft, Oracle Database 12c Certified Implementation Specialist - OracleOracle

Required Skills

Agile Methodology, Banking Capital Markets, Client Relations, Cloud Computing, Collaboration Tools, Commercial Sales, Consumer Packaged Goods (CPG), Databricks Platform, Data Engineering, Data Literacy, Design Thinking, Electronic Hardware, Executive Presence, Inclusion, Manufacturing Equipment, People Leadership, Personal Effectiveness, Risk Management, Semiconductors, Snowflake (Platform), Sourcing and Procurement, Storytelling

Language

English

Language Proficiency

Proficient - C2

Job Type

Regular

Master Skill List

Data Engineering

Remote Type

Office

Work Shift

Variable (India)

Why join Genpact?

• Lead AI-powered transformation – Drive innovation and solve real-world business challenges that matter
• Make an impact – Help global enterprises solve business challenges that matter
• Accelerate your career – Gain hands‑on experience, mentorship, and world‑class learning opportunities to stay ahead
• Work with the best – Join 140,000+ bold thinkers and problem‑solvers who push boundaries every day
• Thrive in a values‑driven culture – Our courage, curiosity, and incisiveness - built on a foundation of integrity and inclusion - allow your ideas to fuel progress

Genpact is an Equal Opportunity Employer and considers applicants for all positions without regard to race, color, religion or belief, sex, age, national origin, citizenship status, marital status, military/veteran status, genetic information, sexual orientation, gender identity, physical or mental disability or any other characteristic protected by applicable laws. Genpact is committed to creating a dynamic work environment that values respect and integrity, customer focus, and innovation.