Lead Data Engineer

Circana; Bengaluru; On-site; INR 3,500,000 - 6,500,000; Full time

Job summary

Circana in Bengaluru is seeking a strategic and technically strong Lead Data Engineer to drive the architecture, development, and optimization of our cloud-based data warehouse and analytics platforms. You will lead a team of data engineers and collaborate with stakeholders across the organization to deliver scalable, high-performance data solutions.

The role requires deep expertise in Python, PySpark, SQL, and cloud data services (AWS/Azure/GCP), plus leadership experience and a strong sense of

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 7+ - 11 years of experience in data engineering, with a strong focus on cloud-native data solutions.
• Deep expertise in Python, PySpark, and SQL for data processing and analytics.
• Proven experience with Cloud platforms (AWS, Azure, or GCP) and their data services.
• Strong understanding of data warehousing concepts, data modeling, and distributed systems.

Responsibilities

• Lead the design and development of complex ETL/ELT pipelines using PySpark, Python, and SQL.
• Architect and manage data warehouse solutions on Cloud platforms (AWS, Azure, GCP).
• Own and drive the adoption and enhancement of in-house data tools.
• Collaborate with data scientists, analysts, and business teams to deliver actionable insights.
• Establish and enforce best practices in data engineering, including coding standards, testing, and documentation.

Skills

Leadership

Stakeholder management

Communication

Agile methodologies

Education

Bachelor’s or Master’s degree in Computer Science, Engineering, or related field

Tools

PySpark

Python

SQL

Airflow

Power BI

Tableau

AWS

Azure

GCP

Job description

Let’s be unstoppable together!
Circana is a leading provider of technology, AI, and data solutions for consumer packaged goods companies, manufacturers, and retailers. Our predictive analytics and Liquid Data® platform help clients measure market share, uncover consumer behavior, and drive growth—powered by six decades of expertise and an expansive, high-quality data set.

Let’s be unstoppable together!
Circana is a leading provider of technology, AI, and data solutions for consumer packaged goods companies, manufacturers, and retailers. Our predictive analytics and Liquid Data® platform help clients measure market share, uncover consumer behavior, and drive growth—powered by six decades of expertise and an expansive, high-quality data set.
At Circana, we are fueled by our passion for continuous learning and growth, we seek and share feedback freely, and we celebrate victories both big and small in an environment that is flexible and accommodating to our work and personal lives.We’re a global company dedicated to fostering inclusivity and belonging. We value and celebrate the unique experiences, cultures, and viewpoints that each individual brings. By embracing a wide range of backgrounds, skills, expertise, and beyond, we create a stronger, more innovative environment for our employees, clients, and communities. With us, you can always bring your full self to work.Join our inclusive, committed team to be a challenger, own outcomes, and stay curious together. Circana is proud to be Certified™ by Great Place To Work®. This prestigious award is based entirely on what current employees say about their experience working at Circana. Learn more at www.circana.com.

What will you be doing?

We are looking for a strategic and technically strong Lead Data Engineer to drive the architecture, development, and optimization of our cloud-based data warehouse and analytics platforms. This role involves leading a team of data engineers, collaborating with stakeholders across the organization, and working with both open-source and proprietary in-house tools to deliver scalable, high-performance data solutions.

Job Responsibilities

• Lead the design and development of complex ETL/ELT pipelines using PySpark, Python, and SQL.
• Architect and manage data warehouse solutions on Cloud platforms (AWS, Azure, GCP), ensuring scalability, reliability, and cost-efficiency.
• Own and drive the adoption and enhancement of in-house data tools, ensuring seamless integration with existing systems.
• Collaborate with data scientists, analysts, and business teams to understand data needs and deliver actionable insights.
• Establish and enforce best practices in data engineering, including coding standards, testing, and documentation.
• Mentor and guide junior and senior engineers, fostering a culture of technical excellence and continuous learning.
• Lead performance tuning, data governance, and automation initiatives across the data platform.
• Participate in strategic planning, roadmap development, and cross-functional leadership discussions.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 7+ - 11 years of experience in data engineering, with a strong focus on cloud-native data solutions.
• Deep expertise in Python, PySpark, and SQL for data processing and analytics.
• Proven experience with Cloud platforms (AWS, Azure, or GCP) and their data services.
• Strong understanding of data warehousing concepts, data modeling, and distributed systems.
• Experience working with and enhancing internal/in-house tools for data processing and analytics.
• Excellent leadership, communication, and stakeholder management skills.
• Experience with orchestration tools like Apache Airflow
• Knowledge of BI tools (e.g., Power BI, Tableau) and data visualization best practices.
• Experience in Agile environments and DevOps practices

Circana Behaviors

As well as the technical skills, experience and attributes that are required for the role, our shared behaviors sit at the core of our organization. Therefore, we always look for people who can continuously champion these behaviors throughout the business within their day-to-day role:

• Stay Curious: Being hungry to learn and grow, always asking the big questions.
• Seek Clarity: Embracing complexity to create clarity and inspire action.
• Own the Outcome: Being accountable for decisions and taking ownership of our choices.
• Center on the Client: Relentlessly adding value for our customers.
• Be a Challenger: Never complacent, always striving for continuous improvement.
• Champion Inclusivity: Fostering trust in relationships engaging with empathy, respect, and integrity.
• Commit to each other: Contributing to making Circana a great place to work for everyone.

Location

This position can be located in the following area(s): Circana India Office, Bangalore Location
An offer of employment may be conditional upon successful completion of a background check in accordance with local legislation and our candidate privacy notice. Your current employer will not be contacted without your permission.