Lead Software Engineer (Big Data)

Circana; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Circana in India is seeking a data engineer with 4–7 years of experience to design, build, and optimize data pipelines and warehouse solutions on AWS, Azure, and GCP. You will collaborate with cross‑functional teams to translate data requirements into scalable, high‑quality data assets.

You will mentor junior engineers, contribute to governance and security, and help drive performance tuning, CI/CD, and analytics delivery using Spark, SQL, and visualization tools like Power BI or Tableau.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 4–7 years of experience in data engineering with cloud focus.
• Proficiency in Python, PySpark, and SQL for data processing and transformation.
• Hands-on experience with Cloud platforms (AWS, Azure, or GCP) and their data services.

Responsibilities

• Design, develop, and optimize ETL/ELT pipelines using PySpark, Python, and SQL.
• Architect and implement data warehouse solutions on Cloud platforms (AWS, Azure, GCP).
• Collaborate with cross-functional teams to understand data requirements and deliver high-quality solutions.
• Learn and work with in-house data tools and frameworks, contributing to their enhancement and integration.
• Ensure data quality, governance, and security across all data assets.
• Mentor junior engineers and promote best practices in data engineering.
• Drive performance tuning, cost optimization, and automation of data workflows.
• Participate in Agile ceremonies, code reviews, and technical design discussions.

Skills

Python

PySpark

SQL

AWS

Azure

GCP

Data Warehousing

Dimensional Modeling

Data Architecture

Git

CI/CD

Agile

Apache Airflow

Power BI

Tableau

Leadership

Education

Bachelor’s or Master’s degree in Computer Science or related field

Tools

Apache Airflow

Power BI

Tableau

Job description

Let’s be unstoppable together! At Circana, we are fueled by our passion for continuous learning and growth, we seek and share feedback freely, and we celebrate victories both big and small in an environment that is flexible and accommodating to our work and personal lives. We’re a global company dedicated to fostering inclusivity and belonging. We value and celebrate the unique experiences, cultures, and viewpoints that each individual brings. By embracing a wide range of backgrounds, skills, expertise, and beyond, we create a stronger, more innovative environment for our employees, clients, and communities. Circana is proud to be Certified™ by Great Place To Work®. This prestigious award is based entirely on what current employees say about their experience working at Circana.

What will you be doing?

• Design, develop, and optimize ETL/ELT pipelines using PySpark, Python, and SQL.
• Architect and implement data warehouse solutions on Cloud platforms (AWS, Azure, GCP).
• Collaborate with cross-functional teams to understand data requirements and deliver high-quality solutions.
• Learn and work extensively with in-house data tools and frameworks, contributing to their enhancement and integration.
• Ensure data quality, governance, and security across all data assets.
• Mentor junior engineers and promote best practices in data engineering.
• Drive performance tuning, cost optimization, and automation of data workflows.
• Participate in Agile ceremonies, code reviews, and technical design discussions.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• 4–7 years of experience in data engineering, with a strong focus on cloud-based data solutions.
• Proficiency in Python, PySpark, and SQL for data processing and transformation.
• Hands-on experience with Cloud platforms (AWS, Azure, or GCP) and their data services.
• Solid understanding of data warehousing concepts, dimensional modeling, and data architecture.
• Experience with version control (Git), CI/CD pipelines, and Agile methodologies.
• Ability to quickly learn and adapt to internal tools and technologies.
• Experience with orchestration tools like Apache Airflow
• Knowledge of data visualization tools (e.g., Power BI, Tableau).
• Strong communication and leadership skills.

Circana Behaviors

• Stay Curious: Being hungry to learn and grow, always asking the big questions
• Seek Clarity: Embracing complexity to create clarity and inspire action
• Own the Outcome: Being accountable for decisions and taking ownership of our choices
• Center on the Client: Relentlessly adding value for our customers
• Be a Challenger: Never complacent, always striving for continuous improvement
• Champion Inclusivity: Fostering trust in relationships engaging with empathy, respect and integrity
• Commit to each other: Contributing to making Circana a great place to work for everyone

Location

• This position can be located in the following area(s): Bangalore/Pune

Prospective candidates may be asked to consent to background checks (in accordance with local legislation and our candidate privacy notice). Your current employer will not be contacted without your permission.