Sr Software Developer - Data (SWF)

BMW TechWorks India Private Limited; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

BMW TechWorks India Private Limited in Pune is seeking a Senior Data Engineer to build scalable ETL pipelines and data transformations using AWS Glue and Athena, and to develop interactive reporting with QuickSight. The role requires hands-on experience with AWS cloud services, Python, Terraform for infrastructure as code, and containerization via Docker, plus CI/CD via GitHub workflows.

You will collaborate with data scientists and business stakeholders to deliver end-to-end analytics solutions

Qualifications

• 6+ years of experience in data engineering or related field.
• Strong proficiency in Python, including data manipulation and automation.
• Hands-on experience with AWS Glue, Athena, and QuickSight.
• Deep understanding of ETL design principles and best practices.
• Experience with Docker for containerization.
• Proficient in AWS CLI and managing AWS resources.
• Familiarity with Terraform for infrastructure as code.
• Experience writing unit and integration tests using pytest.
• Experience with GitHub Actions or other CI/CD tools for deployment automation.

Responsibilities

• Design, develop, and maintain robust ETL jobs to extract, transform, and load data from diverse sources into the data lake and warehouse.
• Implement data transformation pipelines using AWS Glue, integrating with Athena for analytical querying and optimization.
• Develop and automate QuickSight dashboards and reports for business insights and KPI tracking.
• Write clean, efficient, and maintainable Python code for data processing and orchestration.
• Containerize data applications and jobs using Docker for portability and scalability.
• Write and maintain unit and integration tests to ensure data quality.
• Deploy and manage infrastructure and applications using GitHub Actions workflows for CI/CD.
• Collaborate with data scientists, analysts, and business stakeholders to design end-to-end analytical solutions.

Skills

Python

AWS

ETL

Terraform

Docker

GitHub Actions

Snowflake

QuickSight

Athena

S3

IAM

CI/CD

SQL

Education

Bachelor's degree in Computer Science/Information Technology

Tools

GitHub

Terraform

Job description

CI/CD Integration Data Python DevOps AWS Athena Terraform Github Lambda API Gateways, Kinesis, ElasticSearch, ElasticCache, Dynamo DB, Athena, AWS IoT AWS Glue

B.Tech/B.E

Job Description

Location: Pune
Experience: 7-9 years

About the Role

We are looking for a Senior Data Engineer with strong expertise in building scalable ETL pipelines, performing data transformations using AWS Glue and Athena, and developing interactive reporting solutions with Amazon QuickSight.

The ideal candidate should have hands‑on experience with AWS Cloud services, Python, and infrastructure automation using Terraform, along with experience in containerization and CI/CD pipelines via GitHub workflows.

Key Responsibilities

• Design, develop, and maintain robust ETL jobs to extract, transform, and load data from diverse sources into the data lake and warehouse.
• Implement data transformation pipelines using AWS Glue, integrating with Athena for analytical querying and optimization.
• Develop and automate QuickSight dashboards and reports for business insights and KPI tracking.
• Write clean, efficient, and maintainable Python code for data processing and orchestration.
• Containerize data applications and jobs using Docker for portability and scalability.
• Write and maintain unit and integration tests to ensure code reliability and data quality.
• Deploy and manage infrastructure and applications using GitHub Actions workflows for CI/CD.
• Work extensively with AWS services including IAM, S3, Lambda, Snowflake, Lake Formation, and DynamoDB.
• Manage and automate cloud infrastructure using Terraform for consistent, version‑controlled deployments.
• Collaborate with data scientists, analysts, and business stakeholders to design end‑to‑end analytical solutions.

Required Skills & Experience

• 6+ years of experience in data engineering or related field.
• Strong proficiency in Python, including data manipulation and automation.
• Hands‑on experience with AWS Glue, Athena, and QuickSight.
• Deep understanding of ETL design principles and best practices.
• Experience with Docker for containerization.
• Proficient in AWS CLI and managing AWS resources.
• Familiarity with Terraform for infrastructure as code.
• Experience writing unit and integration tests using frameworks like pytest.
• Experience with GitHub Actions or other CI/CD tools for deployment automation.
• Strong problem‑solving and analytical skills with attention to detail.
• Key Skills - Python , AWS (glue, athena, s3, iam, lambda) , Terraform , Github workflow , Git, CI CD

Good to Have

• Prior experience in the automotive domain or working with vehicle data and telematics systems.
• Familiarity with data governance and security frameworks in AWS.

Education

• Bachelor’s degree in computer science, Information Technology, or a related field.