Software Engineer II (Data Engineering)

7-Eleven Global Solution Center, India

Bengaluru

In loco

INR 1.200.000 - 1.800.000

Tempo pieno

5 giorni fa

Candidati tra i primi

Aumenta le possibilità di ricevere una risposta da questo datore di lavoro

Il tuo curriculum e una lettera di presentazione personalizzati, che corrispondono esattamente a ciò che sta cercando.

Supera i filtri ATS

Descrizione del lavoro

7-Eleven Global Solution Center, India is seeking a Software Engineer II (Data Engineering) in Bangalore to design, build, and optimize data pipelines using Spark on Databricks and Synapse. You will ingest data from Azure sources and on-prem systems, implement full/incremental loads, and prepare analytics-ready datasets for reporting.

You will also secure data processing, optimize performance, and collaborate with BI teams to support Power BI and AAS datasets.

Competenze

5+ years of experience in statistics and analytics on large-scale datasets.
Strong SQL and data analysis skills, including SQL DW concepts.
Experience with Spark ecosystem: PySpark, Databricks, Synapse Notebooks (1+ year hands-on).
Proficiency in Azure Data Services: Azure Synapse, Azure Data Factory, Azure Data Lake, Azure security model.
Experience building pipelines for full and incremental data loads from Azure and on-prem sources.
Strong knowledge of data management, processing, cleaning, transformation, aggregation, and modeling.
Working knowledge of Power BI reports and dashboards (AAS / PBI).
Strong programming skills in Python (Java is a plus).
Experience with FastAPI, RESTful services, and microservices architecture.
Proficiency with databases such as MySQL, PostgreSQL, and MongoDB.
Experience deploying applications using Docker and Kubernetes.
Experience with streaming and messaging systems: Kafka, Kinesis, RabbitMQ.
Familiarity with Azure Git and DevOps practices.
Exposure to the retail industry is a plus.

Mansioni

Design, develop, and maintain scalable data processing pipelines using the Spark ecosystem (PySpark, Databricks, Synapse Notebooks).
Develop and manage data pipelines using Azure Data Factory, Synapse, and Data Lake for ingestion and processing.
Implement full and incremental data load strategies across sources and targets.
Perform data cleansing, transformation, aggregation, and modeling for analytics-ready datasets.
Apply Azure security, access control, and data governance practices.
Optimize Spark jobs and SQL workloads for performance and cost efficiency.
Collaborate with BI/analytics teams to support Power BI datasets/models (AAS, PBI).
Troubleshoot data pipelines, perform root-cause analysis and fixes.
Develop reusable frameworks for data ingestion, transformation, validation, monitoring.
Work with stakeholders to translate data requirements into scalable solutions.

Conoscenze

Statistics & analytics

SQL & data analysis

Spark ecosystem

Python programming

Mentoring & leadership

Communication

Formazione

Bachelor's degree in Computer Science or related field

Master's degree is a plus

Strumenti

Azure Data Factory

Azure Synapse

Azure Data Lake

Azure Analysis Services

Power BI

Docker

Kubernetes

MySQL

PostgreSQL

MongoDB

Kafka

Kinesis

Descrizione del lavoro

Job: Software Engineer II (Data Engineering)

Location: Bangalore

Responsibilities

Design, develop, and maintain scalable data processing pipelines using the Spark ecosystem, including PySpark, Databricks, and Synapse Notebooks.
Develop and manage data pipelines using Azure Data Factory, Azure Synapse, and Azure Data Lake to ingest, process, and integrate data from Azure and on-premises sources.
Implement full and incremental data load strategies, ensuring efficient, reliable, and timely movement of data across source systems and target platforms.
Perform data cleansing, transformation, aggregation, and modeling to prepare high-quality datasets for analytics, reporting, and downstream consumption.
Apply Azure security, access control, and data governance practices to ensure secure and compliant data processing and storage.
Optimize Spark jobs, data pipelines, and SQL workloads for performance, scalability, reliability, and cost efficiency.
Collaborate with BI and analytics teams to support Power BI reports and dashboards, including datasets/models hosted on Azure Analysis Services (AAS) and Power BI.
Troubleshoot data pipeline and processing issues, perform root-cause analysis, and implement appropriate fixes to maintain data availability and quality.
Develop reusable frameworks, processes, and best practices for data ingestion, transformation, validation, and monitoring.
Work closely with business stakeholders, data analysts, and engineering teams to translate data requirements into scalable and maintainable data solutions.

Key Skills Required

Data engineering & Analytics

5+ years of experience in statistics and analytics on large-scale datasets
Strong SQL and data analysis skills, including SQL DW concepts
Experience with Spark ecosystem: PySpark, Databricks, Synapse Notebooks (1+ year hands-on)
Proficiency in Azure Data Services:
- Azure Synapse
- Azure Data Factory
- Azure Data Lake
- Azure security model
Experience building pipelines for full and incremental data loads from Azure and on-prem sources
Strong knowledge of data management, processing, cleaning, transformation, aggregation, and modeling
Working knowledge of Power BI reports and dashboards (AAS / PBI)