Senior Data Engineer

Chandigarh News

Mohali, Chandigarh

Presencial

INR 900 000 - 1 500 000

Tempo integral

14 dias+

Uma candidatura completa num minuto

Currículo e carta de apresentação personalizados, prontos a enviar.

Ultrapassa os filtros ATS

Resumo da oferta

Chandigarh News is seeking a Data Engineer to design and build scalable data pipelines using AWS and PySpark. You will ingest, transform, and organize large datasets for analytics and reporting.

You will develop ETL/ELT workflows, build batch and streaming pipelines on cloud platforms like AWS, and write optimized SQL and PySpark queries. You will ensure performance, reliability, and seamless data access while collaborating with data architects, analytics, and platform engineers.

Qualificações

  • Hands-on experience with AWS (mandatory) including Glue, Lambda, Kinesis, SQS, DynamoDB or NoSQL databases.
  • Strong PySpark (batch and streaming) and SQL/data processing skills.
  • Experience building scalable ETL/ELT pipelines for large-scale data.
  • Ability to work with large-scale structured and semi-structured datasets.
  • Familiarity with REST APIs and data integration.
  • Exposure to Customer Data Platforms or similar.
  • Experience with streaming frameworks like Kafka and data governance concepts.
  • 5+ years in data engineering; 4+ years in PySpark pipelines; 4+ years with AWS data services.

Responsabilidades

  • Develop ETL/ELT workflows to integrate data from multiple sources.
  • Build batch and streaming pipelines for large-scale processing on AWS.
  • Write and optimize complex SQL and PySpark transformations for analytics.
  • Ensure pipelines are performant, scalable, and reliable.
  • Troubleshoot data ingestion, processing, and platform integration issues.
  • Support data model implementation and pipeline migrations across environments.
  • Collaborate with data architects, analytics teams, and platform engineers.
  • Participate in agile ceremonies and engineering discussions.

Conhecimentos

AWS
PySpark
SQL
Python
ETL/ELT
Data Pipelines
Streaming
Kafka
REST APIs
DynamoDB

Descrição da oferta de emprego

We are seeking a Data Engineer with strong experience in building scalable data pipelines and processing large datasets using AWS and PySpark. The role involves designing robust data ingestion and transformation frameworks, enabling efficient data integration, and ensuring high performance and reliability of data systems.

Key Responsibilities
  • Develop ETL/ELT workflows to integrate data from multiple enterprise sources.
  • Build batch and streaming pipelines for large-scale data processing, preferable using cloud platforms such as AWS
  • Write and optimize complex SQL and PySpark transformations for analytics and reporting.
  • Ensure pipelines are performant, scalable, and reliable.
  • Troubleshoot and resolve issues related to data ingestion, processing, and platform integrations.
  • Support data model implementation and pipeline configuration migrations across environments.
  • Collaborate with data architects, analytics teams, and platform engineers to enable efficient data access.
  • Participate in agile ceremonies and technical discussions while contributing to engineering best practices.
Required Skills
  • Strong hands-on experience with AWS (mandatory) Key services include: AWS Glue AWS Lambda Amazon Kinesis Data Streams Amazon SQS DynamoDB or other NoSQL databases Athena / Redshift (preferred) Programming & Data Processing Python PySpark (batch and streaming) SQL Data Engineering
  • Experience building scalable data pipelines and ETL/ELT frameworks
  • Experience working with large-scale structured and semi-structured datasets
  • Ability to write and optimize complex SQL queries and PySpark transformations
  • Experience integrating applications using REST APIs. Good to Have
  • Exposure to Adobe Experience Platform (AEP) or other Customer Data Platforms.
  • Experience with streaming frameworks such as Kafka. Understanding of data modeling, data governance, and large-scale analytics systems.
  • Experience 5 years of professional experience in data engineering 4+ years of experience building PySpark-based data pipelines (batch and streaming) 4+ years of experience with AWS data engineering services
Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Senior Data Engineer
Senior Data Engineer

AagatiServe Pvt Ltd • Delhi

Presencial
INR 1 800 000 - 2 400 000
Senior Data Engineer
Senior Data Engineer

Unison Group • Hyderabad

Presencial
INR 800 000 - 1 500 000
Aws Data Engineer
Aws Data Engineer

Qentelli LLC • Bengaluru

Presencial
INR 1 400 000 - 2 200 000
Data Engineer - ETL
Data Engineer - ETL

Forward Eye Technologies • Pune District

Presencial
INR 1 500 000 - 2 100 000
Senior Data Engineer
Senior Data Engineer

Flipkart Group • Bengaluru

Presencial
INR 1 100 000 - 2 100 000
Data Engineer
Data Engineer

Grid Dynamics • Bengaluru

Híbrido
INR 900 000 - 1 400 000
Senior Data Engineer
Senior Data Engineer

Mags Consulting • Chennai

Presencial
INR 1 200 000 - 2 200 000
Senior Data Engineer
Senior Data Engineer

LPL Financials • Hyderabad

Presencial
INR 1 200 000 - 1 800 000
Data Engineer
Data Engineer

Appsierra Group • Maharashtra

Presencial
INR 1 200 000 - 2 100 000
Senior Data Engineer
Senior Data Engineer

Impronics Technologies • Indore

Presencial
INR 1 200 000 - 1 800 000