Senior Data Engineer

Entri Jobs; Ernakulam; Presencial; INR 2.500.000 - 4.200.000; Jornada completa

Descripción de la vacante

Entri Jobs is seeking a Senior Data Engineer to design, build, and optimize data pipelines in an AWS-centric environment. You will manage data lakes and warehouses and partner with analytics and ML teams to enable advanced insights.

The role requires hands-on experience with AWS services (Glue, Lambda, Redshift, S3, Athena) and Python/SQL. You will implement ETL/ELT, ensure data quality, and optimize performance and costs.

Formación

• 3+ years of hands-on data engineering experience, with AWS focus.

Responsabilidades

• Design, build, and optimize data ingestion and transformation pipelines using AWS-native tools (Glue, Lambda, Step Functions, EMR, etc.).
• Develop and manage data lakes and data warehouses using Amazon S3, Redshift, and Athena.
• Implement ETL/ELT workflows for structured, semi-structured, and unstructured data at scale.
• Create and maintain data models and schemas that support analytics and ML workflows.
• Work with AWS Glue Catalog, Lake Formation, and AWS DataBrew for data cataloging, cleaning, and governance.
• Collaborate with data scientists to prepare feature-ready datasets for model training and inference.
• Implement data quality, validation, and monitoring mechanisms to ensure accuracy and reliability.
• Optimize data performance and costs through partitioning, compression, and indexing strategies.
• Automate deployment and orchestration of data workflows using AWS CodePipeline, Docker, and CI/CD best practices.
• Ensure compliance, data security, and access control in alignment with AWS IAM and governance standards.

Conocimientos

Python

SQL

PySpark

Data Modeling

ETL/ELT

Distributed Systems

Educación

Bachelor's degree in Computer Science/Engineering or related field

Herramientas

Python

SQL

AWS Glue

Amazon S3

Amazon Redshift

Athena

Kinesis

Step Functions

Lambda

Docker

PySpark

Descripción del empleo

We are seeking a highly skilled Senior Data Engineer to design, build, and optimize robust data pipelines within an AWS-centric ecosystem. You will be responsible for managing scalable data lakes and warehouses, ensuring data quality, and collaborating with cross-functional teams to support advanced analytics and machine learning initiatives.

Responsibilities

• Design, build, and optimize data ingestion and transformation pipelines using AWS-native tools (Glue, Lambda, Step Functions, EMR, etc.).
• Develop and manage data lakes and data warehouses using Amazon S3, Redshift, and Athena.
• Implement ETL/ELT workflows for structured, semi-structured, and unstructured data at scale.
• Create and maintain data models and schemas that support analytics and ML workflows.
• Work with AWS Glue Catalog, Lake Formation, and AWS DataBrew for data cataloging, cleaning, and governance.
• Collaborate with data scientists to prepare feature-ready datasets for model training and inference.
• Implement data quality, validation, and monitoring mechanisms to ensure accuracy and reliability.
• Optimize data performance and costs through partitioning, compression, and indexing strategies.
• Automate deployment and orchestration of data workflows using AWS CodePipeline, Docker, and CI/CD best practices.
• Ensure compliance, data security, and access control in alignment with AWS IAM and governance standards.

Qualifications

• 3+ years of hands‑on experience in Data Engineering, with at least 3 years in AWS environments.
• Proficiency in Python (pandas, PySpark, boto3) and SQL for data transformation and orchestration.
• Expert‑level knowledge of AWS data services: AWS Glue, AWS Lambda, Amazon S3, Amazon Redshift, Athena, Kinesis, and Step Functions.
• Strong understanding of data lake and data warehouse architectures.
• Experience designing and maintaining end‑to‑end data pipelines in production environments.
• Familiarity with event‑driven and batch data processing workflows.
• Strong background in data modeling, schema design, and query optimization.
• Experience integrating data pipelines with ML/AI environments (e.g., SageMaker or Bedrock).
• Excellent problem‑solving and debugging skills in distributed data systems.

Education

Bachelor's degree in Computer Science, Engineering, or a related field preferred.

Skills

Amazon Redshift, Data Modeling, Data Pipelines, Docker, PySpark, Python, SQL, AWS Glue, ETL/ELT, Amazon S3

Experience

4+ years