Senior Data Engineer – AWS

ReKnew

Hyderabad

On-site

INR 1,200,000 - 2,000,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

ReKnew is seeking a Senior Data Engineer in Hyderabad to design and optimize AWS-based data solutions. This role involves building data pipelines for analytics, leading modernization initiatives, and collaborating with various teams, requiring extensive AWS, Python, and data engineering experience.

Qualifications

7+ years in tech roles, 5+ in data engineering.
Expertise in AWS and Python required.
Experience with data lakes and MPP data warehouses.

Responsibilities

Design and implement robust ETL/ELT pipelines using AWS.
Lead migration of data from legacy platforms.
Provide mentorship to junior data engineers.

Skills

Python

PySpark

SQL

AWS

DevOps

Data Governance

Education

Bachelor's degree in Computer Science or related field

Tools

AWS Glue

Redshift

Delta Lake

Job description

2 days ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

At ReKnew, our mission is to empower enterprises to revitalize their core business and organization by positioning themselves for the new world of AI. We're a startup founded by seasoned practitioners, supported by expert advisors, and built on decades of experience in enterprise technology, data, analytics, AI, digital, and automation across diverse industries. We're actively seeking top talent to join us in this mission.

Job Description

We're seeking a highly skilled Senior Data Engineer with deep expertise in AWS-based data solutions. In this role, you'll be responsible for designing, building, and optimizing large-scale data pipelines and frameworks that power analytics and machine learning workloads. You'll lead the modernization of legacy systems by migrating workloads from platforms like Teradata to AWS-native big data environments such as EMR, Glue, and Redshift. A strong emphasis is placed on reusability, automation, observability, performance optimization, and managing schema evolution in dynamic data lake environments.

Key Responsibilities

Migration & Modernization: Build reusable accelerators and frameworks to migrate data from legacy platforms (e.g., Teradata) to AWS-native architectures such as EMR, Glue, and Redshift.
Data Pipeline Development: Design and implement robust ETL/ELT pipelines using Python, PySpark, and SQL on AWS big data platforms.
Code Quality & Testing: Drive development standards with test-driven development (TDD), unit testing, and automated validation of data pipelines.
Monitoring & Observability: Build operational tooling and dashboards for pipeline observability, including tracking key metrics like latency, throughput, data quality, and cost.
Cloud-Native Engineering: Architect scalable, secure data workflows using AWS services such as Glue, Lambda, Step Functions, S3, and Athena.
Collaboration: Partner with internal product teams, data scientists, and external stakeholders to clarify requirements and drive solutions aligned with business goals.
Architecture & Integration: Work with enterprise architects to evolve data architecture while securely integrating AWS systems with on-premise or hybrid environments. This includes strategic adoption of data lake table formats like Delta Lake, Apache Iceberg, or Apache Hudi for schema management and ACID capabilities.
ML Support & Experimentation: Enable data scientists to operationalize machine learning models by providing clean, well-governed datasets at scale.
Documentation & Enablement: Document solutions thoroughly and provide technical guidance and knowledge sharing to internal engineering teams.
Team Training & Mentoring: Act as a subject matter expert, providing guidance, training, and mentorship to junior and mid-level data engineers, fostering a culture of continuous learning and best practices within the team.

Qualifications

Experience: 7+ years in technology roles, with at least 5+ years specifically in data engineering, software development, and distributed systems.
Programming:
Expert in Python and PySpark (Scala is a plus).
Deep understanding of software engineering best practices.
AWS Expertise:
3+ years of hands-on experience in the AWS data ecosystem.
Proficient in AWS Glue, S3, Redshift, EMR, Athena, Step Functions, and Lambda.
Experience with AWS Lake Formation and data cataloging tools is a plus.
AWS Data Analytics or Solutions Architect certification is a strong plus.
Strong grasp of distributed data processing.
Experience with MPP data warehouses like Redshift, Snowflake, or Databricks on AWS.
Hands-on experience with Delta Lake, Apache Iceberg, or Apache Hudi for building reliable data lakes with schema evolution, ACID transactions, and time travel capabilities.
DevOps & Tooling:
Experience with version control (e.g., GitHub/CodeCommit) and CI/CD tools (e.g., CodePipeline, Jenkins).
Familiarity with containerization and deployment in Kubernetes or ECS.
Data Quality & Governance:
Experience with data profiling, data lineage, and relevant tools.
Understanding of metadata management and data security best practices.
Bonus:
Experience supporting machine learning or data science workflows.
Familiarity with BI tools such as QuickSight, PowerBI, or Tableau.