Senior AWS Data Engineer

RDsolutions India; Hyderabad; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

RDsolutions India seeks a Senior AWS Data Engineer to design and optimize scalable data pipelines in a Databricks-based cloud environment. You will transform data using SQL and Python and model data warehouses in AWS Redshift, enabling robust data analytics and governance across teams.

Collaborate with engineering and analytics to ensure data quality, reliability, and performance while orchestrating workflows with Airflow or MWAA in a large-scale cloud data platform.

Qualifications

• Experience building ETL/ELT pipelines.
• Strong SQL, Python and PySpark skills.
• Solid data modeling and analytics data structures.
• Experience with Databricks including Delta Lake and Databricks SQL.
• Experience with AWS data services (S3, Lambda, Glue, Redshift).
• Experience with workflow orchestration tools such as Airflow or MWAA.

Responsibilities

• Design and maintain scalable data pipelines and transformation workflows using Databricks architecture.
• Develop data transformations using SQL and Python.
• Design and implement data warehouse data model in AWS Redshift.
• Orchestrate data pipelines using Airflow.
• Work with Databricks-based data environments to process and manage large datasets.
• Ensure reliability and performance of data ingestion, transformation, and processing pipelines.
• Collaborate with engineering and analytics teams to support data quality, data governance, and analytics delivery.
• Troubleshoot pipeline issues and support improvements in platform stability and performance.

Skills

ETL pipelines

SQL

Python

Pyspark

Data modeling

Analytics data

Databricks

Delta Lake

Databricks SQL

AWS data services

Airflow

Tools

Databricks

Delta Lake

Databricks SQL

S3

Lambda

Glue

Redshift

MWAA

Airflow

GitHub

Terraform

Jira

Job description

We are seeking Senior AWS Data Engineers with good experience in data engineering, analytical data modeling, and cloud-based data platforms.

Key Responsibilities

• Design and maintain scalable data pipelines and transformation workflows using Databricks architecture
• Develop data transformations using SQL and Python
• Design and implement data warehouse data model in AWS Redshift
• Orchestrate data pipelines using Airflow
• Work with Databricks-based data environments to process and manage large datasets
• Ensure reliability and performance of data ingestion, transformation, and processing pipelines
• Collaborate with engineering and analytics teams to support data quality, data governance, and analytics delivery
• Troubleshoot pipeline issues and support improvements in platform stability and performance

Required Skills:

• Data Engineering experience building ETL / ELT pipelines
• Good experience in SQL, Python & Pyspark
• Strong understanding of data modeling and analytics data structures.
• Working experience with Databricks, including: Delta Lake, Databricks SQL.
• Experience working with AWS data services (S3, Lambda, Glue, Redshift, etc.)
• Experience with workflow orchestration tools such as MWAA or Airflow

Good to have:

• Experience with GitHub, Terraform, and Jira