AWS DE- Multi Cloud

Virtusa; Chennai District; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

Virtusa in Chennai (Tamil Nadu, India) seeks an experienced Data Engineer to join our data engineering practice. You will design and implement scalable batch and real-time data pipelines, work with PySpark and Python, and partner with architects to deliver robust data solutions.

You will leverage AWS services such as Glue, Redshift, S3, EMR, and Athena, model data in Redshift, enforce data quality and governance, and implement secure, auditable pipelines with CI/CD and orchestration via Airflow

Qualifications

• 7+ years in Data Engineering and Data Warehousing.
• Hands-on with AWS Glue, Redshift, S3, EMR, and Athena.
• Design, build, and optimize scalable batch and real-time data pipelines using PySpark and Python.
• Expertise in Redshift modeling, optimization, and S3 data lake architecture.
• Workflow orchestration with Airflow or AWS Step Functions.
• CI/CD integration, deployment automation.
• Data quality, encryption, IAM roles, and governance policies.
• Advanced SQL optimization for large-scale enterprise data.
• Collaborate with cross-functional teams, architects, and stakeholders to deliver solutions.

Skills

Data pipelines

PySpark

Python

SQL optimization

Data warehousing

Airflow

Tools

AWS Glue

Amazon Redshift

Amazon S3

EMR

Athena

Apache Airflow

AWS Step Functions

CI/CD

IAM & Security

Job description

• Experience Required: 7+ years in Data Engineering and Data Warehousing.
• Core AWS Services: Hands-on expertise with AWS Glue, Amazon Redshift, Amazon S3, EMR, and Athena.
• Pipeline Development: Design, build, and optimize scalable batch and real-time data pipelines using PySpark and Python.
• Data Warehousing: Expertise in Amazon Redshift modeling, optimization, and S3 data lake architecture.
• Workflow Orchestration: Manage dependencies and automate tasks using Apache Airflow or AWS Step Functions.
• Infrastructure & DevOps: CI/CD integration, infrastructure support, and deployment automation.
• Data Quality & Security: Implement data validation, encryption, IAM roles, and governance policies.
• Query Performance: Advanced SQL optimization for large-scale enterprise data.
• Collaboration: Partner with cross-functional teams, architects, and stakeholders to deliver solutions.