AWS Data Platform Support Engineer

Viaprom Technologies; Mumbai; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Viaprom Technologies is seeking an AWS Data Platform Support Engineer with 3–5 years of production experience to monitor, troubleshoot and maintain AWS-based data platforms. You will handle Airflow DAGs, EMR jobs and Redshift queries, ensuring reliable daily operations and quick incident resolution.

The role requires strong Python and SQL skills, expertise in AWS services (S3, EMR, CloudWatch, Lambda) and a focus on production support rather than ETL development.

Qualifications

• 3–5 years of hands-on AWS Production Support in a data platform environment.
• Strong hands-on experience with AWS services (S3, EMR, CloudWatch, Lambda) in production.
• Airflow production support including DAG monitoring, error detection and recovery.

Responsibilities

• Monitor and provide production support for AWS-based data platforms and workflows.
• Troubleshoot Airflow DAGs, resolve failures, and validate reruns.
• Investigate EMR clusters and Redshift queries, analyze logs and optimize performance.
• Write and optimize SQL queries for data validation and troubleshooting.
• Manage WLM configuration, queues and concurrency for efficient workloads.
• Monitor S3, Lambda, CloudWatch; address file access issues and alerting.
• Develop Python scripts for automation and operational tasks.
• Conduct incident management and root-cause analysis for recovery.

Skills

AWS

Airflow

Python

SQL

Redshift

S3

EMR

CloudWatch

Lambda

Troubleshooting

Production Support

Tools

Terraform

Apache Spark

Hive

Azure DevOps

Job description

Job Summary:

We are seeking a skilled and proactive AWS Data Platform Support Engineer with 3 to 5 years of experience to join our team. This is primarily a Production Support role responsible for monitoring, troubleshooting, incident resolution and maintaining AWS-based data platforms. The candidate must have strong hands-on experience supporting AWS Cloud, S3, EMR, CloudWatch, Lambda, Amazon Redshift, WLM configuration Apache Airflow, DAGs, Python and SQL in a production support environment. The role requires strong troubleshooting skills across Airflow DAGs, EMR jobs, Redshift queries/WLM and AWS services. Candidates with only ETL/Data Pipeline Development experience and without hands-on production support experience will not be considered.

Key Responsibilities:

• Monitor and provide production support for AWS-based data platforms and workflows.
• Monitor and troubleshoot Apache Airflow DAGs, including DAG/task failures, dependency issues, scheduling errors and retries.
• Investigate Airflow logs, identify root cause, resolve issues and rerun failed DAGs/tasks, followed by post-rerun validation.
• Troubleshoot AWS EMR clusters and jobs, including job failures, resource issues, YARN/Spark issues and analyze EMR/CloudWatch logs.
• Provide production support for Amazon Redshift, including query failures, long-running queries, performance and concurrency issues.
• Write and optimize complex SQL queries for production troubleshooting, data validation, issue investigation and performance analysis.
• Troubleshoot and managet WLM configuration, including queues, concurrency, query queueing and workload-related issues.
• Monitor and troubleshoot AWS S3, Lambda and CloudWatch, including file/access issues, Lambda failures, alerts, logs and metrics.
• Develop and maintain Python scripts for support automation, housekeeping and operational activities.
• Perform incident management, root cause analysis, recovery/rerunning of jobs and participate in production support.

Required Skills & Qualifications:

• 3-5 years of hands-on AWS Production Support Data Platform Support L2-L3 Support experience.
• Strong hands-on experience with AWS Cloud, AWS S3, AWS EMR, AWS CloudWatch and AWS Lambda in production environments.
• Strong hands-on experience with Apache Airflow Production Support, including DAG monitoring, error finding, troubleshooting, rerunning and recovery.
• Experience with Airflow DAG development/modification and debugging; primary experience must be in support rather than ETL pipeline development.
• Strong hands-on experience with Amazon Redshift Production Support, including query troubleshooting and performance investigation.
• Candidates must have strong hands-on Redshift SQL query-writing skills.
• WLM Configuration and Troubleshooting is mandatory, including queues, concurrency and query queueing.
• Strong experience with AWS CloudWatch Logs, Metrics and Alarms for production monitoring and troubleshooting.
• Strong proficiency in SQL and Python, specifically for production troubleshooting, validation, scripting and automation.
• Proven experience in incident management, root cause analysis, production recovery, job rerun and validation

Preferred Skills

• Familiarity with Terraform for basic AWS infrastructure configuration and troubleshooting.
• Exposure to Apache Spark, Hive, Azure Devops.

Candidates Profile

• 3-5 years of experience in AWS Data Platform Production Support L2-L3 Support.
• Majority of experience should be in production support, monitoring, troubleshooting and incident resolution, not ETL development.
• Strong hands-on experience with AWS Cloud, S3, EMR, CloudWatch, Lambda and Redshift.
• Hands-on Apache Airflow production support - monitoring DAGs, identifying errors, checking logs, troubleshooting failed tasks/DAGs and rerunning/recovering jobs.
• Hands-on experience with Redshift WLM configuration and troubleshooting, including queues, concurrency and query performance.