Senior Technical Specialist - AWS/AWS Lambda/AWS Glue/AWS SNS

Datavail; Bengaluru; On-site; INR 3,000,000 - 5,500,000; Full time

Job summary

Datavail is seeking a Senior Data Engineer / AWS Data Engineer for Bengaluru, Mumbai, and Hyderabad to design and maintain scalable data pipelines. You will leverage AWS data services, Python, PySpark, and SQL to integrate cloud and on‑prem data sources and optimize ETL workflows.

The ideal candidate has 10–12 years of hands-on data engineering experience, strong experience with Redshift, RDS, SSIS/SSMS, Terraform, and IaC, and a track record of delivering enterprise-scale data platforms.

Qualifications

• AWS Lambda, Glue, SNS and orchestration services experience.
• Strong hands-on Python programming and PySpark.
• Advanced SQL, data transformations and performance tuning.
• Experience with Amazon Redshift and Amazon RDS.
• ETL development and maintenance with SSIS.
• SQL Server development and troubleshooting with SSMS.
• Infrastructure as Code using Terraform.
• Configuration/automation with YAML.
• Version control with Git.
• Containerization with Podman.
• VS Code as primary IDE.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
• Build cloud-native data solutions using AWS services (Lambda, Glue, SNS).
• Develop data processing jobs with Python and PySpark.
• Create advanced SQL queries and data transformations.
• Work with Redshift and RDS for storage and analytics.
• Develop and maintain SSIS packages and workflows.
• Utilize SSMS for database development and performance optimization.
• Implement IaC via Terraform and YAML.
• Manage code with Git and containerize workloads with Podman.
• Use VS Code as primary development environment.

Skills

AWS

Python

PySpark

SQL

Redshift

RDS

SSIS

SSMS

Git

Podman

VS Code

Education

Bachelor's or Master's degree in Computer Science/IT/Engineering

Tools

Terraform

YAML

Job description

Location

Bengaluru, Mumbai, Hyderabad.

We are looking for an experienced Senior Data Engineer / AWS Data Engineer with 10-12 years of hands-on experience in designing, developing, and maintaining scalable data engineering solutions. The ideal candidate should have strong expertise in AWS data services, Python, PySpark, SQL, ETL, infrastructure as code, and data orchestration. The candidate will be responsible for building robust data pipelines, integrating cloud and on-premises data sources, optimizing data processing workflows, and supporting enterprise-scale data platforms.

Job Description

We are looking for an experienced Senior Data Engineer / AWS Data Engineer with 10-12 years of hands-on experience in designing, developing, and maintaining scalable data engineering solutions. The ideal candidate should have strong expertise in AWS data services, Python, PySpark, SQL, ETL, infrastructure as code, and data orchestration. The candidate will be responsible for building robust data pipelines, integrating cloud and on-premises data sources, optimizing data processing workflows, and supporting enterprise-scale data platforms.

Job Title: Senior Technical Specialist

Education: Any Graduate

Experience: 10+years

Location: Bengaluru, Mumbai, Hyderabad

Technology

Required Expertise

AWSLambda, Glue, SNS, orchestration and related data services

PythonStrong hands-on experience

PySparkStrong hands-on experience

SQLAdvanced SQL, query optimization, data transformation

RedshiftDevelopment, optimization, data loading

RDSDatabase development and management

SSISETL development and maintenance

SSMSSQL Server development and troubleshooting

TerraformInfrastructure as Code

YAMLConfiguration/automation

GitVersion control and source code management

PodmanContainerization

VS CodeDevelopment environment

Good to Have

• Experience with Connect
• Experience with TOLSE / TOLENT
• Experience with CI/CD and DevOps practices.
• Experience working in Agile/Scrum environments.
• Knowledge of cloud security, IAM, monitoring, and logging.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 10 -12 years of professional experience in data engineering, ETL, cloud data platforms, or related technologies.
• Strong problem-solving and analytical skills.
• Excellent communication and collaboration skills.
• Ability to work independently as well as within a cross-functional team.

Target Profile: Senior Data Engineer / AWS Data Engineer / Lead Data Engineer with 10 -12 years of experience

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.
• Develop cloud-native data solutions using AWS services, including Lambda, Glue, SNS, and orchestration services.
• Build and optimize data processing jobs using Python and PySpark.
• Develop complex SQL queries, stored procedures, and data transformations.
• Work with Amazon Redshift and Amazon RDS for data storage, processing, and analytics.
• Develop and maintain ETL packages and workflows using SSIS.
• Use SSMS for database development, troubleshooting, and performance optimization.
• Implement infrastructure and configuration using Terraform and YAML.
• Work with Git for source code management, branching, merging, and version control.
• Containerize and manage development workloads using Podman.
• Use VS Code as the primary development environment.
• Implement event-driven and automated workflows using AWS services such as Lambda, SNS, and orchestration frameworks.
• Monitor, troubleshoot, and optimize data pipelines and AWS workloads.
• Ensure data quality, reliability, security, and performance across data platforms.
• Collaborate with architects, developers, business analysts, and other stakeholders to deliver data solutions.
• Participate in code reviews, technical discussions, and production support activities.

About Us

Datavail is a leading provider of data management, application development, analytics, and cloud services, with more than 1,000 professionals helping clients build and manage applications and data via a world-class tech-enabled delivery platform and software solutions across all leading technologies. For more than 17 years, Datavail has worked with thousands of companies spanning different industries and sizes, and is an AWS Advanced Tier Consulting Partner, a Microsoft Solutions Partner for Data & AI and Digital & App Innovation (Azure), an Oracle Partner, and a MySQL Partner.

About The Team

Datavail's Data Management Services: Datavail's Data Management and Analytics practice is made up of experts who provide a variety of data services including initial consulting and development, designing and building complete data systems, as well as ongoing support and management of database, data warehouse, data lake, data integration, and virtualization and reporting environments. Datavail's team is comprised of not just excellent BI & analytics consultants, but great people as well. Datavail's data intelligence consultants are experienced, knowledgeable and certified in the best in breed BI and analytics software applications and technologies. We ascertain your business objectives, goals and requirements, assess your environment, and recommend the tools which best fit your unique situation. Our proven methodology can help your project succeed, regardless of stage. With the combination of a proven delivery model and top-notch experience ensures that Datavail will remain the Data Management experts on demand you desire. Datavail's flexible and client focused services always add value to your organization.