Senior Software Engineer – AWS Data Pipeline

Jobtailor

Bengaluru

On-site

INR 1,500,000 - 2,100,000

Full time

8 days ago
Application generator

An application made for this job — a tailored resume and cover letter that speak straight to the posting.

Get past ATS filters

Job summary

Jobtailor is seeking a senior data engineer in Bengaluru to design and implement scalable data pipelines using AWS services. You will develop PySpark ETL workflows, build serverless data processing with Lambda, and orchestrate jobs with Step Functions.

You will ensure data quality, governance, and cost optimization while collaborating with Analysts, Architects, and DevOps in a fast-paced environment.

Qualifications

  • 5+ years of experience in data engineering / AWS development.
  • Strong proficiency in Python.
  • Hands-on experience with PySpark for large-scale data processing.
  • Expertise with AWS Glue (ETL jobs).
  • Expertise with AWS Lambda.
  • Expertise with AWS Step Functions.
  • Expertise with AWS Athena.
  • Expertise with AWS S3.
  • Expertise with AWS RDS (MySQL).
  • Strong knowledge of SQL and data querying.
  • Familiarity with IAM roles and AWS security best practices.
  • Preferred: AWS Certification (e.g., AWS Certified Data Engineer / Developer).
  • Preferred: Experience with CI/CD pipelines and DevOps tools.
  • Preferred: Familiarity with Docker / Kubernetes.

Responsibilities

  • Design and implement scalable data pipelines using AWS services.
  • Develop ETL workflows using AWS Glue, PySpark, and Python.
  • Build serverless data processing solutions using AWS Lambda.
  • Orchestrate workflows using AWS Step Functions.
  • Manage and optimize data storage in AWS S3.
  • Work with relational databases like AWS RDS (MySQL).
  • Ensure data quality, validation, and governance.
  • Optimize performance and cost of data pipelines.
  • Debug and troubleshoot data-related issues.
  • Collaborate with cross-functional teams including Data Analysts, Architects, and DevOps.

Skills

Python
PySpark
SQL
Data Engineering
AWS Security Best Practices
CI/CD
Docker
Kubernetes
AWS Glue
AWS Lambda
AWS Step Functions
AWS Athena
AWS S3
AWS RDS MySQL

Tools

AWS Glue
AWS Lambda
AWS Step Functions
AWS Athena
AWS S3
AWS RDS (MySQL)
Docker
Kubernetes
CI/CD Pipelines

Job description

Job Responsibilities
  • Design and implement scalable data pipelines using AWS services
  • Develop ETL workflows using AWS Glue, PySpark, and Python
  • Build serverless data processing solutions using AWS Lambda
  • Orchestrate workflows using AWS Step Functions
  • Manage and optimize data storage in AWS S3
  • Work with relational databases like AWS RDS (MySQL)
  • Ensure data quality, validation, and governance
  • Optimize performance and cost of data pipelines
  • Debug and troubleshoot data-related issues
  • Collaborate with cross-functional teams including Data Analysts, Architects, and DevOps
Requirements
  • 5+ years of experience in data engineering / AWS development
  • Strong proficiency in Python
  • Hands-on experience with PySpark for large-scale data processing
  • Expertise with AWS Glue (ETL jobs)
  • Expertise with AWS Lambda
  • Expertise with AWS Step Functions
  • Expertise with AWS Athena
  • Expertise with AWS S3
  • Expertise with AWS RDS (MySQL)
  • Strong knowledge of SQL and data querying
  • Familiarity with IAM roles and AWS security best practices
  • Preferred: AWS Certification (e.g., AWS Certified Data Engineer / Developer)
  • Preferred: Experience with CI/CD pipelines and DevOps tools
  • Preferred: Familiarity with Docker / Kubernetes
Core Competencies

Demonstrates expertise in designing and implementing scalable data pipelines using AWS services, with strong proficiency in Python and PySpark for ETL workflows. Ensures data quality and governance while optimizing performance and cost of data solutions.

Highest-signal resume keywords
  • AWS Data Engineering
  • Python Programming
  • AWS Glue ETL
  • AWS Lambda
  • AWS Step Functions
Hard Skills
  • Data Pipeline Design
  • ETL Workflows
  • PySpark
  • SQL
  • Data Quality Assurance
Soft Skills
  • Collaboration
  • Problem-Solving
Certifications & Qualifications
  • AWS Certified Data Engineer
  • AWS Certified Developer
Industry Keywords
  • Data Governance
  • Data Validation
  • AWS Security Best Practices
  • Cross-Functional Collaboration
Tools & Technologies
  • AWS S3
  • AWS RDS (MySQL)
  • AWS Athena
  • CI/CD Pipelines
  • Docker
Get your free, confidential resume review.
or drag and drop your file here.
Similar jobs

Similar jobs worth comparing

Aws Data Engineer
Aws Data Engineer

Tata Consultancy Services • India

On-site
INR 900,000 - 1,500,000
Aws Data Engineer
Aws Data Engineer

Shrewd Techlink Services • Hyderabad, Pune District, Bengaluru

On-site
INR 1,800,000 - 3,000,000
AWS Data Engineer
AWS Data Engineer

Careernet • Kolkata District, Bengaluru

On-site
INR 900,000 - 1,500,000
Senior Data Engineer
Senior Data Engineer

AagatiServe Pvt Ltd • Delhi

On-site
INR 1,800,000 - 2,400,000
Aws Data Engineer
Aws Data Engineer

Tranzeal • Bengaluru

Hybrid
INR 1,200,000 - 1,800,000
Sr. Data Engineer (Consultant)
Sr. Data Engineer (Consultant)

Minfy • Chennai District

On-site
INR 1,800,000 - 2,800,000
Data Engineer (AWS)
Data Engineer (AWS)

Weekday (YC W21) • Gurugram District

On-site
INR 1,500,000 - 2,500,000
Aws Data Engineer
Aws Data Engineer

Tata Consultancy Services • Indore District

On-site
INR 1,000,000 - 2,000,000
Lead AWS Data Engineer
Lead AWS Data Engineer

Weekday (YC W21) • Bengaluru

On-site
INR 1,000,000 - 1,500,000
AWS Data Engineer
AWS Data Engineer

Objectways • Bengaluru

On-site
INR 1,500,000 - 2,200,000