Principal Data Engineer

Jobtailor; Hyderabad; On-site; INR 3,500,000 - 7,000,000; Full time

Job summary

Jobtailor is seeking an experienced AWS Data Engineer to design and architect scalable data pipelines using native AWS services. You will drive MDM initiatives with Stibo and optimize ETL workflows across enterprise data platforms, while ensuring security and compliance.

Collaborate with cross-functional teams and mentor engineers in a fast-paced environment. Ideal candidates have 7–10 years of hands-on AWS data engineering experience, strong Spark/Scala skills, and relevant certifications.

Qualifications

• 7–10 years of experience in advanced AWS data engineering including significant MDM with Stibo
• Advanced proficiency in AWS services such as SNS, SQS, API Gateway, Athena, CloudFormation, CloudWatch, DMS, DynamoDB, EMR, Kinesis, OpenSearch, Redshift, SCT
• Expertise in designing and managing data pipelines and ETL workflows on AWS
• Strong experience with Spark using Scala for large-scale data processing
• Hands-on experience with Oozie workflow scheduling and orchestration
• Real-time data streaming with AWS Kinesis
• Experience tuning Redshift for analytics workloads
• DynamoDB for NoSQL solutions
• Automating infrastructure via CloudFormation
• Proficient in monitoring with CloudWatch
• Data migration with SCT and DMS
• MDM implementation with Stibo
• Bachelor’s degree in related field
• AWS Data Analytics – Specialty or AWS Solutions Architect – Professional certifications
• Apache Spark certification preferred

Responsibilities

• Design scalable data pipelines on AWS for complex business needs
• Lead MDM implementations with Stibo in enterprise data platforms
• Collaborate with cross-functional teams to integrate data with analytics platforms
• Mentor junior engineers and drive adoption of new data technologies
• Ensure data security, quality, and compliance across platforms

Skills

AWS Data Engineering

MDM with Stibo

ETL design

Apache Spark/Scala

Oozie

Real-time streaming

Redshift

Kinesis

DynamoDB

CloudFormation

CloudWatch

DMS

MDM - Stibo integration

Education

Bachelor's degree in CS/IT/Data Engineering

AWS certifications (preferred)

Tools

Amazon API Gateway

OpenSearch

Job description

Design and architect scalable data pipelines and solutions using AWS native services to support complex business requirements
Develop, implement, and optimize ETL processes leveraging AWS technologies such as EMR, Redshift, Athena, and DMS
Configure and manage real-time data streaming and ingestion frameworks utilizing Kinesis and Amazon API Gateway
Monitor, troubleshoot, and enhance data infrastructure performance using CloudWatch and Open Search
Automate deployment and infrastructure provisioning with CloudFormation and manage schema migrations using SCT
Lead the implementation and integration of Master Data Management (MDM) solutions using Stibo within enterprise data platforms
Collaborate with cross-functional teams to integrate data solutions with enterprise applications and analytics platforms
Ensure data security, quality, and compliance by implementing best practices and leveraging AWS security features
Lead technical reviews, mentor junior engineers, and drive adoption of emerging data technologies within the organization

Requirements

• 7 to 10 years of experience in advanced AWS data engineering, including significant expertise in Master Data Management (MDM) with Stibo and designing large-scale data solutions
• Advanced proficiency in AWS services including SNS, SQS, Amazon API Gateway, Athena, CloudFormation, CloudWatch, DMS, DynamoDB, EMR, Kinesis, Open Search, Redshift, SCT
• Expertise in designing and managing data pipelines and ETL workflows on AWS
• Strong experience with Spark using Scala for large-scale data processing
• Hands-on experience with Oozie workflow scheduling and orchestration
• Deep knowledge of real-time data streaming with AWS Kinesis
• Proven ability to configure and optimize Amazon Redshift for analytics workloads
• Experience with DynamoDB for NoSQL database solutions
• Ability to automate infrastructure and deployments using AWS CloudFormation
• Proficient in monitoring and logging with AWS CloudWatch
• Skill in managing data migration and schema transformation using AWS SCT and DMS
• Demonstrated expertise in implementing and integrating MDM solutions using Stibo
• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a closely related discipline
• AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect – Professional
• Certification in Apache Spark or relevant big data technologies (preferred)

Core Competencies

Demonstrates advanced expertise in AWS data engineering, focusing on designing scalable data pipelines and implementing Master Data Management (MDM) solutions using Stibo. Proficient in optimizing ETL processes and ensuring data security and compliance within enterprise data platforms.

Highest-signal resume keywords

• AWS Data Engineering
• Master Data Management (MDM)
• ETL Process Optimization
• Real-Time Data Streaming
• AWS CloudFormation Automation

Hard Skills

• AWS EMR
• AWS Redshift
• AWS Athena
• AWS DMS
• AWS Kinesis
• AWS CloudWatch
• AWS SCT
• Apache Spark
• Scala
• Oozie

Soft Skills

• Mentoring
• Collaboration
• Troubleshooting

Certifications & Qualifications

• AWS Certified Data Analytics – Specialty
• AWS Certified Solutions Architect – Professional
• Apache Spark Certification

Industry Keywords

• Data Pipelines
• Data Solutions
• Data Security
• Data Quality
• Compliance

Tools & Technologies

• Amazon API Gateway
• Open Search
• DynamoDB
• Data Ingestion Frameworks
• Data Migration Tools