Principal Data Engineer

Brillio; Hyderabad; On-site; INR 2,500,000 - 4,200,000; Full time

Job summary

Brillio is seeking a Senior Tech Specialist for Applications & Platforms in Hyderabad. The role focuses on advanced AWS data engineering, MDM with Stibo, and designing large-scale data solutions.

Responsibilities include building scalable data pipelines, ETL processes, real-time streaming with Kinesis, and deploying infrastructure via CloudFormation. Mentoring engineers and ensuring data security are key expectations.

Qualifications

• 7–10 years of experience in AWS data engineering.
• MDM experience with Stibo architecture and implementations.
• Designing and delivering large-scale data solutions on AWS.

Responsibilities

• Design and architect scalable data pipelines using AWS to support complex business requirements.
• Develop, implement and optimize ETL processes with EMR, Redshift, Athena and DMS.
• Configure real-time data streaming with Kinesis and API Gateway.
• Monitor and troubleshoot data infrastructure using CloudWatch and OpenSearch.
• Automate deployment and schema migrations with CloudFormation and SCT.
• Lead MDM implementations using Stibo in enterprise data platforms.
• Collaborate with cross-functional teams to integrate data solutions with enterprise apps and analytics.
• Ensure data security, quality and compliance with AWS security features.
• Lead technical reviews, mentor juniors and drive adoption of data tech.

Skills

AWS data engineering

MDM Stibo

Spark Scala

Oozie workflow

Kinesis streaming

Redshift optimization

DynamoDB

CloudFormation

CloudWatch

ETL pipelines

API Gateway

SCT

OpenSearch

DMS

Education

Bachelor’s degree in Computer Science / IT / Data Engineering

AWS Data Analytics – Specialty

AWS Solutions Architect – Professional

Apache Spark certification

Tools

EMR

Athena

Redshift

DMS

Kinesis

OpenSearch

Job description

Senior Tech Specialist, Applications & Platforms

Job requirements

Experience Range: 7 to 10 years of experience in advanced AWS data engineering, including significant expertise in Master Data Management (MDM) with Stibo and designing large-scale data solutions Key Responsibilities:

• Design and architect scalable data pipelines and solutions using AWS native services to support complex business requirements
• Develop, implement, and optimize ETL processes leveraging AWS technologies such as EMR, Redshift, Athena, and DMS
• Configure and manage real-time data streaming and ingestion frameworks utilizing Kinesis and Amazon API Gateway
• Monitor, troubleshoot, and enhance data infrastructure performance using CloudWatch and Open Search
• Automate deployment and infrastructure provisioning with CloudFormation and manage schema migrations using SCT
• Lead the implementation and integration of Master Data Management (MDM) solutions using Stibo within enterprise data platforms
• Collaborate with cross-functional teams to integrate data solutions with enterprise applications and analytics platforms
• Ensure data security, quality, and compliance by implementing best practices and leveraging AWS security features
• Lead technical reviews, mentor junior engineers, and drive adoption of emerging data technologies within the organization

Required Skills:

• Advanced proficiency in AWS services including SNS, SQS, Amazon API Gateway, Athena, CloudFormation, CloudWatch, DMS, DynamoDB, EMR, Kinesis, Open Search, Redshift, SCT
• Expertise in designing and managing data pipelines and ETL workflows on AWS
• Strong experience with Spark using Scala for large-scale data processing
• Hands-on experience with Oozie workflow scheduling and orchestration
• Deep knowledge of real-time data streaming with AWS Kinesis
• Proven ability to configure and optimize Amazon Redshift for analytics workloads
• Experience with DynamoDB for NoSQL database solutions
• Ability to automate infrastructure and deployments using AWS CloudFormation
• Proficient in monitoring and logging with AWS CloudWatch
• Skill in managing data migration and schema transformation using AWS SCT and DMS
• Demonstrated expertise in implementing and integrating MDM solutions using Stibo

Preferred Skills:

• Experience integrating AWS data solutions with third-party analytics and visualization tools
• Proficiency in optimizing performance for large-scale distributed data systems
• Knowledge of emerging AWS data services and staying current with industry best practices
• Experience with Open Search for search and analytics use cases
• Background in mentoring and leading engineering teams in cloud data projects
• Familiarity with advanced data governance and data quality frameworks in MDM environments

Desired Qualifications:

• Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a closely related discipline
• AWS Certified Data Analytics – Specialty or AWS Certified Solutions Architect – Professional
• Certification in Apache Spark or relevant big data technologies (preferred)