Senior Data Engineer (Sr. Technology Consultant)
Position Overview
A Data Engineering role with SRE background, you will be responsible for managing, maintaining,
and troubleshooting cloud data pipelines. The ideal candidate will have extensive experience in
cloud data engineering, with in-depth knowledge of cloud platform services and data pipeline
architecture along with independently tackling problems and troubleshooting issues. Additionally, you
will leverage your software engineering skills to optimize data pipelines and enhance their reliability
with your out of the box thinking and proactive approach.
Key Responsibilities
- Continuously optimize data pipeline efficiency and reduce operational costs and reduce
- the number of issues/failures
- Monitoring and alerting for Data Pipelines
- Analyze, troubleshoot and resolve complex data pipeline issues.
- Automate repetitive tasks in data processing and management
- Experience in monitoring and alerting for Data Pipelines.
- Continuously improve data pipeline reliability through analysis and testing
- Perform SLA-oriented monitoring for critical pipelines and provide suggestions as well as
- implement post-business approval for SLA adherence if needed
- Monitor performance and reliability of data pipelines, Informatica ETL workflows, MDM
- and Control-M jobs.
- Maintain infrastructure reliability for data pipelines, Informatica ETL workflows, MDM and
- Conduct post-incident reviews and implement improvements for data pipelines
- Develop and maintain documentation for data pipeline systems and processes
Qualifications:
- 5+ years of industry experience in the field of Data Engineering support and enhancement
- Bachelor’s degree in Computer Science or related technical field, or equivalent practical
- experience.
- Proficient in any Cloud Platform services (GCP, Azure, AWS, etc.)
- Strong understanding of data pipeline architectures and ETL processes
- Excellent Python programming skills for data processing and automation
- Excellent SQL query writing skills for data analysis and Experience with relational databases
- Familiarity with version control systems like Git
- Software engineering experience in optimizing data pipelines to improve performance and
- reliability
- Experience with Data Visualization using Google Looker Studio, Tableau, Domo, Power BI,
- or similar tools is an added advantage
- Excellent communication and documentation skills
- Strong problem-solving and analytical skills
Work Setup
- Hybrid Setup - Once to twice a month onsite schedule
- Night shift schedule but should be flexible with schedule.