Senior Data Engineer

SII Group India; Dadri; On-site; INR 700,000 - 1,200,000; Full time

Job summary

A leading technology firm in India seeks a Data Engineer responsible for designing, building, and optimizing scalable ETL pipelines using Snowflake and AWS services. The ideal candidate will maintain data models for efficient querying and collaborate with cross-functional teams for data integration and visualization. Strong skills in SQL, Terraform, and data governance are essential. This role requires a proactive problem-solver who can ensure data quality and performance across systems.

Qualifications

• Proficiency in Snowflake for data warehousing, including data loading, transformation, and optimization.
• Hands-on experience with AWS tools for data processing and storage.
• Experience building and maintaining end-to-end data pipelines using relevant tools.
• Solid experience in integrating and visualizing data in reporting tools.
• Experience in Infrastructure as Code using Terraform to manage cloud resources.
• Strong SQL skills for data querying, transformation, and troubleshooting.
• Familiarity with Python for building custom data pipelines and automation.
• Understanding of data governance principles and security practices.

Responsibilities

• Design and build scalable ETL/ELT pipelines to process and transform data.
• Maintain data models within Snowflake for optimized querying.
• Leverage AWS services for data storage and orchestration.
• Use Terraform to automate cloud infrastructure deployments.
• Collaborate with BI teams to integrate data into dashboards.
• Implement best practices for data quality and governance.
• Optimize data systems for performance and reliability.
• Provide ongoing support for data systems and pipelines.

Skills

Experience with Snowflake

AWS Expertise

Data Pipeline Development

Tableau

Terraform

SQL Proficiency

Programming

Data Governance & Security

Communication

Tools

AWS S3

AWS Redshift

AWS Lambda

AWS Glue

Apache Airflow

DBT

Docker

Kubernetes

Job description

• Data Pipeline Development: Design, build, and optimize scalable ETL/ELT pipelines to process and transform data from various sources into Snowflake.
• Data Modeling: Build and maintain data models within Snowflake for optimized querying and reporting.
• Cloud Infrastructure: Leverage AWS services (e.g., S3, Redshift, Lambda, Glue) for data storage, processing, and orchestration.
• Automation & Infrastructure as Code: Use Terraform to automate and manage cloud infrastructure deployments and ensure scalability, reliability, and efficiency.
• Reporting & Visualization: Collaborate with BI teams to integrate data with Tableau for reporting, dashboards, and analytics.
• Data Quality & Governance: Implement best practices for data quality, governance, and security in line with company policies.
• Performance Optimization: Continuously monitor and improve the performance of data systems and pipelines, ensuring low-latency and high-availability.
• Collaboration: Work closely with cross-functional teams (data scientists, analysts, product managers) to deliver actionable insights and products.
• Troubleshooting & Support: Provide ongoing support to ensure that data systems and pipelines are running smoothly and addressing issues as they arise.

Skills & Qualifications:

• Experience with Snowflake: Proficiency in Snowflake for data warehousing, including data loading, transformation, and optimization.
• AWS Expertise: Hands-on experience with AWS tools such as S3, Redshift, Lambda, Glue, and others for data processing and storage.
• Data Pipeline Development: Experience building and maintaining end-to-end data pipelines using tools like Apache Airflow, DBT, or similar.
• Tableau: Solid experience in integrating and visualizing data in Tableau for reporting and dashboard creation.
• Terraform: Experience in Infrastructure as Code (IaC) using Terraform to manage cloud resources.
• SQL Proficiency: Strong SQL skills for data querying, transformation, and troubleshooting.
• Programming: Familiarity with Python or other programming languages for building custom data pipelines and automation.
• Data Governance & Security: Understanding of data governance principles, security best practices, and compliance requirements.
• Communication: Strong communication skills to collaborate with technical and non-technical teams.

Nice to Have:

• Experience with containerization (Docker, Kubernetes).
• Knowledge of machine learning models and integration into data pipelines.
• Agile or Scrum methodology experience.
• Familiarity with CI/CD processes for data engineering workflows.