Lead Data Engineer – R01571409

Brillio; Bengaluru; On-site; INR 3,000,000 - 5,500,000; Full time

Job summary

Brillio in Bengaluru, India, is seeking an experienced Data Engineer with 8–10 years of hands-on experience in data engineering, Snowflake, dbt and AWS cloud services. You will develop and optimize dbt models and manage Snowflake data warehouses while ensuring robust governance and scalable ETL/ELT pipelines.

The role requires strong SQL, Python, and cloud automation skills, plus CI/CD implementation for data deployments and proactive monitoring of data quality across platforms.

Qualifications

• Advanced proficiency in SQL required.
• Expertise in developing and managing dbt models, macros, and tests.
• Hands-on Snowflake data warehousing with Time Travel and Fail Safe.
• Strong ETL/ELT fundamentals.
• Proficiency in Python for data engineering tasks.
• Snowflake warehouses: databases, schemas, roles and security.
• Management of AWS services (S3, Lambda, Secrets Manager, IAM, CloudWatch).
• CI/CD pipelines for data deployments.
• Monitoring, alerting and data quality checks in cloud data platforms.

Responsibilities

• Develop, maintain, and optimize dbt models, macros, and tests.
• Administer Snowflake data warehouses and ensure governance.
• Optimize complex SQL queries and warehouse performance.
• Manage AWS services to build reliable cloud data infrastructure.
• Implement and maintain CI/CD pipelines for data deployments.
• Configure monitoring and data quality checks across platforms.
• Troubleshoot production issues and perform root cause analysis.
• Collaborate with analytics, BI, and engineering teams.
• Medallion modeling across Bronze → Silver → Gold.
• Governance alignment with Data Governance for certifications.
• Downstream support for dashboards and data products.
• Documentation of data lineage and model decisions.
• Maintain and develop APIs.

Skills

SQL proficiency

dbt models

Snowflake

ETL/ELT fundamentals

Python

Snowflake administration

AWS services

CI/CD pipelines

Monitoring & data quality

Education

Bachelor's or Master's in CS/IT/Engineering

Tools

Snowflake

dbt

Python

AWS

CI/CD

Job description

Experience Range: 8- 10 years of experience in data engineering, with hands-on expertise in Snowflake, dbt, and AWS cloud services

Key Responsibilities:

• Develop, maintain, and optimize dbt models, macros, and tests to support scalable ETL/ELT data pipelines
• Administer and manage Snowflake data warehouses, including databases, schemas, roles, and security configurations to ensure robust data governance
• Optimize complex SQL queries and warehouse performance, reducing processing times and improving storage utilization
• Manage and integrate AWS services such as S3, Lambda, Secret Manager, IAM, and CloudWatch to build and maintain reliable cloud data infrastructure
• Implement and maintain CI/CD pipelines for automated deployment and release management of data solutions
• Configure and monitor data quality checks, alerting systems, and performance monitoring to ensure high data reliability and integrity
• Troubleshoot and resolve production issues, conducting thorough root cause analysis to minimize downtime and prevent recurrence
• Collaborate closely with analytics, business intelligence, and engineering teams to deliver high-impact, scalable data solutions aligned with business objectives
• Medallion modeling: Design, build, and maintain dbt models across Bronze → Silver → Gold layers for the assigned domain.
• Governance alignment: Partner with the Data Governance team so models meet certification and quality-gate standards before promotion.
• Downstream support: Support data modeling for downstream analytics platform consumers (dashboards and data products)
• Troubleshooting & support: Diagnose and resolve pipeline issues; participate in on-call/support rotation as needed.
• Documentation: Document data lineage, model logic, and key technical decisions so the work is maintainable by the internal team.
• Maintain and Develop APIs

Required Skills:

• Advanced proficiency in SQL (basic and advanced)
• Expertise in developing and managing dbt models, macros, and tests
• Hands-on experience with Snowflake data warehousing, including Time Travel and Fail Safe features
• Strong understanding of ETL/ELT fundamentals and best practices
• Proficiency in Python for data engineering and automation tasks
• Administration of Snowflake warehouses, databases, schemas, and role-based security
• Management of AWS services such as S3, Lambda, Secret Manager, IAM, and CloudWatch
• Implementation and maintenance of CI/CD pipelines for data deployments
• Configuration of monitoring, alerting, and data quality checks in cloud data platforms

Preferred Skills:

• Experience with modern data platform fundamentals and architecture
• Expertise in optimizing large-scale data pipelines for performance and cost efficiency
• Familiarity with data governance and compliance best practices in cloud environments
• Knowledge of infrastructure-as-code tools for cloud resource management
• Exposure to advanced Snowflake features such as data sharing and secure data exchange

Desired Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field
• Relevant industry certifications in AWS, Snowflake, or dbt are highly desirable