Lead Data Engineer - R01571409

Brillio 2; Bengaluru; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Brillio 2 in Bengaluru is seeking a Lead Data Engineer with 8–10 years of experience to own and evolve our data platform. You will design and optimize dbt models, manage Snowflake warehouses, and integrate AWS services to support scalable analytics.

You will lead governance, implement CI/CD for data deployments, monitor data quality, troubleshoot production issues, and collaborate with analytics and BI teams to deliver high-impact data solutions.

Qualifications

• 8–10+ years of data engineering experience.
• Hands-on with Snowflake, dbt, and AWS.
• Strong SQL and Python for data workflows.

Responsibilities

• Develop, maintain, and optimize dbt models, macros, and tests.
• Administer Snowflake data warehouses including security configurations.
• Optimize SQL queries and warehouse performance.
• Integrate AWS services for cloud data infrastructure.
• Implement CI/CD pipelines for data deployments.
• Configure monitoring, data quality checks, and alerting.
• Troubleshoot production issues and perform root cause analysis.
• Collaborate with analytics, BI, and engineering teams.

Skills

Advanced SQL

dbt development

Python for data engineering

Snowflake optimization

ETL/ELT fundamentals

CI/CD pipelines

AWS services

Data governance

Incident management

Documentation

Education

Bachelor’s or Master’s in CS/IT/Engineering

Tools

Snowflake

dbt

AWS (S3, Lambda, Secrets Manager, IAM, CloudWatch)

CI/CD tools

Job description

Lead Data Engineer

Experience Range: 8- 10 years of experience in data engineering, with hands-on expertise in Snowflake, dbt, and AWS cloud services

Key Responsibilities

• Develop, maintain, and optimize dbt models, macros, and tests to support scalable ETL/ELT data pipelines
• Administer and manage Snowflake data warehouses, including databases, schemas, roles, and security configurations to ensure robust data governance
• Optimize complex SQL queries and warehouse performance, reducing processing times and improving storage utilization
• Manage and integrate AWS services such as S3, Lambda, Secret Manager, IAM, and CloudWatch to build and maintain reliable cloud data infrastructure
• Implement and maintain CI/CD pipelines for automated deployment and release management of data solutions
• Configure and monitor data quality checks, alerting systems, and performance monitoring to ensure high data reliability and integrity
• Troubleshoot and resolve production issues, conducting thorough root cause analysis to minimize downtime and prevent recurrence
• Collaborate closely with analytics, business intelligence, and engineering teams to deliver high-impact, scalable data solutions aligned with business objectives
• Medallion modeling: Design, build, and maintain dbt models across Bronze → Silver → Gold layers for the assigned domain.
• Governance alignment: Partner with the Data Governance team so models meet certification and quality-gate standards before promotion.
• Downstream support: Support data modeling for downstream analytics platform consumers (dashboards and data products)
• Troubleshooting & support: Diagnose and resolve pipeline issues; participate in on-call/support rotation as needed.
• Documentation: Document data lineage, model logic, and key technical decisions so the work is maintainable by the internal team.
• Maintain and Develop APIs

Required Skills

• Advanced proficiency in SQL (basic and advanced)
• Expertise in developing and managing dbt models, macros, and tests
• Hands-on experience with Snowflake data warehousing, including Time Travel and Fail Safe features
• Strong understanding of ETL/ELT fundamentals and best practices
• Proficiency in Python for data engineering and automation tasks
• Administration of Snowflake warehouses, databases, schemas, and role-based security
• Management of AWS services such as S3, Lambda, Secret Manager, IAM, and CloudWatch
• Implementation and maintenance of CI/CD pipelines for data deployments
• Configuration of monitoring, alerting, and data quality checks in cloud data platforms

Preferred Skills

• Experience with modern data platform fundamentals and architecture
• Expertise in optimizing large-scale data pipelines for performance and cost efficiency
• Familiarity with data governance and compliance best practices in cloud environments
• Knowledge of infrastructure-as-code tools for cloud resource management
• Exposure to advanced Snowflake features such as data sharing and secure data exchange

Desired Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field
• Relevant industry certifications in AWS, Snowflake, or dbt are highly desirable