Lead Data Engineer (Snowflake / Databricks)

Inferenz Tech; Ahmedabad District; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Inferenz Tech seeks a Lead Data Engineer to design, develop, and optimize scalable data solutions on Snowflake and/or Databricks. You will architect data pipelines, implement governance and data quality, and mentor a growing team across AWS, Azure, or GCP environments.

The role requires hands-on experience with Snowflake or Databricks, strong SQL and Python skills, and a passion for scalable lakehouse architectures. Collaboration with analysts and stakeholders is essential.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or related field.
• 7–10 years of experience in data engineering with hands-on Snowflake or Databricks.
• Proven experience in leading and mentoring data engineering teams.
• Strong knowledge of data modelling, ETL/ELT, data lakehouse architectures, batch and incremental processing, data quality and validation.

Responsibilities

• Lead design, development, and optimization of scalable data solutions using Snowflake and/or Databricks.
• Build and maintain robust data pipelines with orchestration and monitoring to ensure reliability and performance.
• Develop scalable data models and transformations for data warehousing and lakehouse architectures.
• Tune performance, manage workloads, and optimize storage on Snowflake/Databricks.
• Implement data validation, testing, governance, security, and compliance practices.
• Identify and resolve data pipeline performance bottlenecks across environments.
• Collaborate with analysts, scientists, and stakeholders to translate requirements into scalable solutions.
• Provide technical guidance and mentorship to data engineers; conduct code/design reviews.

Skills

SQL

Python

Data modeling

ETL/ELT

Spark

Airflow

dbt

Git & CI/CD

Education

Bachelor's or Master's in CS/IT/Data Engineering

Tools

Snowflake

Databricks

Apache Airflow

dbt

AWS

Azure

GCP

Job description

Job Summary

We are seeking an accomplished Lead Data Engineer with strong expertise in modern cloud data platforms, with hands‑on experience in Snowflake or Databricks. The ideal candidate will be responsible for designing, developing, and optimizing scalable data solutions, building robust data pipelines, and ensuring data quality, reliability, and performance. The candidate should have strong experience in data engineering, data modelling, ETL/ELT processes, and cloud data platforms, with the ability to work across Snowflake and Databricks. Hands‑on experience with at least one of these platforms is mandatory, while working knowledge of the other is good to have. This role also involves providing technical leadership, mentoring engineers, collaborating with cross‑functional teams, and translating business requirements into scalable technical solutions.

Key Responsibilities

• Lead the design, development, and optimization of scalable, secure, and high-performance data solutions using Snowflake and/or Databricks.
• Design, build, and maintain robust data pipelines using appropriate orchestration and data engineering frameworks, ensuring reliability, scalability, monitoring, retries, and error handling.
• Develop scalable data models and transformations following best practices for data warehousing and modern lakehouse architectures.
• Develop and optimize data solutions on Snowflake or Databricks, including performance tuning, data processing, storage optimization, and workload management.
• Implement data validation, automated testing, monitoring, governance, and security practices to ensure data integrity, reliability, and compliance.
• Identify and resolve performance bottlenecks across data pipelines, queries, jobs, and data processing workloads.
• Build and manage data solutions on cloud platforms such as AWS, Azure, or GCP.
• Partner with data analysts, data scientists, architects, and business stakeholders to translate requirements into scalable technical solutions.
• Provide technical guidance and mentorship to data engineers, conduct code/design reviews, and establish engineering best practices.
• Stay current with advancements in cloud data platforms, data engineering, analytics, and AI/ML technologies and recommend relevant improvements.
• Work with version control, CI/CD, and infrastructure-as-code practices for reliable deployment and management of data platforms.

Qualifications

• Bachelor s or Master s degree in Computer Science, Information Technology, Data Engineering, or a related field.
• 7-10 years of experience in data engineering, with strong hands‑on experience in at least one of Snowflake or Databricks.
• Demonstrated experience in leading and mentoring data engineering teams.
• Strong working knowledge of modern data engineering concepts, including:
  • Data modelling and data warehousing
  • ETL/ELT processes
  • Data Lake / Lakehouse architectures
  • Batch and incremental data processing
  • Data quality and validation
• Snowflake: Strong hands‑on experience in data modelling, performance tuning, access control, and features such as streams, tasks, external tables, or equivalent capabilities.
• Databricks: Strong hands‑on experience with data engineering, Spark-based processing, Delta Lake/Lakehouse concepts, and pipeline development.
• Snowflake or Databricks - one must be hands‑on; knowledge of the other is good to have.
• Proficiency in SQL and Python.
• Spark experience is expected for candidates with Databricks exposure.
• Experience building and managing data pipelines on AWS, Azure, or GCP.
• Experience with data orchestration tools such as Apache Airflow, Azure Data Factory, AWS Glue, or equivalent.
• Familiarity with transformation frameworks such as dbt is good to have.
• Familiarity with version control systems such as Git and CI/CD practices.
• Exposure to infrastructure-as-code tools such as Terraform is good to have.
• Strong analytical, problem-solving, collaboration, and communication skills.
• Demonstrated ability to lead technical projects and mentor junior/mid-level engineers.

Preferred Skills

• Hands‑on experience with both Snowflake and Databricks.
• Experience with dbt and Apache Airflow.
• Experience with streaming technologies such as Kafka, Kinesis, or Pub/Sub.
• Experience with Generative AI, ML, or advanced analytics applications.
• Familiarity with BI/analytics tools such as Power BI, Tableau, Looker, or similar.
• Knowledge of data governance, security, and compliance frameworks.
• Experience with multiple cloud platforms such as AWS, Azure, or GCP.