Data Engineer (Snowflake + Azure Databricks + dbt/Matillion)

Techdome; Hyderabad; On-site; INR 1,200,000 - 2,800,000; Full time

Job summary

Techdome is seeking an experienced Data Engineer to build scalable data transformation pipelines using Snowflake, Azure Databricks, and dbt. You will ingest, transform, and deliver high-quality data to enable data-driven decisions for our clients.

You will own end-to-end pipelines, collaborate with analysts and architects, and contribute to knowledge bases and internal best practices in a fast-paced consulting environment.

Qualifications

• Bachelor's degree in a relevant field.
• Strong SQL and data modeling knowledge.
• Hands-on experience with Snowflake, dbt, and Databricks in production environments.
• Experience with Git, CI/CD, and deployment workflows.
• Familiarity with data quality and orchestration tools.

Responsibilities

• Design, configure, and optimize ingestion, transformation, and orchestration workflows.
• Build scalable ELT pipelines on Snowflake and Databricks using dbt.
• Develop notebooks, jobs, and Delta Lake tables in Azure Databricks.
• Collaborate with onshore/offshore teams to deliver clean, validated datasets.
• Ensure version control and CI/CD practices for data pipelines.

Skills

SQL

PySpark

Data modeling

Git CI/CD

Communication

Education

Bachelor's degree in Computer Science or related

Tools

Snowflake

dbt

Azure Databricks

Matillion DPC

Airflow

Job description

Role Overview

We are looking for an experienced and results-driven Data Engineer to join our growing Data Engineering practice. The ideal candidate will be proficient in building scalable, high-performance data transformation pipelines using Snowflake, Azure Databricks, and dbt, and will thrive in Techdome's fast-paced, client-facing consulting environment. In this role, you will be instrumental in ingesting, transforming, and delivering high-quality data to enable data-driven decision-making across our clients' organizations — owning your deliverables like a true Tech Doctor.

Key Responsibilities

• Design, configure, and optimize ingestion, transformation, and orchestration workflows using Matillion DPC where applicable.
• Design and implement scalable ELT pipelines using dbt on Snowflake, following industry-accepted best practices.
• Build and maintain data processing workloads on Azure Databricks (PySpark/Spark SQL), including notebooks, jobs, Delta Lake tables, and Lakehouse/medallion architectures.
• Build ingestion pipelines from various sources including relational databases, APIs, cloud storage, and flat files into Snowflake and Databricks (Delta Lake).
• Implement data modelling and transformation logic to support layered architecture (staging, intermediate, and mart layers, or medallion architecture) to enable reliable and reusable data assets.
• Leverage orchestration tools (e.g. Airflow, dbt Cloud, Azure Data Factory, or Databricks Workflows) to schedule and monitor data pipelines.
• Apply dbt best practices: modular SQL development, testing, documentation, and version control.
• Perform performance optimizations in dbt/Snowflake/Databricks through clustering, query profiling, materialization, partitioning, caching, and efficient SQL/Spark design.
• Apply CI/CD and Git-based workflows for version-controlled deployments.
• Take complete ownership of assigned pipelines and deliverables — from design through production support — in line with Techdome's ownership culture.
• Contribute to Techdome's internal knowledge base of dbt macros, Databricks patterns, conventions, and testing frameworks; participate in internal tech talks and knowledge-sharing sessions.
• Collaborate with data analysts, data scientists, and data architects across onshore and offshore teams to understand requirements and deliver clean, validated datasets.
• Write well-documented, maintainable code using Git for version control and CI/CD processes.
• Participate in Agile ceremonies including sprint planning, stand-ups, and retrospectives.
• Support consulting engagements through clear documentation, demos, and delivery of client-ready solutions that reflect Techdome's commitment to excellence.

Required Qualifications

• 1 to 4 years of experience in data engineering roles, with 6+ months of hands‑on experience in Snowflake and dbt.
• Mandatory: Hands‑on experience with Azure Databricks — developing and deploying pipelines using PySpark/Spark SQL, Delta Lake, notebooks, clusters, and Databricks Workflows/Jobs in a production environment.
• Hands‑on experience with Matillion Data Productivity Cloud (Matillion DPC) for data ingestion, transformation, or orchestration.
• Experience building and deploying dbt models in a production environment.
• Expert‑level SQL and strong understanding of ELT principles; strong understanding of ELT patterns and data modelling (Kimball/Dimensional preferred).
• Familiarity with data quality and validation techniques: dbt tests, dbt docs, etc.
• Experience with Git, CI/CD, and deployment workflows in a team setting.
• Familiarity with orchestrating workflows using tools like dbt Cloud, Airflow, Azure Data Factory, or Databricks Workflows.

Core Competencies

• Building robust and modular data pipelines using dbt and Databricks.
• Writing efficient SQL and PySpark for data transformation and performance tuning in Snowflake and Databricks.
• Managing environments, sources, and deployment pipelines in dbt.

Cloud Data Platform Expertise

• Strong proficiency with Snowflake: warehouse sizing, query profiling, data loading, and performance optimization.
• Strong proficiency with Azure Databricks: cluster configuration and sizing, Delta Lake optimization (OPTIMIZE, Z-ORDER, vacuum), Unity Catalog basics, and Spark job tuning.
• Experience working with cloud storage (Azure Data Lake, AWS S3, or GCS) for ingestion and external stages.

Technical Toolset

• Python / PySpark: For data transformation, notebook development, and automation on Databricks.
• SQL: Strong grasp of SQL for querying and performance tuning.

Best Practices and Standards

• Knowledge of modern data architecture concepts including layered architecture (staging → intermediate → marts) and Lakehouse/Medallion architecture.
• Familiarity with data quality, unit testing (dbt tests), and documentation (dbt docs).

Security & Governance

• Understanding of access control within Snowflake (RBAC) and Databricks (Unity Catalog/workspace permissions), role hierarchies, and secure data handling.
• Familiarity with data privacy policies (GDPR basics) and encryption at rest/in transit.

Deployment & Monitoring

• Version control using Git; experience with CI/CD practices in a data context.
• Monitoring and logging of pipeline executions, alerting on failures.

Soft Skills — The Techdome Way

• Ownership mindset: take full responsibility for your work, from problem diagnosis to production delivery.
• Client‑first communication: ability to present solutions confidently and handle client demos and discussions.
• Collaboration: work closely with onshore and offshore teams of analysts, data scientists, and architects — every voice is heard and respected.
• Ability to document pipelines and transformations clearly.
• Basic SSIS and Matillion.
• Comfort with ambiguity, competing priorities, and fast‑changing client environments — we move fast and adapt faster.
• Passion for continuous learning and knowledge sharing (tech talks, meetups, internal upskilling).

Nice to Have

• Experience in client‑facing roles or consulting engagements.
• Exposure to AI/ML data pipelines and feature stores.
• Exposure to MLflow for basic ML model tracking (native to Databricks).
• Experience/exposure using data quality tooling.
• Certifications such as Snowflake SnowPro, Databricks Certified Data Engineer Associate, or dbt Certified Developer are a plus.

Why Join Techdome?

• Work on high‑impact projects for global clients — from startups to Fortune 500 enterprises — across diverse industries.
• Be part of an inclusive culture that promotes open communication, teamwork, and respect for every voice.
• Grow with a company that invests in your professional development through mentorship, tech events, and hands‑on exposure to cutting‑edge data and AI technologies.
• Join a journey driven by insight and innovation — where we don't just solve problems, we push the boundaries of technology.