Snowflake Data Engineer + Airflow + DBT

Dreampath Services; Hyderabad; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Dreampath Services is seeking a Senior Data Engineer to design and scale data platforms using AWS, Databricks, DBT, PySpark, Airflow and SQL. You’ll transform operational data into analytics-ready datasets and partner with analytics teams to empower data-driven decisions.

The role emphasizes hands-on ELT/ETL development, data lakehouse architectures, data quality, and cost-efficient optimization in a fast-paced environment. Strong collaboration and communication are essential.

Qualifications

• 7+ years of experience in Data Engineering.
• Proven ability to design and deploy large-scale data pipelines in production.
• Hands-on experience with distributed computing concepts (consistency, fault tolerance, latency).
• Expertise building scalable ELT/ETL with DBT, Databricks, Spark, and Airflow.
• Experience in AWS-based data ecosystems and lakehouse architectures.

Responsibilities

• Design, develop, and maintain scalable DBT models delivering trusted datasets and metrics.
• Build and manage data pipelines using Databricks and PySpark for analytics-ready data.
• Develop and optimize data lakes and lakehouse architectures with Delta Lake / Apache Iceberg.
• Orchestrate end-to-end workflows with Apache Airflow and Prefect for reliability.
• Collaborate with Data Science to productionize feature engineering pipelines.
• Implement data quality frameworks, automated testing, monitoring, and documentation.
• Optimize SQL queries and Spark jobs for performance and cost efficiency.
• Contribute to cloud-native data solutions within the AWS ecosystem.
• Participate in code reviews and promote engineering best practices.
• Build business understanding to ensure accurate data products representing operations.
• Leverage AI-powered tools to boost development productivity.

Skills

AWS

DBT

Databricks

PySpark

Advanced SQL

Airflow

Tools

Terraform

AWS CDK

Pulumi

Job description

We are seeking an experienced Senior Data Engineer with strong expertise in AWS, DBT, Databricks, PySpark, Airflow, and SQL to build and scale modern data platforms. The ideal candidate will have a proven track record of designing, developing, and optimizing large-scale data pipelines and data models that support analytics, machine learning, experimentation, and business reporting.

This role requires exceptional communication skills, a strong ownership mindset, and the ability to collaborate closely with Data Scientists, Analytics teams, and business stakeholders to deliver reliable, high-quality data solutions.

Key Responsibilities

• Design, develop, and maintain scalable DBT models that deliver trusted datasets, business metrics, and analytical features.
• Build and manage data pipelines using Databricks and PySpark to transform raw operational data into analytics-ready datasets.
• Develop and optimize data lakes and lakehouse architectures using Delta Lake and/or Apache Iceberg.
• Orchestrate end‑to‑end workflows using Apache Airflow and Prefect, ensuring high reliability and SLA adherence.
• Partner with Data Science teams to design, develop, and productionize feature engineering pipelines.
• Implement data quality frameworks, automated testing, monitoring, and documentation standards.
• Optimize SQL queries, Spark jobs, and DBT transformations for performance, scalability, and cost efficiency.
• Design and manage cloud‑native data solutions within the AWS ecosystem.
• Participate in code reviews and contribute to engineering best practices.
• Build deep business understanding to ensure data products accurately represent operational processes.
• Leverage AI‑powered development tools such as Cursor, GitHub Copilot, Claude Code, or similar to improve engineering productivity.

Required Skills

Must‑Have Technologies

• DBT (Data Build Tool)
• AWS (Strong Hands‑on Experience)
• Databricks
• PySpark
• Advanced SQL

Additional Technical Skills

• Python
• Data Modeling & Data Warehousing
• Distributed Data Processing Systems
• Infrastructure as Code (Terraform, AWS CDK, or Pulumi)
• CI/CD for Data Engineering
• Data Quality & Testing Frameworks

Required Experience

• 7+ years of experience in Data Engineering.
• Strong experience designing and deploying large‑scale data pipelines in production environments.
• Hands‑on experience with distributed computing systems and concepts such as consistency, fault tolerance, throughput, and latency.
• Proven expertise in building scalable ELT/ETL frameworks using DBT, Databricks, Spark, and Airflow.
• Experience working within AWS‑based data ecosystems.
• Exposure to modern lakehouse architectures and big data technologies.
• Experience supporting analytics, business intelligence, and machine learning workloads.

Preferred Qualifications

• Experience with Kinesis, EMR, Sigma, or Pulumi.
• Exposure to supply chain, logistics, transportation, or operations‑focused data domains.
• Familiarity with feature stores, MLOps, and production ML pipelines.
• Experience working in fast‑paced, high‑growth environments.

Soft Skills

• Outstanding verbal and written communication skills.
• Strong stakeholder management and collaboration abilities.
• Self‑driven, proactive, and capable of working independently.
• Strong problem‑solving and analytical mindset.
• Ability to quickly learn and adapt to new technologies.
• Comfortable mentoring junior engineers and driving engineering excellence.

Keywords

AWS, DBT, Databricks, PySpark, Airflow, SQL, Delta Lake, Apache Iceberg, Data Engineering, Data Modeling, ETL, ELT, Spark, Terraform, Data Warehouse, Big Data, Python, Analytics Engineering, Lakehouse Architecture