Data Engineer

RandomTrees LLC; Chennai District; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

RandomTrees LLC in Chennai is seeking an experienced Data Engineer to design, build, and maintain high-performance ELT pipelines using Snowflake, dbt, and cloud services. You will own the analytics foundation and collaborate with data analysts, analytics engineers, and business stakeholders to ensure scalable, reliable data models.

The role emphasizes cost governance, data governance, and robust testing, with hands-on work across Snowflake, Airflow, and Python.

Qualifications

• 5+ years of professional data engineering experience.
• Deep Snowflake architecture expertise.
• Proven dbt Core/Cloud modeling experience.
• Experience with at least one major cloud provider (AWS/Azure/GCP).
• Strong SQL mastery and Python proficiency.
• Hands-on Git and CI/CD workflow experience.

Responsibilities

• Design, build, and optimize ELT pipelines into Snowflake.
• Develop modular dbt projects with tests and documentation.
• Create production Airflow DAGs with error handling and SLAs.
• Optimize Snowflake warehouses, storage, and cost governance.
• Ingest data from cloud storage via Snowpipe and external stages.
• Enforce data quality, governance, RBAC, and data masking policies.
• Implement version control and automated deployment for data assets.

Skills

Python

SQL

Airflow

Data Modeling

CI/CD

Tools

Snowflake

dbt

GitHub Actions

GitLab CI

Bitbucket Pipelines

Snowpipe

Job description

We are seeking an experienced Data Engineer with 5+ years of hands‑on expertise across the modern data stack to design, build, and maintain high-performance ELT pipelines. In this role, you will be the core technical owner of our analytics foundation—leveraging Snowflake, dbt, hosted on cloud infrastructure (AWS, Azure, or GCP). You will partner closely with data analysts, analytics engineers, and business stakeholders to ensure our data models are scalable, reliable, and governed.

Key Responsibilities

• Data Pipeline Architecture & ELT: Design, develop, and optimize robust, idempotent ELT data pipelines moving data from diverse operational sources into Snowflake.
• Transformation & Modeling (dbt): Architect modular, production-grade dbt projects. Implement dimension and fact tables (Kimball/medallion architecture), write reusable Jinja macros, and establish automated testing (data freshness, uniqueness, referential integrity).
• Workflow Orchestration (Airflow): Build, monitor, and maintain production Airflow DAGs with proper error handling, SLA alerts, retries, and dependency management.
• Snowflak & Optimization: Configure and optimize virtual warehouses, storage, clustering keys, zero-copy cloning, and Time Travel. Enforce cost-governance practices and resource monitors.
• Cloud Infrastructure & Ingestion: Integrate cloud storage services (e.g., AWS S3, Azure Blob/ADLS, or Google Cloud Storage) with Snowflake using Snowpipe, external stages, and event-driven architectures.
• Data Quality & Governance: Establish schema migrations, data testing frameworks, and role-based access control (RBAC), object tagging, and data masking policies.
• CI/CD & DataOps: Implement version control best practices, automated deployment pipelines (GitHub Actions, GitLab CI, or Bitbucket Pipelines), and environment management for data assets.

Qualifications & Requirements

Must-Have:

• Experience: 5+ years of dedicated professional experience in data engineering, data warehousing, or business intelligence backend systems.
• Snowflake: Deep expertise in Snowflake architecture, query profiling, warehouse right-sizing, Snowpipe, and security implementations (RBAC, row-level security).
• dbt (Data Build Tool): Proven track record modeling complex datasets in dbt Core or dbt Cloud, using incremental models, snapshots, tests, documentation, and Jinja/SQL macros.
• Any Major Cloud Platform: Solid working experience with at least one major cloud provider (AWS, Azure, or GCP), particularly with IAM roles, object storage, and serverless compute.
• Programming & SQL: Mastery of advanced SQL (window functions, CTEs, query plan optimization) and intermediate-to-advanced proficiency in Python for data processing and automation.
• Version Control & CI/CD: Hands‑on experience with Git branching workflows and automated testing/deployment pipelines.

Nice-to-Have:

• Experience with streaming technologies (Apache Kafka, AWS Kinesis) or Snowflake Streaming Snowpipe.
• Exposure to Snowflake Dynamic Tables, Streams & Tasks, or Snowpark.
• Relevant certifications: SnowPro Core / Advanced, dbt Certified Developer, or Associate/Professional Cloud Architect/Data Engineer (AWS/Azure/GCP).
• Familiarity with data observability tools (e.g., Monte Carlo, Datafold, Elementary).