Data Engineer

Drishti Consultants; Bengaluru; Hybrid; INR 1,200,000 - 2,400,000; Full time

Job summary

Drishti Consultants in Bengaluru seeks a data engineer to design and operate batch and streaming data pipelines at production scale, build training-ready datasets for ML feature pipelines, and own the data warehouse/lakehouse stack including Snowflake, BigQuery, Databricks, and transformations with dbt.

The role emphasizes orchestrating workflows with Airflow or equivalent, ensuring strong monitoring and lineage; ideal candidates have experience in SaaS environments and in preparing data for

Qualifications

• Experience building pipelines feeding data into AI/ML systems.
• Experience working in SaaS environments.
• Hands-on experience in preparing data for models.

Responsibilities

• Design and operate batch and streaming data pipelines (ETL/ELT) at production scale.
• Build and maintain training-ready datasets for ML/AI feature pipelines, labeling workflows, and data quality for model training.
• Own the data warehouse/lakehouse stack (e.g., Snowflake, BigQuery, Databricks, Spark) and transformations (dbt).
• Orchestrate workflows (Airflow or equivalent) with strong monitoring and lineage.

Job description

Role & responsibilities

• Design and operate batch and streaming data pipelines (ETL/ELT) at production scale.
• Build and maintain training-ready datasets for ML/AI feature pipelines, labeling workflows, and data quality for model training.
• Own the data warehouse/lakehouse stack (e.g., Snowflake, BigQuery, Databricks, Spark) and transformations (dbt).
• Orchestrate workflows (Airflow or equivalent) with strong monitoring and lineage.

Preferred candidate profile

• Experiencing of building pipelines that feed data into AI/ML systems
• Exposure of working within SaaS environment.
• Hands on experience in preparing data for model.