Data Engineer

Bacancy Technology Inc; Ahmedabad District; Sur place; INR 1 500 000 - 3 000 000; Plein temps

Résumé du poste

Bacancy Technology Inc is seeking a Data Engineer with 3+ years of hands-on experience to design, build, and maintain scalable data pipelines and data platforms. You will work closely with data scientists, analysts, and business stakeholders to deliver reliable, high-quality data solutions.

Responsibilities include end-to-end ETL/ELT development, ingestion frameworks from APIs and databases, and working with data warehouses and lakehouse architectures.

Qualifications

• 3+ years of hands-on data engineering experience.
• Strong Python and SQL proficiency.
• Experience with ETL/ELT pipelines and data warehouses.

Responsabilités

• Design, develop, and maintain end-to-end ETL/ELT data pipelines.
• Build and optimize data ingestion frameworks from APIs, databases, files, and streaming sources.
• Work with data warehouses and lakehouse architectures (Snowflake, Redshift, BigQuery, Databricks).
• Develop and maintain batch and near-real-time pipelines.
• Ensure data quality, validation, monitoring, and observability.

Connaissances

Python

SQL

ETL/ELT

Airflow

Data Modeling

Snowflake

BigQuery

Git

Agile

Spark/PySpark

Outils

Airflow

Snowflake

Spark

Kubernetes

Description du poste

We are looking for a skilled Data Engineer with 3+ years of experience to design, build, and maintain scalable data pipelines and data platforms. The ideal candidate will work closely with data scientists, analysts, and business stakeholders to ensure high-quality, reliable, and performant data solutions.

Key Responsibilities

• Design, develop, and maintain end-to-end ETL/ELT data pipelines
• Build and optimize data ingestion frameworks from multiple sources (APIs, databases, files, streaming
• Work with data warehouses and lakehouse architectures (Snowflake, Redshift, BigQuery, Databricks, etc.)
• Develop and maintain batch and near-real-time pipelines
• Ensure data quality, validation, monitoring, and observability
• Optimize SQL queries and data models for performance and scalability
• Collaborate with analytics, BI, and ML teams to support reporting and advanced analytics
• Implement data governance, security, and access controls
• Automate workflows using orchestration tools (Airflow, Luigi, etc.)
• Troubleshoot and resolve data pipeline and production issues

Required Skills & Qualifications

Must-Have

• 3+ years of hands-on experience as a Data Engineer
• Strong proficiency in Python and SQL
• Experience with ETL/ELT pipeline development
• Hands-on experience with data warehouses (Snowflake preferred)
• Experience with Apache Airflow or similar orchestration tools
• Strong understanding of data modeling (star/snowflake schemas)
• Experience working with large-scale datasets
• Familiarity with Git-based version control
• Experience working in agile environments

Good to Have

• Experience with Apache Spark / PySpark
• Exposure to Kafka or streaming platforms
• Knowledge of cloud platforms (AWS / Azure / GCP)
• Experience with Docker & Kubernetes
• Familiarity with data quality frameworks (Great Expectations, Soda, etc.)
• Infrastructure-as-Code tools (Terraform, CloudFormation)