Data Engineer

Sago; Gurugram District; À distance; INR 1 000 000 - 2 000 000; Plein temps

Avantages offerts par ce poste

5 Days working

Remote Job

Medical Insurance

Résumé du poste

Sago is seeking a Data Engineer with 2-5 years of experience to design, build, and maintain reliable data pipelines and analytics-ready datasets. Strong SQL and Python or PySpark skills are required, with hands-on experience on cloud data platforms.

You will develop ETL/ELT pipelines, ensure data quality, and collaborate with BI analysts to deliver dependable data solutions. Remote-friendly role with 5 days a week work, medical insurance, and other benefits.

Qualifications

• 2-5 years of professional experience in data engineering, ETL development, or a similar role.
• Strong SQL skills, including joins, CTEs, window functions, data validation, and query optimization.
• Hands-on experience with Python and/or PySpark for data processing and automation.
• Practical experience building and supporting ETL or ELT pipelines in production.
• Experience with large datasets, incremental loads, upserts, schema changes, and partitioned data.

Responsabilités

• Design, develop, and maintain scalable ETL and ELT pipelines for batch and scheduled data processing.
• Build and optimize data transformations using SQL, Python, and PySpark.
• Ingest and integrate data from databases, APIs, files, SharePoint, and cloud storage.
• Create and maintain curated, analytics-ready datasets for Power BI, Superset, and other reporting platforms.
• Implement data-quality checks, reconciliation controls, monitoring, logging, and failure alerts.
• Troubleshoot pipeline failures, schema mismatches, performance bottlenecks, and data discrepancies.
• Automate recurring workflows and schedule production jobs using orchestration tools such as Apache Airflow.
• Maintain technical documentation, data mappings, business rules, and operational runbooks.
• Collaborate with BI analysts, application teams, and business stakeholders to deliver dependable data solutions.

Connaissances

Strong SQL

Python

PySpark

ETL/ELT development

Data engineering

Stakeholder communication

Outils

Git

Apache Airflow

Apache Spark

Delta Lake

Trino

Dremio

Power BI

Superset

SharePoint

Description du poste

We are looking for a Data Engineer with 2-5 years of experience to design, build, and maintain reliable data pipelines and analytics-ready datasets. The ideal candidate will have strong SQL and Python or PySpark skills, hands‑on experience with cloud data platforms, and the ability to troubleshoot data-quality, performance, and production issues independently.

Key Responsibilities

• Design, develop, and maintain scalable ETL and ELT pipelines for batch and scheduled data processing
• Build and optimize data transformations using SQL, Python, and PySpark
• Ingest and integrate data from databases, APIs, files, SharePoint, and cloud storage
• Create and maintain curated, analytics-ready datasets for Power BI, Superset, and other reporting platforms
• Implement data-quality checks, reconciliation controls, monitoring, logging, and failure alerts
• Troubleshoot pipeline failures, schema mismatches, performance bottlenecks, and data discrepancies
• Optimize large-scale data processing for runtime, memory, storage, and cost efficiency
• Automate recurring workflows and schedule production jobs using orchestration tools such as Apache Airflow
• Maintain technical documentation, data mappings, business rules, and operational runbooks
• Collaborate with BI analysts, application teams, and business stakeholders to deliver dependable data solutions

Requirements

Required Skills and Experience

• 2-5 years of professional experience in data engineering, ETL development, or a similar role
• Strong SQL skills, including joins, common table expressions, window functions, data validation, and query optimization
• Hands-on experience with Python and/or PySpark for data processing and automation
• Practical experience building and supporting ETL or ELT pipelines in production
• Experience working with large datasets, incremental loads, upserts, schema changes, and partitioned data
• Good understanding of relational databases, dimensional modeling, data warehousing, and file formats such as Parquet, CSV, and JSON
• Experience with Git or another version-control system and structured deployment practices
• Strong debugging, analytical, documentation, and stakeholder-communication skills

Preferred Skills

• Experience with AWS services such as Amazon S3, Athena, EMR, and EMR Serverless
• Experience with Apache Spark, Delta Lake, Apache Airflow, Dremio, or Trino-compatible SQL engines
• Familiarity with Power BI, Apache Superset, or other downstream reporting tools
• Experience with REST APIs, SharePoint integrations, and automated file-delivery workflows
• Understanding of data security, access controls, privacy, and production monitoring practices

Benefits

• 5 Days working
• Remote Job
• Medical Insurance