Data Engineer

Sago; India; Remote; INR 600,000 - 1,200,000; Full time

Benefits offered by this job

5 Days working

Remote Job

Medical Insurance

Job summary

Sago is seeking a Data Engineer to design, build, and maintain reliable data pipelines and analytics-ready datasets. The role emphasizes SQL, Python/PySpark, and cloud data platforms, with a focus on data quality and production readiness.

You will implement scalable ETL/ELT processes, ingest data from diverse sources, and collaborate with BI teams to deliver dependable data solutions. This is a remote-friendly position based in India with modern tooling and governance.

Qualifications

• 2–5 years of professional experience in data engineering, ETL development, or a similar role.
• Strong SQL skills with joins, CTEs, window functions, data validation, and query optimization.
• Hands-on experience with Python and/or PySpark for data processing.
• Production experience building/sustaining ETL/ELT pipelines.
• Experience with large datasets, incremental loads, upserts, schema changes, and partitioned data.
• Knowledge of relational databases, dimensional modeling, data warehousing, and file formats (Parquet, CSV, JSON).
• Experience with Git and structured deployment practices.
• Strong debugging, analytical, documentation, and stakeholder communication skills.

Responsibilities

• Design, develop, and maintain scalable ETL and ELT pipelines for batch and scheduled data processing.
• Build and optimize data transformations using SQL, Python, and PySpark.
• Ingest and integrate data from databases, APIs, files, SharePoint, and cloud storage.
• Create and maintain curated, analytics-ready datasets for Power BI, Superset, and other reporting platforms.
• Implement data-quality checks, reconciliation controls, monitoring, logging, and failure alerts.
• Troubleshoot pipeline failures, schema mismatches, performance bottlenecks, and data discrepancies.
• Optimize large-scale data processing for runtime, memory, storage, and cost efficiency.
• Automate recurring workflows and schedule production jobs using orchestration tools such as Apache Airflow.
• Maintain technical documentation, data mappings, business rules, and operational runbooks.
• Collaborate with BI analysts, application teams, and business stakeholders to deliver dependable data solutions.

Skills

SQL

Python

PySpark

ETL development

Data warehousing

Version control (Git)

Debugging

Tools

Apache Airflow

AWS S3

Spark/Delta Lake

Dremio/Trino

Power BI / Superset

Job description

We are looking for a Data Engineer with 2-5 years of experience to design, build, and maintain reliable data pipelines and analytics-ready datasets. The ideal candidate will have strong SQL and Python or PySpark skills, hands-on experience with cloud data platforms, and the ability to troubleshoot data-quality, performance, and production issues independently.

Key Responsibilities

• - Design, develop, and maintain scalable ETL and ELT pipelines for batch and scheduled data processing.
• - Build and optimize data transformations using SQL, Python, and PySpark.
• - Ingest and integrate data from databases, APIs, files, SharePoint, and cloud storage.
• - Create and maintain curated, analytics-ready datasets for Power BI, Superset, and other reporting platforms.
• - Implement data-quality checks, reconciliation controls, monitoring, logging, and failure alerts.
• - Troubleshoot pipeline failures, schema mismatches, performance bottlenecks, and data discrepancies.
• - Optimize large-scale data processing for runtime, memory, storage, and cost efficiency.
• - Automate recurring workflows and schedule production jobs using orchestration tools such as Apache Airflow.
• - Maintain technical documentation, data mappings, business rules, and operational runbooks.
• - Collaborate with BI analysts, application teams, and business stakeholders to deliver dependable data solutions.

Required Skills and Experience

• - 2-5 years of professional experience in data engineering, ETL development, or a similar role.
• - Strong SQL skills, including joins, common table expressions, window functions, data validation, and query optimization.
• - Hands-on experience with Python and/or PySpark for data processing and automation.
• - Practical experience building and supporting ETL or ELT pipelines in production.
• - Experience working with large datasets, incremental loads, upserts, schema changes, and partitioned data.
• - Good understanding of relational databases, dimensional modeling, data warehousing, and file formats such as Parquet, CSV, and JSON.
• - Experience with Git or another version-control system and structured deployment practices.
• - Strong debugging, analytical, documentation, and stakeholder-communication skills.

Preferred Skills

• - Experience with AWS services such as Amazon S3, Athena, EMR, and EMR Serverless.
• - Experience with Apache Spark, Delta Lake, Apache Airflow, Dremio, or Trino-compatible SQL engines.
• - Familiarity with Power BI, Apache Superset, or other downstream reporting tools.
• - Experience with REST APIs, SharePoint integrations, and automated file-delivery workflows.
• - Understanding of data security, access controls, privacy, and production monitoring practices.

* 5 Days working * Remote Job * Medical Insurance