Data Engineer

卡杯; Gurugram District; Remote; INR 900,000 - 1,500,000; Full time

Benefits offered by this job

5 Days working

Remote job

Medical Insurance

Job summary

卡杯 is seeking a Data Engineer in India (Gurugram) with 2-5 years of experience to design, build, and maintain reliable data pipelines and analytics-ready datasets. You will leverage SQL and Python/PySpark to run ETL/ELT pipelines, address data quality, and support reporting platforms.

Responsibilities include ingesting data from multiple sources, building transformations, and implementing monitoring, logging, and alerts.

Qualifications

• 2-5 years of professional data engineering experience.
• Strong SQL skills and data validation.
• Hands-on Python and/or PySpark for data processing.
• Production experience with ETL/ELT pipelines.
• Experience with large datasets, incremental loads, and partitioned data.
• Familiarity with relational databases, data warehousing, and common file formats.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines for batch processing.
• Build data transformations using SQL, Python, and PySpark.
• Ingest data from databases, APIs, files, and cloud storage.
• Create analytics-ready datasets for Power BI, Superset, and reporting tools.
• Implement data-quality checks, monitoring, and alerting.
• Troubleshoot pipeline failures, performance bottlenecks, and data discrepancies.
• Optimize data processing for runtime, memory, and cost.
• Automate workflows with orchestration tools like Apache Airflow.
• Maintain data mappings and runbooks; collaborate with BI analysts and stakeholders.

Skills

SQL

Python

PySpark

ETL/ELT pipelines

Data quality

Data warehousing

Dashboard data conditioning

Debugging

Job description

We are looking for a Data Engineer with 2–5 years of experience to design, build, and maintain reliable data pipelines and analytics-ready datasets. The ideal candidate will have strong SQL and Python or PySpark skills, hands‑on experience with cloud data platforms, and the ability to troubleshoot data‑quality, performance, and production issues independently.
Key Responsibilities

· Design, develop, and maintain scalable ETL and ELT pipelines for batch and scheduled data processing.

· Build and optimize data transformations using SQL, Python, and PySpark.

· Ingest and integrate data from databases, APIs, files, SharePoint, and cloud storage.

· Create and maintain curated, analytics‑ready datasets for Power BI, Superset, and other reporting platforms.

· Implement data-quality checks, reconciliation controls, monitoring, logging, and failure alerts.

· Troubleshoot pipeline failures, schema mismatches, performance bottlenecks, and data discrepancies.

· Optimize large‑scale data processing for runtime, memory, storage, and cost efficiency.

· Automate recurring workflows and schedule production jobs using orchestration tools such as Apache Airflow.

· Maintain technical documentation, data mappings, business rules, and operational runbooks.

· Collaborate with BI analysts, application teams, and business stakeholders to deliver dependable data solutions.

Required Skills and Experience

· 2–5 years of professional experience in data engineering, ETL development, or a similar role.

· Strong SQL skills, including joins, common table expressions, window functions, data validation, and query optimization.

· Hands‑on experience with Python and/or PySpark for data processing and automation.

· Practical experience building and supporting ETL or ELT pipelines in production.

· Experience working with large datasets, incremental loads, upserts, schema changes, and partitioned data.

· Good understanding of relational databases, dimensional modeling, data warehousing, and file formats such as Parquet, CSV, and JSON.

· Experience with Git or another version‑control system and structured deployment practices.

· Strong debugging, analytical, documentation, and stakeholder‑communication skills.
Preferred Skills

* 5 Days working
* Remote Job
* Medical Insurance