Senior Data Engineer - Gurugram

ClanX Network LLP; Gurugram District; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

ClanX Network LLP is seeking a Senior Data Engineer to build and scale reliable data platforms for AI and analytics in industrial asset monitoring. You will design and maintain data pipelines, ensuring data quality and optimal performance while collaborating with engineering and ML teams.

The role requires hands-on experience in data engineering, strong skills in SQL and Python, and an understanding of cloud platforms like AWS or Azure.

Qualifications

• 3–7 years of hands-on experience building production-grade data pipelines.
• Strong expertise in SQL, including query optimization and complex transformations.
• Strong Python programming skills for data engineering workloads.

Responsibilities

• Design, build, and maintain scalable batch ETL/ELT and CDC pipelines.
• Own data ingestion from client systems into the data platform.
• Build reliable, idempotent, incremental, and backfill-safe data pipelines.
• Model and transform raw data into trusted analytical and ML-ready datasets.
• Develop and optimize Spark/PySpark and SQL workloads for large-scale processing.

Skills

Data pipeline development

SQL expertise

Python programming

Spark/PySpark

Data modeling

Workflow orchestration tools

Cloud platforms (AWS/Azure)

Data quality and monitoring

ETL/ELT processes

Analytics and ML datasets

Education

Bachelor's degree in Computer Science, Engineering, Mathematics, or related field

Job description

Mechademy is hiring a Senior Data Engineer to build and scale reliable data platforms, pipelines, and models that power enterprise AI, machine learning, and analytics for industrial asset monitoring and predictive maintenance.

Qualifications

• 3–7 years of hands‑on experience building production‑grade data pipelines
• Strong expertise in SQL, including query optimization and complex transformations
• Strong Python programming skills for data engineering workloads
• Experience with Spark/PySpark or equivalent distributed processing frameworks
• Strong understanding of data modeling, ETL/ELT, CDC, incremental processing, and partitioning
• Experience with workflow orchestration tools such as Dagster, Airflow, or Prefect
• Experience working with AWS or Azure cloud platforms
• Strong focus on data quality, observability, validation, and monitoring
• Experience building scalable datasets for analytics and machine learning use cases
• Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field

Responsibilities

• Design, build, and maintain scalable batch ETL/ELT and CDC pipelines
• Own data ingestion from client systems into the data platform
• Build reliable, idempotent, incremental, and backfill‑safe data pipelines
• Model and transform raw data into trusted analytical and ML‑ready datasets
• Develop and optimize Spark/PySpark and SQL workloads for large‑scale processing
• Implement data quality checks, lineage, monitoring, and alerting mechanisms
• Support machine learning feature pipelines and analytics platforms
• Optimize storage formats, partitioning strategies, and query performance
• Collaborate with ML, product, and engineering teams to deliver reliable data solutions
• Drive best practices for scalability, reliability, and maintainability across the data stack

Location: Gurugram – Hybrid (2–3 days on‑site)