Data Engineer

MindSource; Cupertino (CA); On-site; USD 120,000 - 170,000; Full time

Job summary

MindSource is seeking a Data Engineer with a strong DataOps background to build, maintain, and support large-scale data pipelines powering Sales and Finance analytics. The ideal candidate will have expertise in SQL, Python, Airflow, Spark, and modern data platforms, with hands-on experience supporting production data pipelines and resolving complex issues.

This role emphasizes data quality, monitoring, and collaboration with Data Engineering, Sales, and Finance teams.

Qualifications

• 5+ years of Data Engineering or Software Engineering with a data focus.
• Strong SQL and Python skills.
• Hands-on with Airflow, Spark, Kafka, Trino/Dremio, Iceberg, Docker for ETL.
• Experience supporting production data pipelines, DataOps, incident management, and data quality.
• Strong troubleshooting, documentation, and communication skills.

Responsibilities

• Design, develop, and maintain scalable ETL/data pipelines and warehouse solutions.
• Build and orchestrate workflows using Airflow.
• Develop data solutions using Python, SQL, Spark, Kafka, Trino/Dremio, Iceberg, and Docker.
• Monitor, troubleshoot, and support production pipelines, including incident response, backfills, and deployments.
• Perform root cause analysis (RCA) and resolve issues across shared data platforms.
• Implement data quality checks, monitoring, and validation frameworks.
• Collaborate with Data Engineering, Sales, and Finance teams to deliver reliable data solutions.
• Maintain Git, CI/CD, and SDLC best practices.

Skills

SQL

Python

DataOps

Troubleshooting

Documentation

Communication

Tools

Airflow

Spark

Kafka

Trino/Dremio

Iceberg

Docker

ETL Pipelines

Job description

Local Candidate only who can come for onsite Interview.

Job Summary

We are seeking a Data Engineer with a strong DataOps background to build, maintain, and support large-scale data pipelines powering Sales and Finance analytics. The ideal candidate will have expertise in SQL, Python, Airflow, Spark, and modern data platforms, along with hands‑on experience supporting production data pipelines and resolving complex issues.

Key Responsibilities

• Design, develop, and maintain scalable ETL/data pipelines and warehouse solutions.
• Build and orchestrate workflows using Airflow.
• Develop data solutions using Python, SQL, Spark, Kafka, Trino/Dremio, Iceberg, and Docker.
• Monitor, troubleshoot, and support production pipelines, including incident response, backfills, and deployments.
• Perform root cause analysis (RCA) and resolve issues across shared data platforms.
• Implement data quality checks, monitoring, and validation frameworks.
• Collaborate with Data Engineering, Sales, and Finance teams to deliver reliable data solutions.
• Maintain Git, CI/CD, and SDLC best practices.

Required Qualifications

• 5+ years of Data Engineering or Software Engineering experience with a strong data focus.
• Strong SQL and Python skills.
• Hands‑on experience with Airflow, Spark, Kafka, Trino/Dremio, Iceberg, Docker, and ETL/data pipeline development.
• Experience supporting production data pipelines, DataOps, incident management, and data quality.
• Strong troubleshooting, documentation, and communication skills.

Preferred Qualifications

• Experience with AWS, Azure, or GCP.
• Experience with Sales or Finance analytics.
• Experience using AI coding tools such as Claude, ChatGPT, or GitHub Copilot.