Lead Data Platform Engineer

Quantumtech Inc; United States; On-site; USD 180,000 - 250,000; Full time

Job summary

Quantumtech Inc. is seeking a highly experienced Senior Data Engineer to design, develop, and implement scalable data pipelines and architectures.

You will lead enterprise data initiatives, optimize workflows, and collaborate with cross-functional teams to deliver reliable data solutions. The role requires 15+ years in data engineering, strong Python/SQL/PySpark skills, and deep expertise in Spark, Hadoop, and cloud platforms (AWS/Azure/GCP).

Qualifications

• 15+ years in Data Engineering/Big Data/Software Engineering
• Strong Python, SQL, PySpark
• Expertise in Spark, Hadoop, distributed computing
• Experience with ETL/ELT tools and data integration
• Cloud platforms: AWS, Azure, GCP
• Knowledge of data modeling, warehousing, and data governance
• Experience with Airflow, AWS Glue, Azure Data Factory or similar orchestration tools
• Knowledge of CI/CD, Git, Docker, Kubernetes

Responsibilities

• Design, develop, and maintain scalable data pipelines and architectures.
• Lead development of complex ETL/ELT workflows.
• Build and optimize large-scale data processing systems.
• Develop data solutions using Python, SQL, PySpark, Spark.
• Design and implement data lakes, warehouses, and lakehouse architectures.
• Work with AWS, Azure, and GCP.
• Implement real-time and batch data processing.
• Optimize performance, reliability, security.
• Lead technical discussions and mentor engineers.
• Collaborate with Data Scientists, Analysts, Architects, and stakeholders.
• Establish data engineering best practices and coding standards.
• Monitor pipeline performance and troubleshoot production issues

Skills

Python

SQL

PySpark

Apache Spark

Hadoop

Kafka

Airflow

CI/CD

Docker

Kubernetes

Cloud AWS/Azure/GCP

Education

Bachelor's or Master's in Computer Science/IT

Tools

Snowflake

Databricks

Redshift/BigQuery

Job description

Quantumtech Inc. is seeking a highly experienced Senior Data Engineer to design, develop, and implement scalable data pipelines and architectures.

You will lead enterprise data initiatives, optimize workflows, and collaborate with cross-functional teams to deliver reliable data solutions. The role requires 15+ years in data engineering, strong Python/SQL/PySpark skills, and deep expertise in Spark, Hadoop, and cloud platforms (AWS/Azure/GCP).