Senior Big Data Architect — Data Quality & Platform

Quantum Technologies. LLC; New York (NY); Hybrid; USD 180,000 - 240,000; Full time

Job summary

Quantum Technologies LLC seeks a senior data/technical architect to design and implement large-scale data quality and governance platforms. You will lead architecture decisions across cloud data services, Spark-based pipelines, and data lake technologies while guiding engineering teams to deliver robust, scalable solutions.

The role requires hands-on expertise in Spark, SQL, Python, and distributed storage technologies, with the ability to communicate complex concepts to senior leadership.

Qualifications

• Bachelor's or Master's in CS/Engineering or related quantitative field.
• 15+ years of progressive software engineering with 5+ years in architect/lead roles focusing on data quality, governance, or data platform architecture.
• Expert-level Big Data: Spark (PySpark/Scala/Java), Spark SQL, DataFrames, streaming, and performance tuning.
• Proficient with Hadoop/HDFS, S3, Delta Lake, Iceberg, and streaming systems like Kafka/Kinesis.
• Cloud experience on AWS/Azure/GCP with data services; SQL and Python for data engineering.
• Strong SDLC/DevOps understanding; ability to embed quality gates in CI/CD; leadership experience.

Skills

Big Data architecture

Spark (PySpark/Scala/Java)

Advanced SQL

Python for data engineering

Data quality engineering

Data governance

Data platform architecture

Cloud data platforms

SDLC & DevOps

Leadership & mentoring

Education

Bachelor's or Master's degree in CS/Engineering or related quantitative field

Tools

Hadoop/HDFS

S3 / Cloud Storage

Delta Lake

Apache Iceberg

Apache Kafka

AWS Kinesis

Databricks

Job description

Quantum Technologies LLC seeks a senior data/technical architect to design and implement large-scale data quality and governance platforms. You will lead architecture decisions across cloud data services, Spark-based pipelines, and data lake technologies while guiding engineering teams to deliver robust, scalable solutions.

The role requires hands-on expertise in Spark, SQL, Python, and distributed storage technologies, with the ability to communicate complex concepts to senior leadership.