Principal Big Data Architect – Spark, Data Quality & Cloud

Cloud Analytics Technologies, LLC; New York (NY); On-site; USD 180,000 - 260,000; Full time

Job summary

CLOUD ANALYTICS TECHNOLOGIES is seeking a Senior Data Architect with 15+ years in software engineering and 5+ years in architecture roles. You will design data platforms, drive data quality programs, and mentor engineers across cloud environments like AWS, Azure, or GCP.

You will partner with senior leaders to ensure data governance and scalable data solutions that support business objectives and analytics capabilities.

Qualifications

• Bachelors or masters in CS, Eng, or related field.
• 15+ years in software engineering with 5+ years in architect/lead roles.
• Expert Spark, Spark SQL, DataFrames, streaming, performance tuning.
• Experience with data lake tech and cloud platforms (AWS/Azure/GCP).

Responsibilities

• Lead data architecture initiatives and mentor technical teams.
• Define data quality, profiling, validation, cleansing, reconciliation and anomaly detection.
• Collaborate with senior leadership to align data strategy with business goals.

Skills

Apache Spark

Python

Advanced SQL

Team leadership

DevOps

Data quality engineering

Data governance

Data platform architecture

CI/CD integration

Communication with senior leadership

Education

Bachelor's or master's in Computer Science, Engineering, or related field

Tools

Hadoop/HDFS

Delta Lake

Apache Iceberg

AWS (EMR, Glue, Redshift, Lambda)

Azure Databricks / Synapse

Google Cloud Dataproc / BigQuery

Kafka / Kinesis

Job description

CLOUD ANALYTICS TECHNOLOGIES is seeking a Senior Data Architect with 15+ years in software engineering and 5+ years in architecture roles. You will design data platforms, drive data quality programs, and mentor engineers across cloud environments like AWS, Azure, or GCP.

You will partner with senior leaders to ensure data governance and scalable data solutions that support business objectives and analytics capabilities.