Big Data Technical Architect

Cloud Analytics Technologies, LLC; New York (NY); On-site; USD 180,000 - 260,000; Full time

Job summary

CLOUD ANALYTICS TECHNOLOGIES is seeking a Senior Data Architect with 15+ years in software engineering and 5+ years in architecture roles. You will design data platforms, drive data quality programs, and mentor engineers across cloud environments like AWS, Azure, or GCP.

You will partner with senior leaders to ensure data governance and scalable data solutions that support business objectives and analytics capabilities.

Qualifications

• Bachelors or masters in CS, Eng, or related field.
• 15+ years in software engineering with 5+ years in architect/lead roles.
• Expert Spark, Spark SQL, DataFrames, streaming, performance tuning.
• Experience with data lake tech and cloud platforms (AWS/Azure/GCP).

Responsibilities

• Lead data architecture initiatives and mentor technical teams.
• Define data quality, profiling, validation, cleansing, reconciliation and anomaly detection.
• Collaborate with senior leadership to align data strategy with business goals.

Skills

Apache Spark

Python

Advanced SQL

Team leadership

DevOps

Data quality engineering

Data governance

Data platform architecture

CI/CD integration

Communication with senior leadership

Education

Bachelor's or master's in Computer Science, Engineering, or related field

Tools

Hadoop/HDFS

Delta Lake

Apache Iceberg

AWS (EMR, Glue, Redshift, Lambda)

Azure Databricks / Synapse

Google Cloud Dataproc / BigQuery

Kafka / Kinesis

Job description

• Bachelor's or master's degree in computer science, Engineering, or a related quantitative field.
• 15+ years of progressive experience in software engineering, with at least 5+ years in a Technical Architect, Lead Data Architect, or Principal Data Engineer role, specifically focused on data quality, data governance, or data platform architecture.
• Exceptional hands‑on proficiency and deep architectural understanding of the Big Data ecosystem:
• Apache Spark (PySpark, Scala, or Java): Expert-level experience with Spark SQL, DataFrames/Datasets, streaming, and advanced performance tuning techniques.
• Distributed Storage & Processing: Hadoop, HDFS, S3, Delta Lake, Apache Iceberg, or similar data lake technologies.
• Streaming Technologies: Apache Kafka, AWS Kinesis, or similar high‑throughput messaging systems.
• Cloud Data Platforms: Extensive experience designing and implementing solutions on AWS (e.g., EMR, Glue, Redshift, Lambda, Step Functions, S3), Azure (e.g., Databricks, Synapse Analytics, Data Lake Storage), or Google Cloud Platform (e.g., Dataproc, BigQuery, Cloud Storage).
• Expert-level hands‑on experience with Advanced SQL for complex data analysis, validation, and optimization.
• Expert-level hands‑on experience with Python for data engineering, automation, and developing robust data quality solutions.
• Proven track record of defining, designing, and implementing large-scale data automation frameworks.
• Demonstrated expertise in data quality engineering principles, methodologies, and tools (profiling, validation, cleansing, reconciliation, anomaly detection).
• Experience in leading and mentoring technical teams, fostering a culture of technical excellence and continuous improvement.
• Strong understanding of software development lifecycle (SDLC), DevOps practices, and integrating quality gates into CI/CD pipelines.
• Excellent communication, presentation, and interpersonal skills, with the ability to articulate complex technical concepts to diverse audiences, including senior leadership and non‑technical stakeholders.

CLOUD ANALYTICS TECHNOLOGIES is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

Equal Opportunity Employer

CLOUD ANALYTICS TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

Some of our Business Ventures - Knowledge Analytics

Do restrictive necessities, your capability for advanced analytics or your business model need custom‑made applications and infrastructure configurations? opt for our hosted managed services to urge comprehensive analytics solutions supported industry‑leading Cloud Analytics Technologies, LLC code - delivered by the Cloud Analytics Technologies, LLC consultants UN agency understand these solutions best. Through a mix of with‑it technology and dedicated service and support, Cloud Analytics Technologies, LLC hosted managed services will facilitate you: