Data Architect

Quantum Technologies. LLC; Atlanta (GA); On-site; USD 150,000 - 210,000; Full time

Job summary

Quantum Technologies. LLC is seeking a hands-on Data Architect to own the design, quality, and performance of an AWS-based data platform. You will define architecture, write production code, and mentor the data engineering team.

You’ll also translate business needs into reliable, compliant data solutions in the financial domain. The role emphasizes end-to-end data modeling, secure data handling, cost optimization, and leadership across data engineers.

Qualifications

• 9+ years in data domain (data engineering, warehousing, big data) with 3–4 years in architect/lead roles.
• Proven experience delivering large-scale data platforms on AWS.
• Strong Python and PySpark programming skills.

Responsibilities

• Design end-to-end data architectures on AWS including ingestion, transformation, storage and consumption layers.
• Lead hands-on development of PySpark/Python pipelines and optimize Spark jobs for performance.
• Build and optimize AWS Glue jobs, Lambda functions, and Airflow DAGs for reliable pipelines.
• Implement data quality checks, governance, lineage, and cost controls across platforms.
• Provide technical leadership, code reviews, and mentoring for the data engineering team.

Skills

Python

PySpark

SQL

Apache Spark

Airflow

Terraform

CloudFormation

Git

CI/CD

Data modeling

Data warehouse

Data lake

ETL

AWS

Tools

AWS Glue

S3

Lambda

IAM

CloudWatch

Apache Airflow

Delta Lake

Job description

Work Authorization

• US Citizen


• Green Card


• H-1B


• OPT-EAD


• CPT


• Other

Job Description

We are looking for a hands-on Data Architect to own the design, quality, and performance of our AWS-based data platform.

You will define the architecture, write and review production code, and mentor the data engineering team.

You will also need to understand the financial domain well enough to translate business needs into reliable, compliant data solutions.

Key Responsibilities :

• Architecture & Design Design end-to-end data architecture on AWS, covering ingestion, transformation, storage, and consumption layers Define data modelling standards (dimensional, data lake/lakehouse, partitioning and file format strategies) Make technology and design decisions, and document them through architecture decision records and design reviews Ensure the platform is scalable, secure, cost-efficient, and resilient


• Hands-on Development & Optimization Write high-quality PySpark and Python code for complex pipelines and critical components Tune and optimize Spark jobs (shuffle, skew, partitioning, broadcast joins, memory/executor configuration, caching) Build and optimize AWS Glue jobs, Lambda functions, and Airflow DAGs Optimize S3 data layout, file formats (Parquet/ORC/Iceberg), and file sizing for performance and cost Monitor and control AWS costs across compute and storage Build reusable frameworks and libraries (logging, error handling, data quality checks, reconciliation, and so on) Code Review & Technical Leadership Review code, pull requests, and designs, and enforce engineering best practices Guide, mentor, and upskill data engineers Set coding standards, CI/CD practices, and testing strategy for data pipelines Act as the technical escalation point for production issues and root cause analysis


• Financial Domain & Governance Work with business, risk, finance, and compliance stakeholders to understand requirements Design pipelines for financial data such as transactions, ledgers, payments, positions, and regulatory reporting Ensure data accuracy, lineage, auditability, reconciliation, and controls Apply data security practices: encryption, IAM, masking/tokenization of sensitive data, and access control Required

Skills & Experience

Experience 9+ years in the data domain (data engineering, warehousing, big data), with at least 3-4 years in an architect or technical lead role Proven experience designing and delivering large-scale data platforms on AWS Technical Skills

Programming: Strong Python and PySpark; advanced SQL Processing: Deep Apache Spark expertise, including internals and performance tuning AWS: AWS Glue (ETL, Data Catalog, Crawlers), S3, Lambda, IAM, CloudWatch Orchestration: Apache Airflow (DAG design, dependencies, retries, SLAs, monitoring) Data modeling: Data lake, data warehouse, and dimensional modeling Optimization: Partitioning, file sizing, compression, and cost/performance trade-offs Engineering practices: Git, CI/CD, unit/integration testing, code quality tools, Infrastructure as Code (Terraform or CloudFormation)

Domain Knowledge Solid understanding of the financial domain (banking, capital markets, insurance, payments, or lending) Familiarity with financial data concepts: reconciliation, regulatory reporting, audit trails, data retention, and data privacy Soft Skills Strong communication with both technical and business stakeholders Mentoring and team leadership Ownership mindset and the ability to make sound decisions under ambiguity Good to Have Redshift, Athena, EMR, Lake Formation, Step Functions, Kinesis/Kafka Open table formats (Apache Iceberg, Hudi, Delta Lake) AWS certifications (Solutions Architect, Data Analytics/Data Engineer)

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.