Data Architect -AWS

Digital iTechnology; Atlanta (GA); On-site; USD 140,000 - 190,000; Full time

Job summary

Digital iTechnology in Atlanta seeks an experienced Data Architect to lead end-to-end data platform design on AWS, with hands-on delivery of pipelines using Python, PySpark, Spark, Glue, S3, and Airflow. You will partner with finance and risk teams to ensure accurate, auditable data and compliant governance.

You'll mentor engineers, define data models, optimize costs, and ensure scalable, secure architectures across ingestion, storage, and consumption layers in a production environment.

Qualifications

• 9+ years of experience in data engineering, data warehousing, or big data technologies.

Responsibilities

• Design end-to-end data architectures on AWS for ingestion, transformation, storage and consumption.
• Define data modeling standards including dimensional modeling, data lakes, lakehouse architecture, partitioning and file formats.
• Establish architectural standards, evaluate technologies, and document design decisions.
• Mentor data engineers and lead production troubleshooting, performance optimization, and root cause analysis.
• Collaborate with finance, risk, and compliance stakeholders to translate requirements into technical solutions.

Skills

Python

PySpark

Apache Spark

SQL

Airflow

AWS Glue

S3

IAM

Tools

AWS Glue

Lambda

Airflow

Git

CI/CD

Terraform

CloudFormation

Job description

Looking for an experienced, hands-on Data Architect with strong AWS expertise and financial domain experience to lead the design, development, and optimization of enterprise-scale data platforms.

The ideal candidate should have extensive experience in Python, PySpark, Apache Spark, AWS Glue, S3, and Airflow, along with the ability to define end-to-end data architecture, develop production-grade data pipelines, optimize performance, and mentor data engineering teams. Strong knowledge of financial data, governance, reconciliation, and regulatory compliance is essential.

Key Responsibilities

• Design end-to-end data architectures on AWS, covering ingestion, transformation, storage, and consumption layers.
• Define data modeling standards, including dimensional modeling, data lakes, lakehouse architecture, partitioning, and file formats.
• Establish architectural standards, evaluate technologies, and document design decisions.
• Ensure data platforms are scalable, secure, resilient, and cost-efficient.

Hands-on Development & Optimization

• Develop complex data pipelines using Python, PySpark, and Apache Spark.
• Optimize Spark jobs through partitioning, shuffle management, skew handling, broadcast joins, caching, and memory tuning.
• Build and optimize AWS Glue jobs, Lambda functions, and Apache Airflow DAGs.
• Optimize S3 data layouts, Parquet/ORC/Iceberg formats, and file sizing strategies.
• Develop reusable frameworks for logging, error handling, data quality, validation, and reconciliation.
• Monitor and optimize AWS compute and storage costs.

Technical Leadership

• Review code, pull requests, and architecture designs to enforce engineering best practices.
• Establish coding standards, CI/CD processes, and automated testing strategies.
• Mentor and guide data engineers and provide technical direction.
• Lead production troubleshooting, performance optimization, and root cause analysis.
• Collaborate with finance, risk, compliance, and business stakeholders to translate requirements into technical solutions.
• Design data pipelines for financial transactions, ledgers, payments, positions, and regulatory reporting.
• Ensure data accuracy, lineage, auditability, reconciliation, and regulatory compliance.
• Implement security controls, including encryption, IAM, data masking, tokenization, and access management.

Required Skills & Qualifications

• 9+ years of experience in data engineering, data warehousing, or big data technologies.
• 3–4 years of experience in a Data Architect, Technical Lead, or equivalent role.
• Strong hands-on expertise in Python, PySpark, advanced SQL, and Apache Spark.
• Deep understanding of Apache Spark internals and performance optimization.
• Strong AWS experience with Glue, S3, Lambda, IAM, and CloudWatch.
• Experience with Apache Airflow, including DAG design, dependencies, retries, SLAs, and monitoring.
• Expertise in data lakes, lakehouse architecture, data warehousing, and dimensional modeling.
• Experience with Git, CI/CD, unit/integration testing, and Infrastructure as Code (Terraform or CloudFormation).
• Strong financial domain experience in banking, capital markets, insurance, payments, or lending.
• Understanding of financial reconciliation, regulatory reporting, audit trails, data privacy, and data governance.
• Excellent communication, problem-solving, technical leadership, and mentoring skills.

Good to Have

• Experience with Amazon Redshift, Athena, EMR, Lake Formation, Step Functions, and Kinesis/Kafka.
• Knowledge of Apache Iceberg, Hudi, or Delta Lake.
• AWS certifications such as Solutions Architect, Data Analytics, or Data Engineer.

Note: This is a hands-on technical leadership position requiring the ability to drive architecture decisions while actively developing, reviewing, and optimizing production data pipelines.