Data Architect

Valiant Technologies LLC; Atlanta (GA); On-site; USD 140,000 - 200,000; Full time

Job summary

Valiant Technologies LLC is seeking a hands-on Data Architect to own the design, quality, and performance of our AWS-based data platform. You will define the architecture, write and review production code, and mentor the data engineering team.

You will translate business needs into reliable, compliant data solutions within the financial domain, ensuring data accuracy, lineage, auditability, and governance while optimizing costs and performance.

Qualifications

• 9+ years in the data domain (data engineering, warehousing, big data).
• Proven experience designing and delivering large-scale data platforms on AWS.
• Strong technical skills in Python, PySpark, SQL, and Spark internals.

Responsibilities

• Design end-to-end data architecture on AWS for ingestion, transformation, storage, and consumption.
• Develop and optimize PySpark/Python pipelines; tune Spark jobs and AWS Glue/Lambda workflows.
• Collaborate with stakeholders on financial data governance, security, and regulatory reporting.

Skills

Python

PySpark

SQL

Apache Spark

Data Modeling

Data Warehousing

Tools

AWS Glue

S3

Lambda

IAM

CloudWatch

Apache Airflow

Terraform

CloudFormation

Git

Job description

Work Authorization

• US Citizen


• Green Card


• H-1B


• OPT-EAD


• CPT


• Other

Job Description

We are looking for a hands-on Data Architect to own the design, quality, and performance of our AWS-based data platform.

You will define the architecture, write and review production code, and mentor the data engineering team.

You will also need to understand the financial domain well enough to translate business needs into reliable, compliant data solutions.

Key Responsibilities :

• Architecture & Design Design end-to-end data architecture on AWS, covering ingestion, transformation, storage, and consumption layers Define data modelling standards (dimensional, data lake/lakehouse, partitioning and file format strategies) Make technology and design decisions, and document them through architecture decision records and design reviews Ensure the platform is scalable, secure, cost-efficient, and resilient


• Hands-on Development & Optimization Write high-quality PySpark and Python code for complex pipelines and critical components Tune and optimize Spark jobs (shuffle, skew, partitioning, broadcast joins, memory/executor configuration, caching) Build and optimize AWS Glue jobs, Lambda functions, and Airflow DAGs Optimize S3 data layout, file formats (Parquet/ORC/Iceberg), and file sizing for performance and cost Monitor and control AWS costs across compute and storage Build reusable frameworks and libraries (logging, error handling, data quality checks, reconciliation, and so on) Code Review & Technical Leadership Review code, pull requests, and designs, and enforce engineering best practices Guide, mentor, and upskill data engineers Set coding standards, CI/CD practices, and testing strategy for data pipelines Act as the technical escalation point for production issues and root cause analysis


• Financial Domain & Governance Work with business, risk, finance, and compliance stakeholders to understand requirements Design pipelines for financial data such as transactions, ledgers, payments, positions, and regulatory reporting Ensure data accuracy, lineage, auditability, reconciliation, and controls Apply data security practices: encryption, IAM, masking/tokenization of sensitive data, and access control Required

Skills & Experience

Experience 9+ years in the data domain (data engineering, warehousing, big data), with at least 3-4 years in an architect or technical lead role Proven experience designing and delivering large-scale data platforms on AWS Technical Skills

Programming: Strong Python and PySpark; advanced SQL Processing: Deep Apache Spark expertise, including internals and performance tuning AWS: AWS Glue (ETL, Data Catalog, Crawlers), S3, Lambda, IAM, CloudWatch Orchestration: Apache Airflow (DAG design, dependencies, retries, SLAs, monitoring) Data modeling: Data lake, data warehouse, and dimensional modeling Optimization: Partitioning, file sizing, compression, and cost/performance trade-offs Engineering practices: Git, CI/CD, unit/integration testing, code quality tools, Infrastructure as Code (Terraform or CloudFormation)

Domain Knowledge Solid understanding of the financial domain (banking, capital markets, insurance, payments, or lending) Familiarity with financial data concepts: reconciliation, regulatory reporting, audit trails, data retention, and data privacy Soft Skills Strong communication with both technical and business stakeholders Mentoring and team leadership Ownership mindset and the ability to make sound decisions under ambiguity Good to Have Redshift, Athena, EMR, Lake Formation, Step Functions, Kinesis/Kafka Open table formats (Apache Iceberg, Hudi, Delta Lake) AWS certifications (Solutions Architect, Data Analytics/Data Engineer)

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.