Data Architect

Incedo Inc.

United States

On-site

USD 120,000 - 150,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

A leading IT services firm is seeking a Data Architect to design and implement scalable data pipelines using AWS technologies. This mid-senior level role involves leading the development of real-time data solutions and ensuring data quality across platforms. Ideal candidates will have over 6 years of data engineering experience and strong proficiency in AWS Glue and Apache Kafka. Join our team in fostering innovation for financial applications!

Qualifications

  • 6+ years of hands-on data engineering experience in cloud-based environments (AWS preferred).
  • Strong experience with Apache Kafka / AWS MSK including topic design and partitioning.
  • Proficiency in AWS Glue (PySpark) and Flink / MSF for ETL.

Responsibilities

  • Lead development of real-time data pipelines using AWS DMS and MSK.
  • Build and optimize streaming and batch data pipelines using AWS Glue.
  • Implement monitoring, logging, and event-based alerting.

Skills

Data engineering in cloud-based environments
Apache Kafka / AWS MSK
AWS Glue (PySpark)
AWS DMS
CloudWatch monitoring
Infrastructure as code (CDK/Terraform)

Tools

AWS Glue
Apache Kafka
AWS Flink
DynamoDB
Iceberg

Job description

Assistant Manager - Talent Acquisition @ Incedo Inc. | Global Hiring | Mastering LinkedIn Recruiter

About the Role:

We are seeking a Data Architect / Technical Lead with strong expertise in AWS-based streaming data pipelines, Apache Kafka (MSK), AWS Glue, AWS Flink (MSF), and PySpark, to help design and implement a scalable data ingestion, processing, and reconciliation framework.

You will play a key role in building out the Ingress (CDC) and TPM (Transformation & Processing) layers, and drive the data quality and reconciliation workflows for a high-impact data platform serving financial applications.

Key Responsibilities:

  • Lead the development of real‑time data pipelines using AWS DMS, MSK, and Flink (MSF) for CDC ingestion from multiple SQL Server sources (RDS/on‑prem).
  • Build and optimize streaming and batch data pipelines using AWS Glue (PySpark) to validate, transform, and normalize data to Iceberg and DynamoDB.
  • Define and enforce data quality, lineage, and reconciliation logic with support for both streaming and batch use cases.
  • Integrate with S3 Bronze/Silver layers and implement efficient schema evolution and partitioning strategies using Iceberg.
  • Collaborate with architects, analysts, and downstream application teams to design API and file‑based egress layers.
  • Implement monitoring, logging, and event‑based alerting using CloudWatch, SNS, and EventBridge.
  • Mentor junior developers and enforce best practices for modular, secure, and scalable data pipeline development.

Required Skills:

  • 6+ years of hands‑on data engineering experience in cloud‑based environments (AWS preferred)
  • Strong experience with Apache Kafka / AWS MSK including topic design, partitioning, and Kafka Connect/Debezium
  • Proficiency in AWS Glue (PySpark) and Flink / MSF for both batch and streaming ETL
  • Working knowledge of AWS DMS, S3, Lake Formation, DynamoDB, and Iceberg
  • Solid grasp of schema evolution, CDC patterns, and data reconciliation frameworks
  • Experience with infrastructure‑as‑code (CDK/Terraform) and DevOps practices (CI/CD, Git)

Preferred:

  • Familiarity with PostgreSQL, Athena, Step Functions, and Lambda
  • Knowledge of event‑driven architectures and microservices‑based data APIs
Seniority Level

Mid‑Senior level

Employment Type

Full‑time

Job Function

Consulting

Industries

IT Services and IT Consulting, Financial Services, and Investment Banking

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AWS DATA Architect
AWS DATA Architect

Incedo Inc. • New Jersey

Hybrid
USD 120,000 - 150,000
Data Architect
Data Architect

Talentola • Atlanta (GA)

On-site
USD 140,000 - 180,000
Data Architect
Data Architect

Incedo Inc. • San Rafael (CA)

On-site
USD 100,000 - 140,000
Tech Lead, Data & Inference Engineer
Tech Lead, Data & Inference Engineer

Catalyst Labs • Seattle (WA)

Hybrid
USD 120,000 - 160,000
Tech Lead, Data & Inference Engineer
Tech Lead, Data & Inference Engineer

Catalyst Labs • New York (NY)

Hybrid
USD 120,000 - 160,000
Technical Program Manager
Technical Program Manager

Incedo Inc. • New York (NY)

On-site
USD 150,000 - 200,000
Data Architect -AWS
Data Architect -AWS

Digital iTechnology • Atlanta (GA)

On-site
USD 140,000 - 190,000
Tech Lead, Data & Inference Engineer
Tech Lead, Data & Inference Engineer

Catalyst Labs • Connecticut

Hybrid
USD 120,000 - 180,000
Tech Lead, Data & Inference Engineer
Tech Lead, Data & Inference Engineer

Catalyst Labs • Jacksonville (FL)

Hybrid
USD 130,000 - 160,000
Tech Lead, Data & Inference Engineer
Tech Lead, Data & Inference Engineer

Catalyst Labs • Greenwich (CT)

Hybrid
USD 120,000 - 160,000