Real-Time Data Architect — AWS, Kafka & PySpark

Incedo Inc.; United States; On-site; USD 120,000 - 150,000; Full time

Job summary

A leading IT services firm is seeking a Data Architect to design and implement scalable data pipelines using AWS technologies. This mid-senior level role involves leading the development of real-time data solutions and ensuring data quality across platforms. Ideal candidates will have over 6 years of data engineering experience and strong proficiency in AWS Glue and Apache Kafka. Join our team in fostering innovation for financial applications!

Qualifications

• 6+ years of hands-on data engineering experience in cloud-based environments (AWS preferred).
• Strong experience with Apache Kafka / AWS MSK including topic design and partitioning.
• Proficiency in AWS Glue (PySpark) and Flink / MSF for ETL.

Responsibilities

• Lead development of real-time data pipelines using AWS DMS and MSK.
• Build and optimize streaming and batch data pipelines using AWS Glue.
• Implement monitoring, logging, and event-based alerting.

Skills

Data engineering in cloud-based environments

Apache Kafka / AWS MSK

AWS Glue (PySpark)

AWS DMS

CloudWatch monitoring

Infrastructure as code (CDK/Terraform)

Tools

AWS Glue

Apache Kafka

AWS Flink

DynamoDB

Iceberg

Job description

A leading IT services firm is seeking a Data Architect to design and implement scalable data pipelines using AWS technologies. This mid-senior level role involves leading the development of real-time data solutions and ensuring data quality across platforms. Ideal candidates will have over 6 years of data engineering experience and strong proficiency in AWS Glue and Apache Kafka. Join our team in fostering innovation for financial applications!