Data Engineer III

Dexian; North Carolina; On-site; USD 140,000 - 190,000; Full time

Job summary

Dexian is seeking a Senior Data Engineer to design, build, and maintain scalable data pipelines powering analytics and data products. The role emphasizes cloud-native solutions on AWS, distributed systems, and hands-on stream processing with Apache Flink using Python and Java.

You will design and maintain batch and real-time pipelines, implement ingestion with AWS DMS, Kafka, and Lambda, and work with Aurora PostgreSQL for data persistence. Collaboration with analytics and BI teams is essential.

Qualifications

• 5+ years of data engineering or backend engineering experience.
• Strong hands-on Python and Java in distributed systems.
• Experience building streaming apps with Flink and Kafka.
• Solid experience with AWS data services.
• Strong SQL and relational data modeling skills.
• Experience with large-scale distributed data systems.

Responsibilities

• Design, build, and maintain scalable data pipelines for batch and real-time workloads.
• Develop and optimize data processing jobs using PySpark, Python, and Java.
• Build stream processing apps with Apache Flink.
• Implement reliable data ingestion with AWS DMS, Kafka, and AWS Lambda.
• Design data persistence with Aurora (PostgreSQL) and related AWS services.
• Monitor, debug, and optimize pipelines for performance and cost.
• Collaborate with product, analytics, and platform teams to translate requirements into tech solutions.

Skills

Python

Java

PySpark

Apache Flink

Kafka

AWS Glue

AWS DMS

AWS Lambda

Amazon Aurora

SQL

Distributed systems

Tools

AWS Glue

AWS DMS

Kafka

Apache Flink

Qlik

Job description

Description: Senior Data Engineer - Data Platforms

Role Overview

We are seeking a Senior Data Engineer to design, build, and maintain scalable data engineering solutions that power analytics, reporting, and data products across the organization. This role works closely with product managers, data analysts, BI developers, and other engineers to deliver reliable, high quality data pipelines across both batch and real time streaming architectures.

The ideal candidate has deep experience building cloud native data solutions on AWS, strong fundamentals in distributed systems, and hands on experience with stream processing frameworks such as Apache Flink, using Python and Java.

Key Responsibilities

Data Engineering & Platform Development

• Design, develop, and maintain scalable data pipelines supporting batch and real time streaming workloads
• Build and optimize data processing jobs using AWS Glue, PySpark, Python, and Java
• Develop and maintain stream processing applications using Apache Flink
• Implement reliable data ingestion solutions using AWS DMS, Kafka, and AWS Lambda
• Design and manage data persistence layers using Amazon Aurora (PostgreSQL) and related AWS services

Streaming & Batch Processing

• Design and support real time and near real time streaming pipelines using Kafka and Apache Flink
• Build and maintain stateful stream processing jobs, including windowing, aggregation, and event time processing
• Develop efficient batch processing workflows for large scale data transformation and enrichment

Data Quality, Reliability & Performance

• Implement monitoring, logging, and alerting for batch and streaming data pipelines
• Diagnose and resolve data quality, performance, and scalability issues
• Apply best practices for schema evolution, checkpointing, fault tolerance, and back pressure handling
• Optimize pipelines for performance and cost efficiency

Analytics & Consumption

• Enable downstream analytics and reporting through well modeled, well documented datasets
• Partner with analytics and BI teams supporting tools such as Qlik
• Support data consumers by improving data discoverability, usability, and trust

Collaboration & Engineering Excellence

• Collaborate with product, analytics, and platform teams to translate requirements into technical solutions
• Participate in architecture discussions, design reviews, and code reviews
• Contribute to data engineering standards, reusable frameworks, and documentation
• Mentor junior engineers and promote best practices across the team

Core Technologies

• AWS Glue
• PySpark
• Python
• Java
• Apache Flink (Stream Processing)
• Kafka
• AWS DMS
• AWS Lambda
• Amazon Aurora (PostgreSQL)
• Streaming and Batch Data Processing
• Qlik (analytics / BI consumption)

Preferred / Nice to Have Experience

• Amazon Redshift
• Data warehousing concepts (dimensional modeling, star/snowflake schemas)
• Experience building or supporting enterprise data warehouses or data lakes
• Familiarity with event driven architectures and real time analytics use cases
• Experience with data governance, metadata management, or lineage tools

Required Qualifications

• 5+ years of experience in data engineering or backend engineering
• Strong hands on experience with Python and Java in distributed systems
• Experience building streaming data applications using Apache Flink and Kafka
• Solid experience with AWS data services
• Strong SQL skills and understanding of relational data modeling
• Experience working with large scale, distributed data systems

What Success Looks Like

• Streaming and batch pipelines are reliable, scalable, and observable
• Real time data is processed with low latency and high correctness
• Data is trusted and easily consumable by analytics and downstream systems
• Systems are designed with fault tolerance, performance, and maintainability in mind
• Engineering best practices are consistently applied and shared

Dexian stands at the forefront of Talent + Technology solutions with a presence spanning more than 70 locations worldwide and a team exceeding 10,000 professionals. As one of the largest technology and professional staffing companies and one of the largest minority-owned staffing companies in the United States, Dexian combines over 30 years of industry expertise with cutting-edge technologies to deliver comprehensive global services and support.

Dexian connects the right talent and the right technology with the right organizations to deliver trajectory-changing results that help everyone achieve their ambitions and goals. To learn more, please visit https://dexian.com/.

Dexian is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.