Senior Data Engineer — Real-Time Data Pipeline Architect

Neptune Technology Group Inc.; Duluth (GA); On-site; USD 120,000 - 180,000; Full time

Job summary

Neptune Technology Group Inc. in the United States seeks a Senior Data Engineer to own and modernize the data pipeline architecture.

You will migrate ETL workloads from Redshift stored procedures and legacy SSIS to scalable, maintainable pipelines built with AWS Glue and S3, and steer the move toward near real-time analytics using stream processing. You will design resilient data workflows, optimize the data warehouse, collaborate with product and analytics teams, and establish best practices

Qualifications

• Bachelor's or Master's in CS, Engineering, or related field.
• 5+ years in data engineering roles.
• Extensive experience with AWS Glue, S3, and Redshift.
• Proven migration of ETL workloads from SSIS or stored procedures to cloud-native pipelines.
• Strong SQL skills in Redshift or similar.
• Experience with or interest in stream processing (Flink, Spark Streaming, Kafka Streams).
• Familiarity with data-pipeline orchestration, monitoring, and error handling.
• Experience with IaC and CI/CD for data pipelines.

Responsibilities

• Design and implement modern ETL and ELT pipelines leveraging AWS Glue, S3, and related services to replace legacy stored procedures and SSIS jobs.
• Architect data transformation workflows that are testable, version-controlled, and observable.
• Optimize and maintain the Redshift data warehouse, focusing on materialized views, query performance, and cost efficiency.
• Advance the platform from batch ETL to near real-time stream processing by evaluating and deploying technologies such as Apache Flink, ClickHouse, Kafka, Kinesis, or equivalent.
• Design pipelines capable of supporting both near real-time and batch workloads during the transition.
• Collaborate with product and analytics teams to ensure data models support reporting, AI/ML initiatives, and customer-facing features.
• Establish patterns and best practices for pipeline development adopted by the broader team.
• Participate in production support and incident response for data infrastructure.

Skills

SQL

Data modeling

Cloud data engineering

ETL/ELT design

Python

Education

Bachelor's or Master's in Computer Science or related field

Tools

AWS Glue

S3

Redshift

SSIS

PySpark

Python

Apache Flink

ClickHouse

Kafka

Kinesis

Apache Spark Streaming

Kafka Streams

Airflow

dbt

Step Functions

Job description

Neptune Technology Group Inc. in the United States seeks a Senior Data Engineer to own and modernize the data pipeline architecture.

You will migrate ETL workloads from Redshift stored procedures and legacy SSIS to scalable, maintainable pipelines built with AWS Glue and S3, and steer the move toward near real-time analytics using stream processing. You will design resilient data workflows, optimize the data warehouse, collaborate with product and analytics teams, and establish best practices