AWS Data Engineer - Real-Time Data Pipelines & Architecture

TechDigital Group; Kentucky; Hybrid; USD 90,000 - 120,000; Full time

Job summary

A leading tech company in Kentucky is seeking a Data Engineer to lead technical milestones and develop scalable data architectures. Responsibilities include creating POCs, managing data ingestion services, and designing event-driven architectures using AWS. The ideal candidate should have strong skills in Python, AWS services, and NoSQL databases. This position requires collaboration with stakeholders on data architecture and optimization opportunities.

Responsibilities

• Lead the team technically to complete milestones on time.
• Understand complete requirement, create Architecture and update all stakeholders.
• Create POCs.
• Manage delivery/release to customer.
• Develop Services for data ingestion and synchronization.
• Ingest data from multiple sources using Python.
• Design event-driven architecture using AWS EventBridge or SNS/SQS.
• Implement scalable data pipelines integrating on-prem and AWS environments.
• Develop efficient Python scripts to handle large datasets.
• Work with various NoSQL databases.
• Develop applications in a cloud-native architecture.
• Monitor data workflows, troubleshoot issues, and optimize performance.
• Transition existing pipeline to MSSQL server.
• Collaborate on existing data architecture.
• Design and document target data architecture and pipelines.
• Identify opportunities for optimization.
• Collaborate on business logic decomposition.

Skills

AWS

Glue

SNS/SQS

Python

Py-Spark

Data Lake

Cloud Watch

Cloud Trail

DB Design

SQL

Job description

A leading tech company in Kentucky is seeking a Data Engineer to lead technical milestones and develop scalable data architectures. Responsibilities include creating POCs, managing data ingestion services, and designing event-driven architectures using AWS. The ideal candidate should have strong skills in Python, AWS services, and NoSQL databases. This position requires collaboration with stakeholders on data architecture and optimization opportunities.