Data Engineer - RAG Pipelines & AWS

Accenture Federal Services; Tampa (FL); On-site; USD 119,000 - 162,000; Full time

Job summary

Accenture Federal Services, a technology company serving the US federal government, seeks a Data Engineer to design, develop, and manage large-scale data workflows. The role emphasizes RAG pipelines, ETL/ELT processes, and scalable AWS-based data systems.

You will ingests data, builds embeddings, and orchestrates retrieval flows, leveraging services like S3, Lambda, Step Functions, EMR, Glue, and NiFi to deliver robust data solutions across defense, national security, and civilian programs.

Qualifications

• Solid understanding of ETL/ELT processes and data modeling best practices.
• Hands-on experience implementing workflows in Prefect (Prefect 2.0 preferred). In-depth knowledge of ElasticSearch indexing, cluster management, and search optimization.
• Proficiency in Python and familiarity with common data libraries (Pandas, PySpark, requests, etc.).

Responsibilities

• Design, build, and maintain RAG pipelines, including document ingestion, indexing, embedding workflows, and model retrieval flows.
• Develop and manage structured and unstructured data pipelines supporting analytics, ML, and application workloads. Build and optimize ETL/ELT pipelines in AWS using services such as S3, Lambda, Step Functions, EMR, Glue, ECS/EKS, and IAM best practices. Implement and operate NiFi flows for high-throughput, low-latency data ingestion and transformation.
• Develop, orchestrate, and schedule workflows using Prefect, ensuring reliability, observability, and proper error handling. Implement indexing, search, and retrieval patterns using ElasticSearch, including schema design, cluster management, and query optimization.
• Collaborate closely with architecture, ML, and application teams to support scalable data solutions.
• Ensure data quality, lineage, governance, and security across all pipelines.
• Monitor system performance and troubleshoot issues across distributed data systems.

Skills

ETL/ELT design

Prefect 2.0

ElasticSearch indexing

Python (Pandas, PySpark)

AWS data services (S3, Lambda, Step Fn

NiFi data flows

Tools

ElasticSearch

Prefect

NiFi

OpenSearch

Docker

Kubernetes

AWS

Job description

Accenture Federal Services, a technology company serving the US federal government, seeks a Data Engineer to design, develop, and manage large-scale data workflows. The role emphasizes RAG pipelines, ETL/ELT processes, and scalable AWS-based data systems.

You will ingests data, builds embeddings, and orchestrates retrieval flows, leveraging services like S3, Lambda, Step Functions, EMR, Glue, and NiFi to deliver robust data solutions across defense, national security, and civilian programs.