Apache Kafka/Nifi - Software Data Engineer

Atlas Tech; North Charleston (SC); On-site; USD 110,000 - 150,000; Full time

Benefits offered by this job

Competitive salary

Generous benefits package

Impactful work in national security

Job summary

Atlas Technologies, Inc. seeks a Data Engineer with Kafka/NiFi experience to design, build, and maintain data systems supporting DoD/NIWC. Hybrid role in North Charleston, SC, with responsibilities spanning ETL, data architecture, and deployment automation.

You will collaborate across teams to ensure data systems meet mission objectives, while embracing Agile and DevOps practices in a fast-paced environment.

Qualifications

• This role requires U.S. Citizenship and the ability to maintain a DoD Secret clearance.
• 3–5 years of data engineering experience with modern data platforms.
• Experience with open-source, distributed data streaming and orchestration tools.

Responsibilities

• Design, build, and maintain ETL pipelines moving data into centralized repositories.
• Develop automated deployment pipelines and IaC templates to automate tasks.
• Design and optimize data architectures for performance, cost, security, and scale.
• Create data lake/warehouse solutions and data storage strategies.
• Document architectures and present proposals to engineering teams.
• Collaborate with multiple engineering groups to align infrastructure with mission objectives.

Skills

Apache Kafka

Apache NiFi

Docker

Kubernetes

CI/CD

Agile

DevOps

PostgreSQL

Redis

MinIO

Grafana/ELK

Nginx

Debezium

Education

Bachelor's degree or equivalent experience

Tools

Helm charts

Jenkins/GitHub Actions

Terraform/IaC

Job description

Atlas Technologies, Inc., a rapidly expanding company supporting the U.S. Military and other Government entities, is seeking a fast-learning and self-motivated Data Engineer with Apache Kafka/Nifi experience to design, build, and maintain the systems and infrastructure that collect, store, and transform data for the Department of Defense (DoD) and the Naval Information Warfare Center (NIWC). This effort is primarily centered around Data Engineering support for PMW160 Tactical Networks. This is a Hybrid role in North Charleston, SC.

You Will

• Support designing and building ETL (Extract, Transform, Load) processes and pipelines to move data from disparate sources into centralized repositories
• Develop automated deployment pipelines and Infrastructure-as-Code templates using scripts and orchestration tools to automate repetitive data tasks, reducing manual intervention and error
• Design and optimize data architectures to improve performance, cost-efficiency, security, and scalability
• Design and implement large-scale processing systems and database architecture solutions (e.g., Data Lakes, Data Warehouses)
• Provide recommendations for data engineering best practices and process improvements
• Contribute to team objectives and support data engineering initiatives
• Prepare technical documentation and present architecture proposals to engineering teams
• Partner with other engineering groups to understand their data requirements and with leadership to align technical infrastructure with mission objectives

You Have

• U.S. Citizenship
• Possession of and the ability to maintain a Department of Defense (DoD) Secret Level Security Clearance.
• 3-5 years' experience supporting data engineering efforts and initiatives
• Experience with containerization technologies (Docker, Kubernetes) is highly valued
• Experience with helm charts and deploying managing and scaling applications
• Working knowledge of Agile methodology and DevOps practices
• Experience with open-source, distributed event streaming platform used to collect, process, store, and analyze data feeds (Apache Kafka)
• Experience with data orchestration tools (Apache Nifi)
• Experience with open-source monitoring and observability architectures to include Grafana LGTM+ Stack and Elastic (ELK) Stack
• Experience with high-performance, open-source networking infrastructure tools (Nginx, Apache APISIX)
• Experience with open-source, distributed event streaming platform used to collect, process, store, and analyze data feeds (Apache Kafka, Debezium)
• Experience with data orchestration tools (Apache Nifi)
• Experience with open-source data storage and caching services (Postgres, Redis, MinIO)
• Strong documentation and communication skills to work effectively across technical teams
• Collaborative mindset with ability to work in fast-paced environments
• Self-motivated with strong problem-solving skills and attention to detail
• Ability to manage multiple projects and priorities simultaneously
• Bachelor's degree preferred or equivalent experience

We'd Be Impressed If You Had

• Bachelor's degree in Computer Science, Engineering, or related field
• 5+ years' experience in data engineering roles
• Experience with CI/CD tools (Jenkins, GitLab CI, Azure DevOps, GitHub Actions)
• Proficiency with version control systems and collaborative development tools
• Experience with configuration management and automation platforms (Ansible, Chef, Puppet)
• Programming/scripting experience (Python, PowerShell, Bash, Go)
• Experience with data engineering design and visualization tools (Swagger)
• Knowledge of microservices architecture and serverless computing

Want to hear more? Read on to see what our Team Members say about their experience:

“To me, the best part about working at Atlas is the level of transparency afforded to everyone in the company. I’ve had plenty of jobs where I knew what I was doing, but at Atlas there is a concerted effort to make sure we also understand why, and how our role directly impacts the overall mission.” - Jess

“With LEARNING as a core value, Atlas is constantly promoting ways to increase knowledge and stay at the forefront of the latest technology. Team members are regularly encouraged to continue learning and Atlas provides training resources and skills enhancement opportunities internally as well as education reimbursement if you prefer to obtain skills through outside resources or higher education.” - Wendy

Atlas Tech is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, sexual orientation, gender identity, national origin, disability, genetic information, or protected veteran status. We are committed to fostering an inclusive workplace where diverse perspectives are valued and respected.

Accessibility & Accommodations

Atlas Tech strives to make our website, www.atlas-tech.com , accessible to all users. If you need assistance completing the application process or require a reasonable accommodation due to a disability, please contact us at [e-mail] .

This contact information is designated for accommodation requests only and cannot be used to inquire about the status of an application. Atlas Tech is committed to providing reasonable accommodations in accordance with the Americans with Disabilities Act (ADA).

• competitive salary
• generous benefits package
• opportunity to make a positive impact in your own community