Data Engineer Software Developer

RMantra Solutions; Northern (KY); Hybrid; USD 100,000 - 150,000; Full time

Job summary

RMantra Solutions is seeking a data engineer to design and maintain scalable data systems and pipelines for analytics and reporting. You will unify data from multiple network operations sources, optimize ingestion and storage, and support CI/CD for automated deployment.

The role requires 4+ years in data engineering, experience with AWS/Azure/GCP, Elastic/Kafka, and DoD security standards. Onsite work may be required; strong collaboration and documentation skills are essential.

Qualifications

• Bachelor’s degree in CS, Engineering, or related field with 4+ years of experience.
• Experience designing, building, and maintaining data pipelines (ETL/ELT) across multiple sources.
• Experience with data warehouses/lakes (AWS Redshift, S3, Hadoop, Snowflake or similar).
• Proficiency with Python, Java, SQL, and shell scripting.
• Experience with Elastic Stack (Elasticsearch, Logstash, Kibana) and/or Kafka for ingestion and processing.
• Familiarity with DoD security standards and RMF/STIGs.
• Experience with CI/CD, Git/Bitbucket, and DevOps practices.
• Strong communication and documentation skills; ability to commute to Scott AFB, IL for onsite.

Responsibilities

• Design, build, and maintain scalable data systems and pipelines for analytics and reporting.
• Unify and integrate data from multiple network operations systems into a consistent platform.
• Develop data warehouses and data lakes and optimize for performance and cost.
• Analyze and optimize ingestion throughput, query performance, and storage efficiency.
• Automate ETL/ELT and support CI/CD pipelines for automated builds and deployments.
• Collaborate with analysts, visualization engineers, and stakeholders; document architecture.
• Troubleshoot data ingestion, processing, and platform performance issues.

Skills

ETL/ELT pipelines

Python

Java

SQL

Shell scripting

Elastic Stack

Kafka

CI/CD

DevOps

Git/Bitbucket

Agile (Scrum/SAFe)

Kubernetes

Cloud platforms (AWS/Azure/GCP)

Kibana

Education

Bachelor’s degree in Computer Science, Engineering, or related technical discipline

Tools

Kafka

Spark

NiFi

Hadoop

Elasticsearch

Logstash

Databricks

ELK Stack

Kibana

Job description

• Designs, builds, and maintains scalable, reliable data systems and pipelines to ingest, process, and transform large, complex, and disparate data sources for analytics and reporting.
• Unifies and integrates data from multiple network operations systems into a consistent, accessible platform, ensuring data quality, accuracy, security, and compliance with DoD and DISA requirements.
• Develops and manages data warehouses and data lakes to support analytics, reporting, and operational dashboards, optimizing infrastructure for performance, scalability, and cost-effectiveness through automation and best practices.
• Continually analyze and optimize ingestion throughput, query performance, indexing strategies, and storage efficiency across Elastic and Kafka platforms.
• Automates data-related tasks such as ETL/ELT processes, data validation, cleansing, monitoring, and supports CI/CD pipelines for automated build, test, and deployment.
• Collaborates with data analysts, visualization engineers, and other stakeholders to deliver integrated, actionable data solutions, and documents data architecture, integration patterns, and operational procedures.
• Troubleshoots and resolves issues related to data ingestion, processing, and platform performance, storage, memory, partitioning, and cluster performance.

Basic Qualifications:

• Bachelor’s degree in Computer Science, Engineering, or related technical discipline and 4+ years of relevant experience.
• Hands-on experience designing, building, and maintaining data pipelines (ETL/ELT) and integrating multiple, disparate data sources.
• Experience with data warehouse and/or data lake technologies (e.g., AWS Redshift, S3, Hadoop, Snowflake, or similar).
• Proficiency with data engineering tools and languages such as Python, Java, SQL, and shell scripting.
• Experience with Elastic Stack (Elasticsearch, Logstash, Kibana) and/or Kafka for data ingestion and processing.
• Familiarity with network operations data (logs, metrics, events, ticketing, CMDB) and related data models.
• Experience with Linux/UNIX system administration and automation.
• Knowledge of data security, access controls, and compliance frameworks (DoD, DISA, RMF, STIGs).
• Experience with CI/CD pipelines, containerized pipelines, version control (Git, Bitbucket), and DevOps practices.
• Strong communication and documentation skills for collaboration and knowledge sharing.
• Must be able to commute to Scott AFB, IL for up to full-time onsite support.
• Ability to obtain interim Secret DoD Security clearance before the start date.
• Ability to obtain Security+ certification or equivalent DoD 8570 IAT II certification within 14 days of the start date.

REQUIRED SKILLS:

• Experience with DISA, DISN, or other DoD network environments, including developing and deploying software applications that meet DoD security standards (e.g., STIGs), and knowledge of network security, encryption, and access controls in classified environments.
• Experience with big data technologies and platforms such as Kafka, Spark, NiFi, Hadoop, Elasticsearch, Logstash, Databricks, and ELK Stack for text mining, summarization, search, and entity extraction.
• Experience with cloud-based data platforms (AWS, Azure, GCP, AWS GovCloud), cloud-integrated platforms, and infrastructure as code, including networking and security policies.
• Familiarity with Kubernetes deployment, platform upgrades, patching, configuration management tools (Ansible, Puppet, Chef), and monitoring, alerting, and observability tools (Prometheus, Grafana, Elastic).
• Experience with BI tools like Kibana and developing and maintaining technical documentation and runbooks.
• Strong understanding and practical experience with Agile methodologies (Scrum, SAFe), and related tools such as Atlassian JIRA and Confluence.
• Certified Confluent Developer and Certified Elastic Engineer.
• Excellent communication and collaboration skills, with experience working remotely with geographically dispersed teams and within a matrixed organization to combine software, integration, and data engineering practices.
• Apply knowledge of system architecture, networks, and centralized logging (ELK) to support data transformation initiatives.

Join a dynamic team where your ideas matter and your career thrives.