Data Engineering Software Developer Sr.

Amatriot Group, LLC; United States; On-site; USD 175,000 - 200,000; Full time

Job summary

Amatriot Group, LLC seeks a Senior Data Engineer in Washington, DC to design, build, and sustain scalable data systems and pipelines across complex enterprise environments. The role focuses on ingesting, processing, transforming, and unifying data sources while ensuring data quality, security, and compliance with DoD and DISA requirements.

The candidate will work with Confluent Kafka, Elastic Stack, AWS, OpenShift/Kubernetes, data warehouses, and data lakes; supporting automation, CI/CD,

Qualifications

• Hands on experience designing, building and maintaining ETL/ELT data pipelines.
• Experience with data warehouse and data lake technologies (AWS Redshift, S3, Hadoop, Snowflake).
• Experience with Elastic Stack (Elasticsearch, Logstash, Kibana) and Kafka.
• Linux/UNIX system administration and automation skills.
• CI/CD pipelines and DevOps practices; version control (Git/Bitbucket).
• Familiarity with DoD DISA RMF and STIG requirements.
• Strong written and verbal communication; documentation skills.
• Active Security clearance (TSSCI) or higher.

Responsibilities

• Data Pipeline Engineering: design, build, and maintain scalable pipelines.
• Unify data from multiple systems into a consistent platform.
• Create and maintain Kafka, Elastic, and Logstash pipelines.
• Automate ETL/ELT processes and data quality checks.
• Develop data warehouses and lakes for analytics and dashboards.
• Optimize infrastructure for scalability, performance, and cost.
• Support CI/CD, platform automation, and operational efficiency.
• Collaborate across data, software, and network operations teams.

Skills

ETL/ELT pipelines

Data ingestion

Data integration

Python/Java/SQL

CI/CD pipelines

Linux/UNIX

Shell scripting

Security awareness

Communication

Education

Bachelors degree in CS/Engineering

Tools

Kafka

Elastic Stack

AWS

OpenShift/Kubernetes

ELK Stack

Job description

Location

Washington DC 100 Onsite Clearance Top SecretSCI Job Type Full time Target Salary Range 175000 200000

This represents the potential salary range for this position depending on education level years of experience andor certifications in addition to other position specific requirements which may impact salary

Position Overview

The Senior Data Engineer designs builds integrates and sustains scalable data systems and pipelines that support analytics reporting and operational insight across complex enterprise environments. This role focuses on ingesting processing transforming and unifying large and disparate data sources while ensuring data quality security performance and compliance with DoD and DISA requirements. The Senior Data Engineer works with Confluent Kafka Elastic Stack AWS Red Hat OpenShiftKubernetes data warehouses and data lakes to deliver reliable and efficient data solutions. The role also supports automation CICD platform optimization visualization capabilities and collaboration across data software and network operations teams.

Key Responsibilities

Data Pipeline Engineering and IntegrationDesign build and maintain scalable and reliable data systems and pipelines for ingesting processing and transforming large complex and disparate data sourcesUnify and integrate data from multiple network operations systems into a consistent and accessible platformCreate and maintain Kafka Elastic and Logstash pipelines to support enterprise data ingestion and processingAutomate ETLELT processes data validation cleansing monitoring and related data engineering activitiesData Platforms and ArchitectureImplement data solutions using Confluent Kafka Elastic Stack AWS and Red Hat OpenShiftKubernetesDevelop and manage data warehouses and data lakes supporting analytics reporting and operational dashboardsOptimize data infrastructure for scalability reliability performance and cost effectivenessApply automation and engineering best practices to improve platform maintainability and operational efficiencyPerformance and Platform OptimizationAnalyze and optimize ingestion throughput query performance indexing strategies storage efficiency and cluster utilization across Elastic and Kafka platformsTroubleshoot and resolve issues related to data ingestion processing storage memory partitioning and cluster performanceIdentify and resolve platform bottlenecks affecting data availability and downstream analyticsVisualization and Analytics SupportSupport Kibana visualizations and dashboardsSupport visualization capabilities using React JavaScript and HTMLCollaborate with data analysts visualization engineers and other stakeholders to deliver integrated and actionable data solutionsSecurity Compliance and DevOpsEnsure data quality accuracy security and compliance with applicable DoD and DISA requirementsSupport CICD pipelines for automated build test and deployment activitiesApply version control and DevOps practices to data engineering workflowsSupport secure operation of data platforms through appropriate access controls and configuration standardsDocumentation and CollaborationDocument data architecture integration patterns platform configurations and operational proceduresMaintain technical documentation and runbooks supporting data platforms and pipelinesCollaborate across software integration data analytics and network operations teams to support enterprise data initiatives

Education

Bachelors degree in Computer Science Engineering or a related technical discipline with 8 years of relevant experience

Required Experience

• Hands on experience designing building and maintaining ETLELT data pipelines and integrating multiple disparate data sources
• Experience with data warehouse andor data lake technologies such as AWS Redshift Amazon S3 Hadoop Snowflake or similar platforms
• Experience with Elastic Stack including Elasticsearch Logstash and Kibana andor Kafka for data ingestion and processing
• Experience with LinuxUNIX system administration and automation
• Experience with CICD pipelines containerized pipelines version control technologies such as Git and Bitbucket and DevOps practices
• Familiarity with network operations data including logs metrics events ticketing CMDB data and related data models
• Proficiency with data engineering technologies and languages such as Python Java SQL and shell scripting
• Knowledge of data security access controls and compliance frameworks including DoD DISA RMF and STIG requirements
• Strong communication and documentation skills supporting collaboration and knowledge sharing
• CertificationsCurrent Security certification or higherequivalent certification

Clearance

Active TSSCI security clearance

Other Requirements

Willingness and ability to complete and maintain additional customer suitability screenings throughout employment

Preferred Qualifications

Experience supporting DISA DISN or other DoD network environmentsExperience developing and deploying software applications that meet DoD security standards including applicable STIGsKnowledge of network security encryption and access controls within classified environmentsExperience with big data technologies and platforms such as Kafka Spark NiFi Hadoop Elasticsearch Logstash Databricks and ELK StackExperience applying big data technologies to text mining summarization search and entity extractionExperience with cloud based data platforms including AWS Azure GCP or AWS GovCloudExperience with cloud integrated platforms and Infrastructure as Code including networking and security policiesFamiliarity with Kubernetes deployments platform upgrades and patchingExperience with configuration management technologies such as Ansible Puppet or ChefExperience with monitoring alerting and observability technologies such as Prometheus Grafana and ElasticExperience with business intelligence and visualization tools such as KibanaExperience developing and maintaining technical documentation and operational runbooksStrong understanding and practical experience with Agile methodologies including Scrum and SAFeExperience with Atlassian Jira and ConfluenceConfluent Developer certificationElastic Certified Engineer certificationExperience working remotely with geographically dispersed teams and within matrixed organizationsExperience combining software integration and data engineering practicesExperience applying knowledge of system architecture networking and centralized logging using ELK to support data transformation initiatives