Data Engineer w/ Accumulo (MUST have DoD Clearance)

SandTech Solutions; San Antonio (TX); On-site; USD 110,000 - 150,000; Full time

Job summary

SandTech Solutions is seeking a data engineer to design, build, and maintain robust data pipelines for complex environments. The role focuses on NiFi-based data ingestion, RESTful API development, and scalable storage using Hadoop, Accumulo, and Elasticsearch.

You will collaborate with cross-functional teams to ensure data quality, security, and compliance across the data lifecycle. Ideal candidates will have strong programming skills (Python/Java/Scala) and a proven track record in data

Qualifications

• Bachelor's degree in CS/IT or related field.
• Proven experience as a Data Engineer with data integration and processing.
• Strong Apache NiFi proficiency for data ingestion and transformation.
• Experience developing RESTful APIs.
• In-depth knowledge of Hadoop, Accumulo, and Elasticsearch.
• Proficiency in Python, Java, or Scala.
• Understanding of data security and compliance requirements.
• Excellent problem-solving and collaboration skills.

Responsibilities

• Design, develop, and maintain data pipelines to support C2IE deployments.
• Utilize Apache NiFi for data ingestion, transformation, and routing.
• Develop and maintain RESTful APIs for data exchange between systems.
• Implement and manage data storage solutions, including Hadoop, Accumulo, and Elasticsearch.
• Collaborate with cross-functional teams to understand data requirements and implement solutions.
• Ensure data quality, integrity, and security throughout the data lifecycle.
• Monitor and optimize data workflows for performance and reliability.
• Troubleshoot and resolve issues related to data integration and processing.
• Stay updated with trends in data engineering and apply improvements.

Skills

Apache NiFi

RESTful APIs

Python

Java

Scala

Data security

Data processing

Data integration

Collaboration

Education

Bachelor's degree in Computer Science / IT or related field

Tools

Elasticsearch

Hadoop

Accumulo

Elastic Stack

Docker

Kubernetes

Jenkins

Job description

The ideal candidate will have extensive experience with data integration, processing, and management, specifically utilizing Apache Nifi, RESTful APIs, Hadoop, Accumulo, and Elasticsearch.

Key Responsibilities

• Design, develop, and maintain data pipelines to support C2IE deployments.
• Utilize Apache NiFi for data ingestion, transformation, and routing.
• Develop and maintain RESTful APIs for seamless data exchange between systems.
• Implement and manage data storage solutions, including Hadoop, Accumulo, and Elasticsearch.
• Collaborate with cross-functional teams to understand data requirements and implement effective solutions.
• Ensure data quality, integrity, and security throughout the data lifecycle.
• Monitor and optimize data workflows to ensure high performance and reliability.
• Troubleshoot and resolve issues related to data integration and processing.
• Stay updated with the latest trends and technologies in data engineering and apply them to improve existing processes.

Required Skills And Qualifications

• Bachelor's degree in Computer Science, Information Technology, or related field.
• Proven experience as a Data Engineer or similar role, with a focus on supporting complex data environments.
• Strong proficiency in Apache NiFi for data integration and processing.
• Experience in developing and managing RESTful APIs.
• In-depth knowledge of data storage solutions, including Hadoop, Accumulo, and Elasticsearch.
• Proficiency in programming languages such as Python, Java, or Scala.
• Strong understanding of data security and compliance requirements.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.

Preferred Qualifications

• Experience with Elastic Search and the Elastic Stack
• Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) for data engineering.
• Knowledge of DevOps practices and tools like Jenkins, Docker, and Kubernetes.
• Experience with data visualization tools and techniques.
• Relevant certifications in data engineering or related fields.