Data Engineer: Build Scalable Pipelines with NiFi & Accumulo

SandTech Solutions; San Antonio (TX); On-site; USD 110,000 - 150,000; Full time

Job summary

SandTech Solutions is seeking a data engineer to design, build, and maintain robust data pipelines for complex environments. The role focuses on NiFi-based data ingestion, RESTful API development, and scalable storage using Hadoop, Accumulo, and Elasticsearch.

You will collaborate with cross-functional teams to ensure data quality, security, and compliance across the data lifecycle. Ideal candidates will have strong programming skills (Python/Java/Scala) and a proven track record in data

Qualifications

• Bachelor's degree in CS/IT or related field.
• Proven experience as a Data Engineer with data integration and processing.
• Strong Apache NiFi proficiency for data ingestion and transformation.
• Experience developing RESTful APIs.
• In-depth knowledge of Hadoop, Accumulo, and Elasticsearch.
• Proficiency in Python, Java, or Scala.
• Understanding of data security and compliance requirements.
• Excellent problem-solving and collaboration skills.

Responsibilities

• Design, develop, and maintain data pipelines to support C2IE deployments.
• Utilize Apache NiFi for data ingestion, transformation, and routing.
• Develop and maintain RESTful APIs for data exchange between systems.
• Implement and manage data storage solutions, including Hadoop, Accumulo, and Elasticsearch.
• Collaborate with cross-functional teams to understand data requirements and implement solutions.
• Ensure data quality, integrity, and security throughout the data lifecycle.
• Monitor and optimize data workflows for performance and reliability.
• Troubleshoot and resolve issues related to data integration and processing.
• Stay updated with trends in data engineering and apply improvements.

Skills

Apache NiFi

RESTful APIs

Python

Java

Scala

Data security

Data processing

Data integration

Collaboration

Education

Bachelor's degree in Computer Science / IT or related field

Tools

Elasticsearch

Hadoop

Accumulo

Elastic Stack

Docker

Kubernetes

Jenkins

Job description

SandTech Solutions is seeking a data engineer to design, build, and maintain robust data pipelines for complex environments. The role focuses on NiFi-based data ingestion, RESTful API development, and scalable storage using Hadoop, Accumulo, and Elasticsearch.

You will collaborate with cross-functional teams to ensure data quality, security, and compliance across the data lifecycle. Ideal candidates will have strong programming skills (Python/Java/Scala) and a proven track record in data