Realtime Data Engineer | Spark, Kafka & Java

Valiant Technologies LLC; Dallas (TX); On-site; USD 84,000 - 120,000; Full time

Job summary

Valiant Technologies LLC in Dallas, TX seeks an experienced Data Engineer to design and implement real-time data pipelines. You will work with Spark, PySpark, Python, Kafka and Cassandra to extract data from multiple sources and load into our data platforms. Strong Java + web tech skills (HTML5/CSS3/JS) required for APIs and microservices.

You will contribute to Agile/Scrum processes, ensure code quality, and help automate testing and deployment in a fast-paced environment.

Qualifications

• Ability to translate application stories and use cases into functional applications.
• Experience building near real-time data pipelines with Spark/PySpark and Python.
• Experience extracting data from NoSQL/Relational DBs and loading into Cassandra.
• Strong Java development with web technologies (HTML5/CSS3/JS).

Responsibilities

• Translate stories and use cases into scalable data solutions.
• Build near real-time data pipelines with Apache Spark, PySpark, and Python.
• Design, build, and maintain Java code for data processing and APIs.
• Ensure performance, quality and responsiveness of applications.
• Maintain code quality, organization and automation; write unit/integration tests.

Skills

Apache Spark

PySpark

Python

NoSQL

Cassandra

Apache Kafka

Java

HTML5

CSS3

JavaScript

jQuery

AngularJS

Spring Boot

Git

SVN

JUnit

Mockito

Log4j 2

Agile/Scrum

Tools

Apache Spark

PySpark

Apache Flink

Apache Kafka

Java

HTML5

CSS3

JavaScript

jQuery

AngularJS

Spring Boot

Git

SVN

JUnit

Mockito

Log4j 2

Job description

Valiant Technologies LLC in Dallas, TX seeks an experienced Data Engineer to design and implement real-time data pipelines. You will work with Spark, PySpark, Python, Kafka and Cassandra to extract data from multiple sources and load into our data platforms. Strong Java + web tech skills (HTML5/CSS3/JS) required for APIs and microservices.

You will contribute to Agile/Scrum processes, ensure code quality, and help automate testing and deployment in a fast-paced environment.