Data Engineer - Real-Time Pipelines & Microservices

MACHINE LEARNING TECHNOLOGIES LLC; Dallas (TX); On-site; USD 84,000 - 120,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC seeks a Data Engineer in Dallas for a 10-month engagement with possible extension. The role requires building near real-time data pipelines using Spark/PySpark, Python, Kafka, and Cassandra, and developing robust Java-based services and APIs.

You will work in an Agile environment, design scalable data architectures, and ensure code quality with tests and logging. On-site work with flexible contract options (W-2, 1099, C2C) and U.S. work authorization.

Qualifications

• Ability to translate application storyboards and use cases into functional applications.
• Build near real time data pipeline with Apache Spark| Pyspark & Python to extract data from NoSQL DBs and Load it into Cassandra Data Enterprise Graph.
• Build real time data pipeline with Apache Flink|<br/>Apache Kafka and Python to extract data from Relational and NoSQL DBs and Load it into Cassandra Data Enterprise Graph.
• Design| build| and maintain efficient| reusable| and reliable Java code
• Ensure the best possible performance| quality| and responsiveness of the applications
• Help maintain code quality| organization| and automatization
• Strong debugging| problem solving and investigative skills.
• Ability to assimilate disparate information (log files| error messages etc.) and pursue leads to find root cause of problems
• Experience in Agile/Scrum methodology
• Proven ability to create innovative solutions.
• Strong agile/scrum development experience
• Expertise developing web applications and APIs using Micro Services| with demonstrable production-scale experience.
• Web applications development using Java| HTML5| CSS3| JavaScript| jQuery| Angular JS| Spring Boot.
• Skilled n Version control tools e.g.,| GIT| SVN| etc. to maintain common source code the entire process of workflow.
• Writing unit / Integration tests using JUnit / Mockito for code coverage as per the standards and Log4j 2 for logging.

Skills

Apache Spark

Python

Apache Kafka

Java

REST APIs

Spring Boot

JavaScript

HTML5

CSS3

jQuery

AngularJS

Git

SVN

JUnit

Mockito

Log4j2

NoSQL

Relational DBs

Tools

Apache Flink

Cassandra

Job description

MACHINE LEARNING TECHNOLOGIES LLC seeks a Data Engineer in Dallas for a 10-month engagement with possible extension. The role requires building near real-time data pipelines using Spark/PySpark, Python, Kafka, and Cassandra, and developing robust Java-based services and APIs.

You will work in an Agile environment, design scalable data architectures, and ensure code quality with tests and logging. On-site work with flexible contract options (W-2, 1099, C2C) and U.S. work authorization.