Data Engineer Ab initio

Robotics Technologies

Houston (TX)

On-site

USD 89,000 - 126,000

Full time

11 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Robotics Technologies in Houston, TX seeks a Data Engineer Ab Initio to design high-performance data pipelines and ensure low-latency processing with exact-once semantics. The role focuses on continuous flows, checkpointing, and robust monitoring across big data platforms.

You will build and optimize Ab Initio graphs, work with Kafka-based event streams, and collaborate with data scientists to deliver reliable data solutions for batch and real-time ingestion.

Qualifications

  • Deep understanding of credit card process system.
  • Deep knowledge of GDE, EME, Conduct-It, and Continuous Flows.
  • Familiarity with Kafka, queues, and real-time stateful services.
  • Proficiency in Unix/Linux shell scripting, SQL, Oracle, and Teradata.

Responsibilities

  • Build and maintain Ab Initio continuous flow graphs for batch and real-time ingestion and transformation.
  • Deliver exactly-once processing by combining orchestration with checkpointing and recovery.
  • Monitor live production streams for 24/7 reliability and troubleshoot data issues.
  • Develop ETL pipelines for both batch and real-time ingestion and transformation.
  • Apply data validation, security, integrity, and compliance across big data platforms.
  • Collaborate with data scientists and analysts to deliver data solutions.

Skills

Ab Initio
Kafka
Unix/Linux shell scripting
SQL
Java / Scala / Python
CI/CD

Tools

Docker
Kubernetes
GDE / EME / Conduct-It / Continuous Flows
Oracle
Teradata
Apache Kafka

Job description

The Data Engineer Ab initio role in Houston, TX focuses on designing and implementing high-performance, event-driven Ab Initio data pipelines. The position emphasizes low-latency processing, high availability, and production reliability through continuous-flow resilience, checkpointing, and active monitoring.

Role Overview

Build and maintain Ab Initio continuous flow graphs for both batch and real-time ingestion and transformation. Deliver exactly-once processing behavior by combining continuous flow orchestration with checkpointing and recovery, while ensuring data quality, security, integrity, and compliance across big data platforms.

Key Responsibilities
  • Create complex Ab Initio continuous flow graphs, including stateful joins, sliding time windows, and aggregations.
  • Implement event-driven pipelines using Kafka, MQ, and file streams.
  • Ensure resilience of continuous flows through checkpointing and recovery to support exactly-once processing.
  • Apply advanced Ab Initio components such as Reformat, Rollup, Join, and Partition to achieve low-latency performance.
  • Monitor live production streams for 24/7 reliability, and troubleshoot data issues proactively.
  • Develop ETL pipelines for batch and real-time ingestion and transformation.
  • Implement and enforce data validation, data security, integrity, and compliance across big data platforms.
  • Monitor and troubleshoot performance issues in large-scale clusters.
  • Collaborate with data scientists, analysts, and application teams to deliver high-quality data solutions.
  • Automate workflows and improve operational efficiency using scripting and orchestration tools.
Required Qualifications
  • Deep understanding of credit card process system.
  • Deep knowledge of GDE (Graphical Development Environment), EME (Enterprise Meta Environment), Conduct-It, and Continuous Flows.
  • Understanding of Kafka, message queues, and real-time stateful services.
  • Proficiency in Unix/Linux shell scripting, SQL, and database technologies such as Oracle and Teradata.
  • Experience with Java, Scala, Python, or Kafka is a plus.
  • Familiarity with Linux/Unix environments and shell scripting.
  • Understanding of data security, governance, and compliance standards.
  • Experience with cloud-based big data platforms.
  • Exposure to containerization tools such as Docker and Kubernetes for big data workloads.
  • Knowledge of CI/CD pipelines for data engineering projects.
Technology Stack
  • Ab Initio; GDE (Graphical Development Environment); EME (Enterprise Meta Environment); Conduct-It; Continuous Flows
  • Kafka; MQ; Reformat; Rollup; Join; Partition; ETL
  • Unix/Linux shell scripting; SQL; Oracle; Teradata
  • Java; Scala; Python; Linux/Unix
  • Docker; Kubernetes
  • CI/CD; checkpointing; recovery
Location, Duration, and Employment Details
  • Job ID: J54188
  • Location: Houston, TX (onsite)
  • Duration: 8 Months + Extension
  • Salary: USD 78 - 78 per hourly
  • Employment Type: W-2, 1099, C2C
  • Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other
  • Client: To Be Discussed Later
  • Domain Exposure: Material
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer Ab initio
Data Engineer Ab initio

Ethereum Technologies LLC • Houston (TX)

On-site
USD 89,000 - 126,000
Data Engineer Ab initio
Data Engineer Ab initio

Quantum Technologies LLC • Houston (TX)

On-site
USD 89,000 - 126,000
Data Engineer Ab initio
Data Engineer Ab initio

Valiant Technologies LLC • Houston (TX)

On-site
USD 89,000 - 126,000
Real-Time Data Engineer: Ab Initio & Kafka Pipelines
Real-Time Data Engineer: Ab Initio & Kafka Pipelines

Compunnel Inc. • Berkeley Heights (NJ)

On-site
USD 120,000 - 160,000
Abintio Data Engineer
Abintio Data Engineer

Envision Technology Solutions • Berkeley Heights (NJ)

On-site
USD 120,000 - 170,000
Data Engineer (Ab Initio, OzoneCH, Flink)
Data Engineer (Ab Initio, OzoneCH, Flink)

Compunnel Inc. • Berkeley Heights (NJ)

On-site
USD 120,000 - 160,000
Ab Initio Consultant
Ab Initio Consultant

Mphasis • Berkeley Heights (NJ)

On-site
USD 140,000 - 190,000
Ab Initio Developer
Ab Initio Developer

7Th Sky Techologies llc • Berkeley (CA)

Hybrid
USD 120,000 - 180,000
Ab Initio Developer
Ab Initio Developer

7Th Sky Techologies llc • Charlotte (NC)

Hybrid
USD 100,000 - 140,000
Ab Initio Data Engineer — Real-Time Pipelines
Ab Initio Data Engineer — Real-Time Pipelines

Quantum Technologies. LLC • Houston (TX)

On-site
USD 89,000 - 126,000