Energy AI Data Engineer for Real-Time Lakehouse

Applied Computing; Houston (TX); On-site; USD 150,000 - 190,000; Full time

Job summary

Applied Computing, founded in 2024, builds Orbital, a physics-informed foundation model for energy operations. We seek a Data Engineer to architect pipelines across AWS and Databricks, turning high-frequency and historian data into scalable Lakehouse architectures for deep learning and real-time LLM workloads.

You’ll work with PostgreSQL, Python, and PySpark, ensuring data is contextualised, synchronised, and optimised for AI and real-time analytics in energy environments.

Qualifications

• Experience building scalable data pipelines for time-series data.
• Strong Python development and SQL optimization experience.
• Hands-on with AWS data services and Databricks/PySpark.

Responsibilities

• Ingest and contextualise time-series and historian data for AI pipelines.
• Build real-time and batch data movement into a Lakehouse.
• Coordinate Databricks Lakeflow/Connectors for data integration.
• Ensure secure, high-throughput data transfers in AWS.
• Track schema changes and data lineage across pipelines.
• Prepare data for AI training and real-time inference workloads.
• Tune PostgreSQL and Spark for large-scale time-series workloads.
• Deploy pipelines on AWS EKS with persistent storage.

Skills

Python

PostgreSQL

Time-series data

Data pipelines

MLOps concepts

Streaming basics

Education

Bachelor's degree in CS or related

Tools

AWS

Databricks

PySpark

Kafka

Flink

Spark Streaming

OPC UA

DCS/SCADA logs

EKS

S3

KMS

Job description

Applied Computing, founded in 2024, builds Orbital, a physics-informed foundation model for energy operations. We seek a Data Engineer to architect pipelines across AWS and Databricks, turning high-frequency and historian data into scalable Lakehouse architectures for deep learning and real-time LLM workloads.

You’ll work with PostgreSQL, Python, and PySpark, ensuring data is contextualised, synchronised, and optimised for AI and real-time analytics in energy environments.