Energy AI Data Engineer - Real-Time Lakehouse

Applied Computing; Houston (TX); On-site; USD 120,000 - 160,000; Full time

Job summary

Applied Computing in Houston seeks a Data Engineer to architect and maintain high-frequency time-series and historian data pipelines, enabling AI-ready data for deep learning models and real-time LLM workloads.

You will work across AWS (EKS, S3, EBS, IAM, KMS, CloudWatch) and Databricks/PySpark, ensuring data is contextualised, synchronised, and optimised for AI applications in energy operations.

Qualifications

• Strong Python for data processing, scripting, and orchestration.
• Expertise in PostgreSQL (partitioning, indexing, performance).
• Extensive AWS experience for data pipelines (EKS, S3, IAM, KMS).
• Experience with Databricks and PySpark for large-scale processing.
• Familiarity with time-series industrial data (DCS/SCADA logs, historians).
• Knowledge of streaming frameworks (Kafka, Flink) or MLOps data versioning is a plus.

Responsibilities

• Ingest, contextualise, and move data from diverse sources into Lakehouse.
• Build and optimise real-time and batch pipelines for AI workloads.
• Maintain data lineage, governance, and schema drift controls.
• Ensure performance tuning for PostgreSQL and Spark-based workloads.
• Collaborate across teams to align data availability with ML/LLM needs.

Skills

Python

PostgreSQL

AWS

Databricks

PySpark

Time-series data

Tools

Databricks

PySpark

AWS

Kubernetes

PostgreSQL

Job description

Applied Computing in Houston seeks a Data Engineer to architect and maintain high-frequency time-series and historian data pipelines, enabling AI-ready data for deep learning models and real-time LLM workloads.

You will work across AWS (EKS, S3, EBS, IAM, KMS, CloudWatch) and Databricks/PySpark, ensuring data is contextualised, synchronised, and optimised for AI applications in energy operations.