DataOps Engineer: Iceberg Lakehouse Platform Lead

Woongjin, Inc

Englewood Cliffs (NJ)

On-site

USD 99,000 - 121,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Woongjin, Inc. is seeking a mid‑level engineer to build and operate a Docker‑based data platform using Apache Iceberg as the lake‑house format. You will own end‑to‑end delivery pipelines, monitoring, security, and incident response to ensure reliable scale.

The role covers Iceberg operations, Docker‑based image creation and testing, ETL/ELT pipeline development, and CI/CD automation. You’ll work with Spark/Flink/Presto, Python/Ansible, and observability tools while collaborating with security

Qualifications

Bachelor’s degree in Computer Science, IT, Data Engineering, or related field.
5+ years of hands‑on experience building and operating large‑scale data platforms (lake‑house, data‑warehouse, or big‑data ecosystems).
Proven production experience with Apache Iceberg (table creation, partition management, schema evolution, catalog integration).
Strong Docker skills: multi‑stage builds, Docker‑Compose testing, routine image security scanning.
Experience with at least one major data‑processing engine (Spark, Flink, or Presto/Trino) and its connection to Iceberg tables.
Proficiency in Python and/or Ansible for automating infrastructure and platform tasks.
Experience building CI/CD pipelines that include Docker linting, vulnerability scanning, and automated deployment of data‑pipeline code.
Familiarity with observability tooling (Prometheus, Grafana, OpenTelemetry, Loki) and ability to create useful alerts and dashboards.
Ability to respond to incidents, write clear root‑cause analysis reports, and contribute to post‑mortem actions.
Willingness to participate in an on‑call rotation as a first‑line responder.
Availability to work on‑site in New Jersey for the initial assignment and relocate to Dallas by October 2026.

Responsibilities

Iceberg operations: support tables, manage schema changes, partitions, snapshot retention, and keep the catalog synchronized.
Docker image creation & testing: write multi‑stage Dockerfiles for Spark/Flink/Presto, run local test environments with Docker‑Compose, and conduct vulnerability scans.
Data pipeline development: build ETL/ELT jobs that ingest raw data and write to Iceberg tables; add simple streaming components using Kafka, Pulsar, or Kinesis when needed.
CI/CD automation: configure pipelines to lint Dockerfiles, scan images, version Iceberg metadata, and deploy pipelines without downtime.
Automation with Ansible/Python: script cluster provisioning, catalog configuration, vacuum/compaction, and other routine housekeeping tasks.
Observability: instrument services with OpenTelemetry, Prometheus, Grafana, and Loki; create dashboards showing pipeline latency, resource usage, table health, and error rates; set up basic alerts.
SLA monitoring: measure data freshness, job success rates, and query response times against targets and report deviations.
Incident response: join on‑call rotation, diagnose and resolve pipeline failures or Iceberg metadata issues; write root‑cause analyses and suggest improvements.
Security & compliance support: help enforce image signing, mTLS, IAM roles, and bucket policies; coordinate with security for GDPR, HIPAA, or ISO 27001 requirements.
Knowledge sharing: keep internal docs up to date and run short demos on Iceberg, Docker, and automation techniques.