Data Engineer: Scalable Pipelines & AI Automation

LightBox Employer; Shelton (CT); On-site; USD 90,000 - 130,000; Full time

Job summary

LightBox is seeking a Data Engineer to join the Data Integration team. You will design, build, and maintain reliable data pipelines and data-intensive applications that support LightBox products.

This hands-on, mid-career role involves solving practical data problems with ownership and collaboration from the team. You will work with large datasets, transform data for product features, and partner with engineering, product, and data stakeholders to deliver trusted, scalable data solutions.

Qualifications

• BS degree in CS/Engineering or equivalent experience.
• 2+ years building and maintaining production ETLs.
• Strong SQL, Python, and at least one scalable data framework (Spark, DuckDB, Polars).
• Experience with batch and/or event-driven pipelines and orchestration tools.
• Familiarity with AWS or GCP and modern data formats (Parquet/Delta Lake).
• GitHub workflows, PRs, CI/CD, and code reviews.

Responsibilities

• Build, test, and maintain batch and event-driven data pipelines and ETL/ELT workflows.
• Transform and organize data for product features and downstream users.
• Investigate data-quality issues and implement fixes.
• Improve workflows for reliability, efficiency, and maintainability.
• Collaborate with engineering, product, and data stakeholders to deliver solutions.
• Adhere to version control, testing, monitoring, and documentation practices.

Skills

SQL skills

Python

Data pipelines

ETL/ELT design

Team collaboration

Communication

Problem solving

Education

Bachelor's degree in Computer Science, Engineering, or a related field

Tools

Spark

DuckDB

Polars

Prefect

Airflow

Dagster

GitHub Actions

AWS

GCP

Parquet

Delta Lake

Job description

LightBox is seeking a Data Engineer to join the Data Integration team. You will design, build, and maintain reliable data pipelines and data-intensive applications that support LightBox products.

This hands-on, mid-career role involves solving practical data problems with ownership and collaboration from the team. You will work with large datasets, transform data for product features, and partner with engineering, product, and data stakeholders to deliver trusted, scalable data solutions.