Lead Data Engineer

BCforward; Columbus (OH); On-site; USD 140,000 - 190,000; Full time

Job summary

BCforward in Columbus, OH is seeking a hands-on Lead Data Engineer to architect and deliver large-scale data pipelines and to lead AI-assisted engineering adoption.

You will design ETL/ELT pipelines, build Spark workloads, develop Java and Python services, and guide CI/CD, testing, observability, and security practices.

Qualifications

• 10+ years in software engineering with deep data engineering experience.
• Strong Java and Python proficiency.
• Experience with AWS.
• Hands-on experience leading AI-assisted development tools.

Responsibilities

• Lead design and delivery of large-scale ETL/ELT pipelines and data lake solutions.
• Build and tune distributed workloads in Apache Spark (Parquet, Avro).
• Develop Java-based services and REST APIs that support data workflows.
• Drive CI/CD, automated testing, observability and secure-by-design practices.
• Coach engineers on safe, effective use of AI-assisted development tools.

Skills

Java

Python

AWS

AI-assisted development tools

Tools

Snowflake

Airflow

Dagster

Control-M

Kafka

Spark Structured Streaming

Delta Lake

Iceberg

Hudi

Job description

Presently we are unable to sponsor and request applicants to apply who are authorized to work without sponsorship

We’re looking for a hands-on Lead Data Engineer to architect and deliver large-scale data pipelines and to lead adoption of AI-assisted engineering practices on the team.

What you’ll do

• Lead design and delivery of large-scale ETL/ELT pipelines and data lake solutions
• Build and tune distributed workloads in Apache Spark (Parquet, Avro)
• Develop Java-based services and REST APIs that support data workflows
• Drive CI/CD, automated testing, observability and secure-by-design practices
• Coach engineers on safe, effective use of AI-assisted development tools

Must-haves

• 10+ years in software engineering, with deep data engineering experience
• Strong Java and Python
• AWS
• Hands-on experience leading AI-assisted development tools

Nice to have

• Snowflake, Airflow/Dagster/Control-M, Kafka or Spark Structured Streaming
• Delta Lake, Iceberg or Hudi
• Regulated-environment experience (PII, audit, compliance)