Cloud Data Engineering Lead — Modernization & AI POCs

Ledgent Technology; United States; Remote; USD 125,000 - 150,000; Full time

Job summary

Ledgent Technology is seeking a Data Engineering Technical Lead to spearhead the modernization of legacy data systems into a cloud-native Lakehouse. The role focuses on designing scalable data pipelines, defining data patterns, and enforcing engineering standards to enable analytics, AI, and BI across the organization.

The ideal candidate has 8+ years in data engineering with 3+ in senior/lead roles, deep expertise in Databricks, Spark, dbt, Fivetran, and Airflow, and strong SQL with

Qualifications

• 8+ years of experience in data engineering or related technical fields, with at least 3+ years in a lead or senior role.
• Proven experience designing and implementing data design patterns (e.g., CDC, SCD, Medallion, Data Vault, streaming, and batch patterns).
• Deep expertise with Databricks, Apache Spark, dbt, Fivetran, Census, Airflow, and Kafka. Solid experience across Azure and/or GCP (e.g., Synapse, Data Factory, BigQuery, Pub/Sub).
• Hands-on experience modernizing legacy ETL (SSIS/SSRS) workloads into cloud-native pipelines.
• Demonstrated ability to build POCs and POVs that validate new tools, frameworks, or architectures.
• Working knowledge of AI-assisted engineering tools for development, observability, or optimization.
• Proficiency in SQL and one programming language (Python, Scala, or Java).
• Strong problem-solving, architectural thinking, and collaboration skills.

Responsibilities

• Design and deliver scalable data pipelines for cloud-native Lakehouse environments.
• Define data design patterns and enforce engineering standards across teams.
• Drive proofs of concept (POCs) and POVs to evaluate emerging technologies.
• Collaborate with analytics, BI, and AI initiatives to ensure scalable, cost-efficient platforms.
• Leverage AI-assisted tooling to accelerate modernization and reduce technical debt.

Skills

Databricks

Apache Spark

dbt

Fivetran

Census

Airflow

Kafka

SQL

Python

Scala/Java

Tools

Azure

GCP

Synapse

Data Factory

BigQuery

Pub/Sub

Job description

Ledgent Technology is seeking a Data Engineering Technical Lead to spearhead the modernization of legacy data systems into a cloud-native Lakehouse. The role focuses on designing scalable data pipelines, defining data patterns, and enforcing engineering standards to enable analytics, AI, and BI across the organization.

The ideal candidate has 8+ years in data engineering with 3+ in senior/lead roles, deep expertise in Databricks, Spark, dbt, Fivetran, and Airflow, and strong SQL with