Remote AI-First Senior Data Engineer - Databricks

Echo Global Logistics; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Bonus eligibility

Job summary

Echo Global Logistics seeks a Senior Data Engineer to design and deliver scalable data pipelines on an Azure-based platform powered by Databricks. You will lead Lakehouse architecture and build production-grade data products driving analytics and AI initiatives across the business.

You will mentor engineers, enforce data quality and CI/CD, and collaborate with analytics engineers, data scientists, and application teams to validate solutions and adopt emerging technologies.

Qualifications

• AI-first development mindset and data engineering leadership.
• 5+ years building production-grade data pipelines.
• Strong Python and SQL skills with Databricks/Spark experience.
• Lakehouse architecture and Medallion modeling knowledge.
• Experience with data quality, testing, monitoring, and CI/CD.
• REST APIs and enterprise system integrations with Git workflows.
• Bachelor's degree in a relevant field and excellent communication.

Responsibilities

• Lead design and development of scalable data pipelines in Databricks.
• Build and optimize Lakehouse architecture using Bronze/Silver/Gold.
• Develop data solutions with Python, PySpark, Spark SQL, Delta Lake.
• Integrate APIs, enterprise systems, SaaS platforms, and external data.
• Establish data quality, testing, monitoring, and performance standards.
• Mentor engineers and promote engineering best practices.
• Drive CI/CD, automation, and AI-first development practices.
• Partner with analytics engineers, data scientists, and app teams.
• Evaluate new technologies via proof-of-concept projects.

Skills

AI-first development mindset

5+ years data engineering

Python

SQL

Databricks

Apache Spark

PySpark

Production data pipelines

Lakehouse Medallion modeling

Data quality and testing

CI/CD and Git

Azure Data Factory / Airflow

REST APIs & enterprise integration

Communication & collaboration

Education

Bachelor's degree in Computer Science, Engineering, Information Systems, or equivalent

Tools

Databricks

Apache Spark

PySpark

Azure Data Factory

Airflow

Job description

Echo Global Logistics seeks a Senior Data Engineer to design and deliver scalable data pipelines on an Azure-based platform powered by Databricks. You will lead Lakehouse architecture and build production-grade data products driving analytics and AI initiatives across the business.

You will mentor engineers, enforce data quality and CI/CD, and collaborate with analytics engineers, data scientists, and application teams to validate solutions and adopt emerging technologies.