Sr. Data Engineer

Echo Global Logistics; United States; Remote; USD 120,000 - 180,000; Full time

Benefits offered by this job

Bonus eligibility

Job summary

Echo Global Logistics seeks a Senior Data Engineer to design and deliver scalable data pipelines on an Azure-based platform powered by Databricks. You will lead Lakehouse architecture and build production-grade data products driving analytics and AI initiatives across the business.

You will mentor engineers, enforce data quality and CI/CD, and collaborate with analytics engineers, data scientists, and application teams to validate solutions and adopt emerging technologies.

Qualifications

• AI-first development mindset and data engineering leadership.
• 5+ years building production-grade data pipelines.
• Strong Python and SQL skills with Databricks/Spark experience.
• Lakehouse architecture and Medallion modeling knowledge.
• Experience with data quality, testing, monitoring, and CI/CD.
• REST APIs and enterprise system integrations with Git workflows.
• Bachelor's degree in a relevant field and excellent communication.

Responsibilities

• Lead design and development of scalable data pipelines in Databricks.
• Build and optimize Lakehouse architecture using Bronze/Silver/Gold.
• Develop data solutions with Python, PySpark, Spark SQL, Delta Lake.
• Integrate APIs, enterprise systems, SaaS platforms, and external data.
• Establish data quality, testing, monitoring, and performance standards.
• Mentor engineers and promote engineering best practices.
• Drive CI/CD, automation, and AI-first development practices.
• Partner with analytics engineers, data scientists, and app teams.
• Evaluate new technologies via proof-of-concept projects.

Skills

AI-first development mindset

5+ years data engineering

Python

SQL

Databricks

Apache Spark

PySpark

Production data pipelines

Lakehouse Medallion modeling

Data quality and testing

CI/CD and Git

Azure Data Factory / Airflow

REST APIs & enterprise integration

Communication & collaboration

Education

Bachelor's degree in Computer Science, Engineering, Information Systems, or equivalent

Tools

Databricks

Apache Spark

PySpark

Azure Data Factory

Airflow

Job description

This is an AI-first engineering role. We expect you to leverage AI tools to design, build, and deliver solutions - the value you bring is knowing what to build, how to solve problems, and how to validate results.
As our Senior Data Engineer, you’ll lead the design and development of scalable data pipelines and data products on our Azure-based platform, with Databricks at the core.
You’ll build trusted, production-grade data that powers analytics, operational intelligence, and AI initiatives across the business.

What You’ll Do

• · Lead the design and development of scalable data pipelines in Databricks
• · Build and optimize Lakehouse architecture using Medallion (Bronze/Silver/Gold) design
• · Develop data solutions using Python, Pyspark, Spark SQL, and Delta Lake
• · Integrate APIs, enterprise systems, SaaS platforms, and external data sources
• · Establish data quality, testing, monitoring, and performance standards
• · Mentor engineers and promote engineering best practices
• · Drive CI/CD, automation, and AI-first development practices
• · Partner with analytics engineers, data scientists, and application teams
• · Evaluate new technologies through proof-of-concept projects

What You Bring

Required

• · AI-first development mindset
• · 5+ years of data engineering experience, including senior or lead responsibilities
• · Expert Python and SQL skills
• · Deep experience with Databricks, Apache Spark, and Pyspark
• · Experience building production data pipelines
• · Lakehouse architecture and Medallion modeling experience
• · Data quality, testing, and monitoring expertise
• · CI/CD and Git experience
• · Experience with Azure Data Factory, Airflow, or similar orchestration tools
• · REST API and enterprise system integrations
• · Bachelor’s degree in computer science, Engineering, Information Systems, or equivalent experience
• · Strong communication and collaboration skills

Preferred

• · Azure Databricks
• · Azure Data Factory
• · Azure Data Lake Storage
• · Delta Lake
• · Apache Airflow
• · Databricks Asset Bundles
• · Databricks Apps
• · MLflow, Mosaic AI, Unity Catalog
• · Experience supporting AI/ML workloads
• · Databricks Data Engineer Certification

Why You’ll Love Working Here

You’ll join a high-performing team that’s investing heavily in cloud, data, and AI. We empower our engineers with modern tools, continuous learning, and opportunities to influence technology strategy while solving meaningful business challenges.

Work environment/physical demands summary:

This job operates in an office environment and uses a computer, telephone and other office equipment as needed to perform duties. The noise level in the work environment is typical of that of an office with an open seating floor plan. The employee may encounter frequent interruptions throughout the work day. The employee is regularly required to sit, talk, or hear.

All qualified applicants will receive consideration for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity, national origin, status as a qualified individual with a disability, or Vietnam era or other protected veteran.

#Remote

Benefits

For more information about our benefit offerings, please visit our careers page at https://www.echo.com/company/careers.

This role is eligible for a bonus that is based on a combination of personal and business performance.