Principal Data Engineer

Eucloid Data Solutions; Philadelphia (Philadelphia County); On-site; USD 150,000 - 210,000; Full time

Job summary

Eucloid Data Solutions Pvt Ltd seeks an accomplished Data Engineer with Databricks expertise to build and optimize scalable data platforms in the USA. You will lead Databricks projects, design high‑performance ETL pipelines, and drive governance, security, and cost optimization across cloud data platforms.

You will collaborate with cross‑functional teams including data scientists, analysts, and reports engineers to deliver production-ready data products and analytics capabilities for clients.

Qualifications

• Bachelor's or Master's in a quantitative field.
• 10+ years of data engineering and cloud data platforms.
• 4+ years on Databricks including architecture and pipelines.
• Proficient in Python, SQL, and PySpark for enterprise data solutions.
• Experience with AWS/Azure/GCP and data governance practices.

Responsibilities

• Serve as the primary technical lead for the client's Databricks platform, data engineering initiatives, and enterprise reporting ecosystem.
• Partner with business and technology stakeholders to define data platform strategy, architecture roadmap, and scalable analytics solutions aligned with business goals.
• Design, develop, and oversee implementation of high-performance ETL/ELT pipelines, Lakehouse architectures, and data products using Databricks, PySpark, SQL, and cloud native technologies.
• Drive platform reliability, scalability, security, governance, data quality, and cost optimization through engineering best practices and architectural oversight.
• Lead and mentor data engineering teams, establish development standards, conduct design reviews, and promote a culture of engineering excellence.
• Collaborate with business users, analysts, reporting teams, and data scientists to deliver reliable, production-ready data and reporting solutions.

Skills

Databricks

PySpark

Spark SQL

Delta Lake

Unity Catalog

Lakehouse architecture

AWS/Azure/GCP

Python

SQL

Airflow

dbt

Terraform

Kafka

Delta Live Tables

Databricks Workflows

Education

Bachelor's or Master's in CS/Engineering/Math

Tools

Databricks

Airflow

dbt

Terraform

Kafka

Delta Live Tables

Unity Catalog

Delta Lake

Spark SQL

Job description

We are seeking a talented and driven Data Engineer with expertise in Databricks to join our team in USA. The ideal candidate is passionate about building data platforms, designing scalable solutions, and working in modern cloud-native environments. As a Data Engineer at Eucloid, you will play a key role in developing and optimizing robust data architectures using Databricks Lakehouse Platform, while collaborating with cross-functional teams to drive impactful analytics and machine learning outcomes for our clients.

Responsibilities

• Serve as the primary technical lead and trusted advisor for the client's Databricks platform, data engineering initiatives, and enterprise reporting ecosystem.
• Partner with business and technology stakeholders to define data platform strategy, architecture roadmap, and scalable analytics solutions aligned with business goals.
• Design, develop, and oversee implementation of high-performance ETL/ELT pipelines, Lakehouse architectures, and data products using Databricks, PySpark, SQL, and cloud native technologies.
• Drive platform reliability, scalability, security, governance, data quality, and cost optimization through engineering best practices and architectural oversight.
• Lead and mentor data engineering teams, establish development standards, conduct design reviews, and promote a culture of engineering excellence.
• Collaborate with business users, analysts, reporting teams, and data scientists to deliver reliable, production-ready data and reporting solutions.
• Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, Statistics, or a related quantitative discipline.
• 10+ years of experience in Data Engineering, Data Warehousing, and Cloud Data Platforms.
• 4+ years of hands-on experience designing and implementing solutions on Databricks.
• Strong expertise in Databricks, PySpark, Spark SQL, Delta Lake, Unity Catalog, and Lakehouse architecture.
• Proven experience building and managing large-scale data platforms on AWS, Azure, or GCP.
• Proficiency in Python, PySpark, SQL, and data modeling for enterprise-scale data solutions.
• Hands-on experience with Airflow, dbt, Terraform, Kafka, Databricks Workflows, and Delta Live Tables.
• Strong stakeholder management, communication, and team leadership skills; Databricks and/or Cloud certifications are preferred.