Data Platform Lead — Spark, Snowflake & Lakehouse

Univedge Consulting LLC; United States; On-site; USD 150,000 - 210,000; Full time

Job summary

Univedge Consulting LLC is seeking an experienced Lead Data Engineer to design and implement enterprise-scale data platforms, pipelines, and integrations. The role emphasizes Python, PySpark, SQL, Spark, Databricks, Snowflake, and modern Lakehouse architectures.

You will guide engineering teams and align with data architects and stakeholders. You will architect data structures, ensure governance and security, and drive delivery across multiple teams and platforms, with a focus on reliability and

Qualifications

• 12+ years IT/data engineering experience with data modeling and pipelines.
• Hands-on Python, PySpark, SQL, Spark experience.
• Experience with ETL/ELT pipelines and enterprise platforms.
• Knowledge of Databricks, Snowflake, Delta Lake, and data warehouses.
• Strong collaboration with analytics, architecture, and business teams.

Responsibilities

• Design, develop, and maintain scalable data pipelines (batch/real-time).
• Build ingestion, transformation, and processing workflows across systems.
• Develop enterprise data warehouses, data lakes, and processing systems.
• Write complex SQL and optimize performance across pipelines.
• Troubleshoot production data issues and improve reliability.
• Data Architecture & Platform Engineering responsibilities including schemas and models.
• Lead teams, translate requirements into technical plans, and ensure delivery.
• Ensure governance, quality, security, and HIPAA considerations.
• Analytics, AI & Data Science enablement collaboration and data preparation.
• DevOps & DataOps practices to improve deployment and monitoring.

Skills

Python

PySpark

SQL

Spark

Data Modeling

ETL/ELT pipelines

Cloud data platforms

Leadership

Data governance

Collaboration

Education

Bachelor's degree in CS/IT or related

Tools

Databricks

Snowflake

Delta Lake

OCI

Job description

Univedge Consulting LLC is seeking an experienced Lead Data Engineer to design and implement enterprise-scale data platforms, pipelines, and integrations. The role emphasizes Python, PySpark, SQL, Spark, Databricks, Snowflake, and modern Lakehouse architectures.

You will guide engineering teams and align with data architects and stakeholders. You will architect data structures, ensure governance and security, and drive delivery across multiple teams and platforms, with a focus on reliability and