Lead Data Engineer

The Jupiter Group, Inc; Houston (TX); On-site; USD 140,000 - 190,000; Full time

Job summary

The Jupiter Group, Inc. seeks a senior data architect to design, build, and scale cloud data platforms. You will lead Databricks-driven pipelines on AWS, drive modern architectures like Lakehouse, and enable analytics and AI initiatives across the organization.

You will mentor engineers, collaborate with stakeholders, and govern data across large-scale environments, ensuring security and compliance while improving data integration and governance practices.

Qualifications

• Bachelor’s degree in CS/DS/Engineering or related field.
• 7–10 years building cloud data architectures; 5+ years on AWS.
• Extensive, hands-on Databricks experience.

Responsibilities

• Design and build scalable cloud data pipelines and platforms.
• Lead Databricks-driven data architectures on AWS.
• Mentor engineers and analysts to raise technical capability.
• Foster data governance, security, and compliance across teams.

Skills

Python (PySpark & Pandas)

SQL (T-SQL)

Cloud data architecture (AWS)

Databricks

Leadership & communication

Data governance & cataloging

Education

Bachelor’s degree in Computer Science / Data Science / Engineering

Tools

Databricks

Power BI

Git

SSMS

Job description

In this role, you will design, build, and deliver high-performing, scalable data pipelines and platforms in the cloud. You will drive the move toward modern architectures such as the Lakehouse model and Databricks, improve data integration and governance practices, enable analytics and AI/ML initiatives, and coach engineers and analysts to raise the technical bar and capability of the team.

What You Bring (Must-Haves)

• A bachelor’s degree in Computer Science, Data Science, Engineering, or a related discipline
• Seven to ten years of hands-on experience implementing cloud data architecture, at least five of them in AWS
• Extensive, practical experience with Databricks
• Fluency in Python (PySpark and Pandas), T-SQL, and SQL
• Working knowledge of architectural frameworks: medallion, data mesh, and SOA-based data layers
• Comfort working with Power BI, Git, and SSMS
• Familiarity with data governance and cataloging
• Excellent communication and teamwork skills, plus the ability to lead and influence others
• Experience designing and building scalable data architectures on AWS and Databricks
• Experience leading cloud migrations and modernizing legacy data environments
• Any experience working with data governance, security, and compliance in complex, large-scale environments