Databricks Solutions Architect – Streaming/Data Pipelines (Onsite)

Pyramid Consulting, Inc; Jersey City (NJ); On-site; USD 76,000 - 83,000; Full time

Job summary

Pyramid Consulting, Inc. seeks a Databricks Consultant for an onsite Jersey City, NJ engagement. This 6+ months contract focuses on architecting scalable data solutions with strong governance and seamless stakeholder alignment.

You will design batch and streaming pipelines using Flink and Kafka, leveraging Unity Catalog, Live Tables, and Workflows while applying Azure/AWS cloud integration expertise. Automotive industry client with long-term potential.

Qualifications

• 5+ years of proven Solutions Architect experience with Databricks, Flink, and Kafka.
• Strong hands-on expertise in Databricks, Apache Spark, PySpark, SQL, and Client Lake.

Responsibilities

• Architect scalable, high-performance data solutions with strong governance.
• Lead solution design, technical discussions, and architectural decisions.
• Engage with stakeholders to ensure alignment and guide engineering teams.
• Design and implement batch and streaming data pipelines using Apache Flink and Apache Kafka.
• Utilize Unity Catalog, Client Live Tables, and Databricks Workflows for effective data management.
• Leverage your knowledge of Azure/AWS cloud platforms, ETL/ELT, and data integration.

Skills

Databricks

Flink

Kafka

Spark

PySpark

SQL

Tools

Unity Catalog

Client Live Tables

Databricks Workflows

Job description

Pyramid Consulting, Inc. seeks a Databricks Consultant for an onsite Jersey City, NJ engagement. This 6+ months contract focuses on architecting scalable data solutions with strong governance and seamless stakeholder alignment.

You will design batch and streaming pipelines using Flink and Kafka, leveraging Unity Catalog, Live Tables, and Workflows while applying Azure/AWS cloud integration expertise. Automotive industry client with long-term potential.