Databricks Engineer: Genie AI & Data Pipelines

Quantum Technologies. LLC

Weehawken (NJ)

On-site

USD 84,000 - 120,000

Full time

7 hours ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Quantum Technologies. LLC seeks a Databricks Engineer with Genie AI to design and implement scalable data pipelines on the Databricks platform.

The role involves enabling AI-powered conversational analytics through Genie Agents and working across data, analytics, and business teams. The candidate will build Bronze/Silver/Gold data architectures, utilize Unity Catalog for governance, and optimize Spark jobs and SQL workloads to deliver robust data solutions.

Qualifications

Experience with Databricks/Azure Databricks to build scalable data pipelines.
Strong skills in PySpark, Python and SQL for data engineering tasks.
Hands-on knowledge of Apache Spark and Delta Lake architectures.
Experience with ETL/ELT and Lakehouse/Medallion architecture patterns.
Familiarity with Databricks SQL for data querying and analytics.
Knowledge of Unity Catalog for data governance and access control.

Responsibilities

Build and optimize data pipelines using Databricks, PySpark, Python and SQL.
Develop ETL/ELT solutions using Delta Lake and Lakehouse architecture.
Implement Bronze, Silver and Gold Medallion data architectures.
Work with Unity Catalog for data governance and access management.
Configure and support Genie AI/Genie Agents for natural-language data querying.
Optimize Spark jobs, Databricks SQL workloads and data pipelines.
Collaborate with data, analytics and business teams to deliver scalable data solutions.