Databricks Engineer: Genie AI & Lakehouse Pipelines

Valiant Technologies LLC

Weehawken (NJ)

On-site

USD 87,000 - 117,000

Full time

4 days ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Valiant Technologies LLC in New Jersey is seeking a Databricks Data Engineer to build scalable data pipelines on the Databricks platform and enable Genie AI-powered conversational analytics. You will implement Bronze-Silver-Gold medallion architecture, use Unity Catalog for governance, and leverage Delta Lake within a Lakehouse framework.

The role demands strong PySpark, Python and SQL skills, Spark optimization, and collaboration with analytics and business teams.

Qualifications

Strong hands-on experience with Databricks / Azure Databricks.
Hands-on PySpark, Python and SQL development.
Experience with Apache Spark and Delta Lake.
Familiarity with ETL/ELT and data engineering best practices.
Experience with Databricks SQL and Unity Catalog.
Knowledge of Lakehouse/Medallion architecture concepts.
Exposure to Genie AI / Genie Agents is a plus.

Responsibilities

Build and optimize data pipelines using Databricks, PySpark, Python and SQL.
Develop ETL/ELT solutions using Delta Lake and Lakehouse architecture.
Implement Bronze, Silver and Gold Medallion architecture.
Work with Unity Catalog for data governance and access management.
Configure and support Genie AI / Genie Agents for natural-language data querying and conversational analytics.
Optimize Spark jobs, Databricks SQL workloads and data pipelines.
Collaborate with data, analytics and business teams to deliver scalable data solutions.