Databricks Engineer: GenAI Data Pipelines & Lakehouse

Artifintech; Weehawken (NJ); On-site; USD 84,000 - 120,000; Full time

Job summary

Artifintech seeks an experienced Databricks Engineer with Genie AI to build scalable data pipelines on the Databricks platform and enable AI-powered conversational analytics.

The role focuses on constructing robust ETL/ELT processes, leveraging Delta Lake, and implementing medallion architecture while ensuring governance via Unity Catalog. Collaboration with analytics and business teams will drive scalable data solutions.

Qualifications

• Experience with Databricks or Azure Databricks in data engineering.
• Hands-on with PySpark, Python and SQL.
• Familiarity with Delta Lake and Lakehouse architecture.
• Experience implementing Bronze, Silver and Gold medallion architectures.

Responsibilities

• Build and optimize data pipelines using Databricks, PySpark, Python and SQL.
• Develop ETL/ELT solutions using Delta Lake and Lakehouse architecture.
• Implement Bronze, Silver and Gold Medallion architecture.
• Work with Unity Catalog for data governance and access management.
• Configure and support Genie AI / Genie Agents for natural-language data querying and conversational analytics.
• Optimize Spark jobs, Databricks SQL workloads and data pipelines.
• Collaborate with data, analytics and business teams to deliver scalable data solutions.

Skills

Data engineering

SQL & Python

Spark / PySpark

Genie AI exposure

Tools

Databricks

Azure Databricks

PySpark

Python

SQL

Delta Lake

Unity Catalog

Lakehouse Architecture

Genie AI / Genie Agents

Job description

Artifintech seeks an experienced Databricks Engineer with Genie AI to build scalable data pipelines on the Databricks platform and enable AI-powered conversational analytics.

The role focuses on constructing robust ETL/ELT processes, leveraging Delta Lake, and implementing medallion architecture while ensuring governance via Unity Catalog. Collaboration with analytics and business teams will drive scalable data solutions.