Genie AI-Driven Databricks Engineer

Machine Learning Technologies, LLC; Weehawken (NJ); On-site; USD 87,000 - 117,000; Full time

Job summary

Machine Learning Technologies, LLC in New Jersey is seeking an experienced Databricks Data Engineer to build scalable data pipelines and enable Genie AI-powered analytics on the Databricks platform. The role requires hands-on data engineering expertise and strong knowledge of Delta Lake and Lakehouse architectures.

You will implement Bronze-Silver-Gold medallion design, manage Unity Catalog, and optimize Spark jobs, Databricks SQL workloads, and data workflows while collaborating with data and

Qualifications

• Hands-on Databricks/Azure Databricks experience required.
• Experience building data pipelines with PySpark, Python and SQL.
• Familiarity with Delta Lake, Lakehouse architectures, and data governance.

Responsibilities

• Build and optimize data pipelines using Databricks, PySpark, Python and SQL.
• Develop ETL/ELT solutions with Delta Lake and Lakehouse architecture.
• Implement Bronze, Silver and Gold Medallion architecture.
• Work with Unity Catalog for data governance and access management.
• Configure and support Genie AI/Genie Agents for NL data querying and analytics.
• Optimize Spark jobs and Databricks SQL workloads.
• Collaborate with data, analytics and business teams to deliver scalable data solutions.

Skills

Databricks

Azure Databricks

PySpark

Python

SQL

Apache Spark

Delta Lake

Genie AI

Data Engineering

Lakehouse Architecture

Medallion Architecture

Unity Catalog

Tools

Databricks

Azure Databricks

Python

SQL

PySpark

Delta Lake

Spark

Job description

Machine Learning Technologies, LLC in New Jersey is seeking an experienced Databricks Data Engineer to build scalable data pipelines and enable Genie AI-powered analytics on the Databricks platform. The role requires hands-on data engineering expertise and strong knowledge of Delta Lake and Lakehouse architectures.

You will implement Bronze-Silver-Gold medallion design, manage Unity Catalog, and optimize Spark jobs, Databricks SQL workloads, and data workflows while collaborating with data and