Databricks Engineer - Lakehouse Data Pipelines & Spark

Quantum Technologies. LLC; Austin (TX); In loco; USD 120.000 - 190.000; Tempo pieno

Descrizione del lavoro

Quantum Technologies. LLC in Austin, TX, seeks a Databricks Engineer to design, build, and optimize scalable data pipelines on the Databricks Lakehouse Platform, enabling enterprise analytics and data-driven decisions.

You will work with Python, PySpark, SQL, Spark, and Delta Lake to ingest data, transform workflows, and ensure governance, security, and performance. Collaboration with data architects, analysts, and business teams is essential.

Competenze

• 5+ years of experience in data engineering or big data development.
• 3+ years of hands-on experience with Databricks and Apache Spark.
• Strong programming experience with Python, PySpark, and SQL.
• Understanding of distributed computing, data architecture, and performance tuning.
• Strong analytical, debugging, and problem-solving skills.

Mansioni

• Design, develop, and optimize scalable data pipelines on Databricks Lakehouse Platform.
• Process large datasets and implement data transformation workflows.
• Collaborate with data architects, analysts, and business teams to deliver analytics solutions.
• Ensure data governance, access controls, security, and performance optimization.
• Implement data quality checks, monitoring, and error-handling mechanisms.

Conoscenze

Databricks Lakehouse

Python

PySpark

SQL

Apache Spark

Delta Lake

Data ingestion

ETL/ELT pipelines

Git / CI/CD

Data governance

Formazione

Bachelor’s degree in CS/IT/Data Engineering

Master’s degree preferred

Databricks Data Engineer certification advantageous

Strumenti

Databricks

Apache Spark

Airflow

Azure Data Factory

Delta Lake

Unity Catalog

Descrizione del lavoro

Quantum Technologies. LLC in Austin, TX, seeks a Databricks Engineer to design, build, and optimize scalable data pipelines on the Databricks Lakehouse Platform, enabling enterprise analytics and data-driven decisions.

You will work with Python, PySpark, SQL, Spark, and Delta Lake to ingest data, transform workflows, and ensure governance, security, and performance. Collaboration with data architects, analysts, and business teams is essential.