Senior Databricks Engineer: Lakehouse & Spark Pros

Cloud Analytics Technologies, LLC; Austin (TX); In loco; USD 150.000 - 190.000; Tempo pieno

Descrizione del lavoro

Cloud Analytics Technologies, LLC in Austin, TX seeks a Databricks Engineer to design and optimize scalable data pipelines on the Databricks Lakehouse Platform. You will build robust ETL/ELT processes, transform large datasets, and support enterprise analytics initiatives.

The role requires strong PySpark, Python, SQL, and Spark expertise, plus experience with Delta Lake, Unity Catalog, and orchestration tools like Airflow or Data Factory.

Competenze

• 5+ years of data engineering or big data experience.
• 3+ years with Databricks and Apache Spark.
• Strong Python, PySpark, and SQL skills.
• Experience with data ingestion, ETL/ELT pipelines.
• Knowledge of data modeling and data warehousing concepts.
• Familiarity with cloud storage (ADLS, S3, GCS).

Mansioni

• Design, develop, and optimize scalable data pipelines on Databricks.
• Process large datasets and implement data transformations.
• Collaborate with data architects and analysts to deliver analytics.
• Configure and optimize Databricks clusters and jobs.
• Implement data quality checks and governance standards.

Conoscenze

Databricks

PySpark

Python

SQL

Apache Spark

Delta Lake

ETL/ELT

Data pipelines

Formazione

Bachelor's in CS/IT/Data Eng

Master's preferred

Databricks Data Engineer Certification advantageous

Strumenti

Azure Databricks

AWS Databricks

Databricks SQL

Delta Live Tables

Unity Catalog

Airflow

Descrizione del lavoro

Cloud Analytics Technologies, LLC in Austin, TX seeks a Databricks Engineer to design and optimize scalable data pipelines on the Databricks Lakehouse Platform. You will build robust ETL/ELT processes, transform large datasets, and support enterprise analytics initiatives.

The role requires strong PySpark, Python, SQL, and Spark expertise, plus experience with Delta Lake, Unity Catalog, and orchestration tools like Airflow or Data Factory.