Databricks Data Engineer: Scalable Spark Pipelines

Artifintech; Madison (WI); On-site; USD 110,000 - 150,000; Full time

Benefits offered by this job

Health insurance

Job summary

Artifintech seeks an experienced Data Engineer to design and maintain scalable data pipelines using Databricks and PySpark. You will develop ETL/ELT workflows with Python and SQL, and work with Apache Spark, Delta Lake, and Unity Catalog across major cloud platforms.

The role involves data integration from multiple sources, performance optimization, and implementing data quality checks. Ideal candidates will have hands-on experience with Databricks, PySpark, and cloud environments, and will

Qualifications

• Experience building data pipelines with Databricks and PySpark.
• Strong Python and SQL skills for ETL/ELT development.
• Hands-on with cloud platforms (AWS, Azure, GCP) and big data tooling.

Responsibilities

• Develop data pipelines using Databricks and PySpark.
• Build ETL/ELT workflows with Python and SQL.
• Work with Apache Spark, Delta Lake, and Unity Catalog.
• Integrate data from multiple sources and optimize SQL
• Implement data quality and validation processes.

Skills

Databricks

PySpark

Python

SQL

Cloud technologies

Tools

Apache Spark

Delta Lake

Unity Catalog

Job description

Artifintech seeks an experienced Data Engineer to design and maintain scalable data pipelines using Databricks and PySpark. You will develop ETL/ELT workflows with Python and SQL, and work with Apache Spark, Delta Lake, and Unity Catalog across major cloud platforms.

The role involves data integration from multiple sources, performance optimization, and implementing data quality checks. Ideal candidates will have hands-on experience with Databricks, PySpark, and cloud environments, and will