Data Engineer – Databricks & Snowflake

Quantum Technologies. LLC; Dallas (TX); On-site; USD 90,000 - 150,000; Full time

Job summary

Quantum Technologies LLC in Dallas is seeking a skilled Data Engineer to design, develop, and maintain scalable data pipelines and cloud-based data platforms. You will build reliable data integration solutions and support enterprise data warehousing initiatives.

The role emphasizes ETL/ELT development using Databricks and Snowflake, with strong SQL, Python, PySpark, and cloud experience across AWS, Azure, or Google Cloud.

Qualifications

• Design, develop and maintain scalable ETL and ELT data pipelines.
• Build and optimize data processing workflows using Databricks.
• Develop data warehouse solutions using Snowflake.
• Implement data ingestion and transformation processes.
• Work with structured and unstructured data from multiple sources.
• Develop complex SQL queries and optimize database performance.
• Implement data quality checks and validation procedures.
• Integrate data from cloud platforms and enterprise applications.
• Collaborate with analytics and business intelligence teams.
• Monitor pipeline performance and troubleshoot data processing issues.

Responsibilities

• Design, develop, and maintain scalable ETL and ELT data pipelines.
• Build and optimize data processing workflows using Databricks.
• Develop data warehouse solutions using Snowflake.
• Implement data ingestion and transformation processes.
• Work with structured and unstructured data from multiple sources.
• Develop complex SQL queries and optimize database performance.
• Implement data quality checks and validation procedures.
• Integrate data from cloud platforms and enterprise applications.
• Collaborate with analytics and business intelligence teams.
• Monitor pipeline performance and troubleshoot data processing issues.

Skills

Python

PySpark

SQL

Databricks

Apache Spark

Snowflake Data Warehousing

ETL/ELT Development

AWS Azure Google Cloud

Data Modeling

Data Integration

Delta Lake Lakehouse

Apache Airflow

Data Quality

Tools

Databricks

Airflow

Job description

We are looking for a skilled Data Engineer with strong expertise in designing, developing, and maintaining scalable data pipelines and cloud-based data platforms. The candidate will be responsible for building reliable data integration solutions and supporting enterprise data warehousing initiatives.

Key Responsibilities:

• Design, develop, and maintain scalable ETL and ELT data pipelines.
• Build and optimize data processing workflows using Databricks.
• Develop data warehouse solutions using Snowflake.
• Implement data ingestion and transformation processes.
• Work with structured and unstructured data from multiple sources.
• Develop complex SQL queries and optimize database performance.
• Implement data quality checks and validation procedures.
• Integrate data from cloud platforms and enterprise applications.
• Collaborate with analytics and business intelligence teams.
• Monitor pipeline performance and troubleshoot data processing issues.

Required Skills:

• Python, PySpark, and SQL.
• Databricks and Apache Spark.
• Snowflake Data Warehousing.
• ETL/ELT Development.
• AWS, Azure, or Google Cloud.
• Data Modeling and Data Integration.
• Delta Lake and Lakehouse Architecture.
• Apache Airflow or similar orchestration tools.
• Data Quality and Performance Optimization.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.