Data Engineer: PySpark, SQL & Cloud Data Pipelines

TechDigital Group; Minnesota; On-site; USD 80,000 - 120,000; Full time

Job summary

An innovative firm is seeking a skilled Data Engineer to develop and optimize ETL/ELT pipelines using PySpark and SQL. In this role, you will work with both structured and unstructured data to build scalable data solutions, ensuring data quality and performance optimization. Collaborating closely with Data Scientists and Analysts, you will design data models and implement robust data workflows on cloud platforms. This position offers a fantastic opportunity to contribute to cutting-edge data projects and enhance your skills in a dynamic and supportive environment.

Qualifications

• Strong experience in Python and PySpark for data engineering tasks.
• Deep understanding of SQL and ETL/ELT development using Spark.

Responsibilities

• Develop and maintain ETL/ELT pipelines using PySpark and SQL.
• Collaborate with Data Scientists and Analysts to integrate data workflows.

Skills

Python

PySpark

SQL

ETL/ELT Development

Cloud Data Services

Data Warehousing

Orchestration Tools

Version Control (Git)

Tools

AWS Glue

Databricks

Azure Synapse

GCP BigQuery

Airflow

Apache Oozie

Snowflake

Redshift

Job description

An innovative firm is seeking a skilled Data Engineer to develop and optimize ETL/ELT pipelines using PySpark and SQL. In this role, you will work with both structured and unstructured data to build scalable data solutions, ensuring data quality and performance optimization. Collaborating closely with Data Scientists and Analysts, you will design data models and implement robust data workflows on cloud platforms. This position offers a fantastic opportunity to contribute to cutting-edge data projects and enhance your skills in a dynamic and supportive environment.