Senior Cloud Data Engineer - Azure, Databricks & Spark

7Th Sky Tech; Minneapolis (MN); Hybrid; USD 120,000 - 160,000; Full time

Job summary

7Th Sky Tech is seeking a Senior Data Engineer with 8+ years of hands-on experience to design and maintain scalable data platforms. You will work with Python, SQL, ETL/ELT processes, and cloud-based data solutions to enable data-driven decisions.

The role involves building data pipelines, data warehouses, lakehouse architectures, and coordinating with Data Scientists and Software Engineers. Hybrid work model in a technology-focused environment is offered.

Qualifications

• 8+ years of professional data engineering experience.
• Proficiency in Python and SQL.
• Hands-on ETL/ELT design and data pipelines.
• Experience with Apache Spark / PySpark.
• Cloud platforms (AWS, Azure, GCP) experience.
• Strong data warehousing and dimensional modeling knowledge.
• Familiarity with data security, governance, and metadata management.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize data processing workflows with Python and SQL.
• Develop data ingestion from databases, APIs, files, and more.
• Design and maintain data warehouses, data lakes, and lakehouse architectures.
• Work with distributed processing technologies like Spark/PySpark.
• Create and optimize complex SQL queries for performance.
• Implement data quality, monitoring, and error-handling frameworks.
• Ensure security, governance, and scalability of data platforms.
• Mentor junior engineers and contribute to architecture decisions.

Skills

Python

SQL

ETL/ELT

Spark/PySpark

Cloud platforms

Data modeling

Data warehousing

DevOps

Airflow

Education

Bachelor's degree in CS/Engineering

Tools

Databricks

Snowflake

Redshift

BigQuery

Airflow

Git

Docker

Kubernetes

Jenkins/GitHub Actions

Terraform

Job description

7Th Sky Tech is seeking a Senior Data Engineer with 8+ years of hands-on experience to design and maintain scalable data platforms. You will work with Python, SQL, ETL/ELT processes, and cloud-based data solutions to enable data-driven decisions.

The role involves building data pipelines, data warehouses, lakehouse architectures, and coordinating with Data Scientists and Software Engineers. Hybrid work model in a technology-focused environment is offered.