Senior Data Engineer: Azure Data Platform & Pipelines

7Th Sky Tech; Phoenix (AZ); Hybrid; USD 140,000 - 190,000; Full time

Job summary

7Th Sky Tech is seeking a Senior Data Engineer with 8+ years of hands-on experience to design, build, and maintain scalable data platforms and pipelines. You will collaborate with data scientists and software engineers to enable data-driven decision making and deliver high-performance data solutions.

The role requires strong Python, SQL, ETL/ELT, and cloud platform expertise; experience with data warehouses and distributed processing is essential.

Qualifications

• 8+ years of hands-on experience in data engineering or related roles.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize data processing workflows using Python and SQL.
• Develop data ingestion from databases, APIs, files, etc.
• Design and maintain data warehouses, data lakes, and lakehouse architectures.
• Work with large datasets using distributed processing technologies like Spark/PySpark.
• Develop data models and optimize SQL queries for performance.
• Implement data quality, validation, monitoring, and error handling.
• Integrate data from structured and unstructured sources.
• Build and maintain batch and real-time streaming pipelines.
• Collaborate with cross-functional teams to meet business requirements.
• Troubleshoot pipeline failures, performance, and data quality issues.
• Ensure data security, governance, scalability, and compliance.
• Mentor junior engineers and contribute to architectural decisions.
• Document pipelines and architectures clearly.

Skills

Python

SQL

ETL/ELT

Distributed data processing

Spark/PySpark

Data modeling

Data warehousing

CI/CD

Git

Cloud platforms

Apache Kafka

Education

Bachelor's degree in CS/Engineering or related field

Master's degree (preferred)

Tools

Databricks

Snowflake

Redshift

Synapse

Data Factory

ADLS

BigQuery

Azure Databricks

Job description

7Th Sky Tech is seeking a Senior Data Engineer with 8+ years of hands-on experience to design, build, and maintain scalable data platforms and pipelines. You will collaborate with data scientists and software engineers to enable data-driven decision making and deliver high-performance data solutions.

The role requires strong Python, SQL, ETL/ELT, and cloud platform expertise; experience with data warehouses and distributed processing is essential.