Senior Data Engineer - Build Scalable Pipelines (Hybrid)

7Th Sky Tech; New York (NY); Hybrid; USD 140,000 - 175,000; Full time

Job summary

7Th Sky Tech is looking for a Senior Data Engineer with 8+ years of hands-on experience to design, develop, and maintain scalable data pipelines and architectures. The role focuses on Python, SQL, ETL/ELT, and cloud data platforms to enable data-driven decision-making.

The ideal candidate will work closely with Data Scientists, Software Engineers, and Business Analysts in a Hybrid environment, across AWS, Azure, and GCP ecosystems, ensuring performance and data quality at scale.

Qualifications

• 8+ years of hands-on experience in Data Engineering or related roles.
• Strong programming experience with Python.
• Advanced SQL skills with experience in query optimization.
• Hands-on experience with ETL/ELT development and data pipeline architecture.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines.
• Build and optimize data processing workflows using Python and SQL.
• Develop data ingestion solutions from databases, APIs, files, and other sources.
• Design and maintain enterprise data warehouses, data lakes, and lakehouse architectures.
• Work with large datasets using distributed processing technologies such as Apache Spark/PySpark.
• Develop data models and optimize complex SQL queries for performance.
• Implement data quality, validation, monitoring, and error-handling frameworks.
• Integrate data from structured and unstructured sources.
• Build and maintain batch and real-time/streaming data pipelines.
• Work with cloud data platforms such as AWS, Azure, or GCP.
• Implement CI/CD and DevOps practices for data engineering workflows.
• Collaborate with cross-functional teams to understand business and technical requirements.
• Troubleshoot pipeline failures, performance issues, and data quality problems.
• Ensure data security, governance, scalability, and compliance.
• Mentor junior and mid-level data engineers and contribute to technical architecture decisions.
• Document data pipelines, architectures, processes, and technical solutions.

Skills

Python

SQL

ETL/ELT

Apache Spark

PySpark

Data warehousing

Airflow

CI/CD

Git

Education

Bachelor's degree

Tools

Databricks

Snowflake

AWS Glue

Azure Data Factory

BigQuery

Kubernetes

Docker

Airflow

Git

Job description

7Th Sky Tech is looking for a Senior Data Engineer with 8+ years of hands-on experience to design, develop, and maintain scalable data pipelines and architectures. The role focuses on Python, SQL, ETL/ELT, and cloud data platforms to enable data-driven decision-making.

The ideal candidate will work closely with Data Scientists, Software Engineers, and Business Analysts in a Hybrid environment, across AWS, Azure, and GCP ecosystems, ensuring performance and data quality at scale.