Senior Data Engineer - Databricks, Spark & Airflow

Compunnel, Inc.

Norfolk (VA)

Hybrid

USD 110,000 - 150,000

Full time

3 days ago

Be an early applicant

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

Compunnel, Inc. is seeking a Data Engineer to design, develop, optimize, and maintain scalable data platforms supporting enterprise analytics.

The role requires 5+ years of experience in data engineering, ETL, and data platform engineering, with strong SQL, Python, SSIS, Airflow, Spark, and Databricks skills. The engineer will collaborate with business stakeholders in an onsite environment to deliver reliable, secure data solutions, focusing on data quality, governance, and scalable

Qualifications

5+ years of experience in Data Engineering, ETL Development, or Data Platform Engineering.
Strong SQL skills with complex queries, data modeling, and performance tuning.
Proficiency in Python for data engineering and ETL.
Hands-on SSIS experience.
Airflow workflow orchestration and Spark data processing expertise.
Experience with Databricks and lakehouse architectures using Iceberg.
Understanding of data warehousing, ETL/ELT concepts, and distributed data systems.
Knowledge of data quality, monitoring, and troubleshooting techniques.
Bachelor's degree in CS/IT/Engineering or related field.

Responsibilities

Design, develop, and optimize scalable data pipelines for batch and real-time processing.
Build and maintain robust ETL/ELT solutions with Python, SQL, and SSIS.
Develop and manage workflow orchestration using Apache Airflow.
Implement data transformation and processing with Apache Spark.
Design and support lakehouse architectures using Databricks and Apache Iceberg.
Ensure data quality, security, and governance across platforms.
Monitor, troubleshoot, and optimize ingestion and processing workflows.
Collaborate with analysts, scientists, and architects to deliver scalable data solutions.
Perform performance tuning of databases, Spark jobs, and pipelines.
Support production deployments and maintenance for critical data systems.
Follow coding, documentation, testing, and release-management best practices.