Senior Data Engineer - PySpark/Databricks (Hybrid, Charlotte)

Gemburg Inc.; Charlotte (NC); Hybrid; USD 96,000 - 165,000; Part time

Job summary

Gemburg Inc. in Charlotte, North Carolina, seeks an experienced Senior Data Engineer to design and maintain scalable data pipelines using PySpark, Databricks, Python and SQL. The role focuses on ETL/ELT development and large-volume data processing with performance optimization.

The ideal candidate will write clean, production-ready code, collaborate with data architects and analysts, and contribute to reliable data solutions in a hybrid contract setting.

Qualifications

• 10+ years of experience in data engineering or related software/data development roles.
• Strong hands-on experience with Python and PySpark.
• Strong SQL skills, including complex queries, joins, aggregations, and query optimization.
• Experience building and maintaining ETL/ELT data pipelines.
• Knowledge of Apache Spark architecture, distributed processing, and performance tuning.
• Experience with Delta Lake and lakehouse architecture.
• Strong coding, debugging, troubleshooting, and problem-solving skills.
• Experience with Git and software development best practices.
• Familiarity with cloud platforms such as AWS, Azure, or GCP.
• Strong communication and collaboration skills.

Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark and Databricks.
• Write, test, debug, and optimize Python, PySpark, and SQL code.
• Build and maintain ETL/ELT processes for structured and unstructured data.
• Develop data transformations and workflows to process large volumes of data.
• Work with Databricks notebooks, Delta Lake, and Spark-based data processing frameworks.
• Optimize Spark jobs, troubleshoot performance issues, and improve pipeline efficiency.
• Implement data quality checks, validation, error handling, and monitoring.
• Integrate data from multiple sources, including databases, APIs, cloud storage, and enterprise systems.
• Collaborate with data architects, analysts, and business stakeholders to deliver reliable data solutions.
• Follow coding standards, version control practices, testing procedures, and CI/CD processes.

Skills

Python

PySpark

SQL

ETL/ELT

Delta Lake

Apache Spark

Databricks

Git

Cloud platforms (AWS/Azure/GCP)

CI/CD

Agile

Tools

Databricks

Airflow

Azure Data Factory

Databricks Workflows

Job description

Gemburg Inc. in Charlotte, North Carolina, seeks an experienced Senior Data Engineer to design and maintain scalable data pipelines using PySpark, Databricks, Python and SQL. The role focuses on ETL/ELT development and large-volume data processing with performance optimization.

The ideal candidate will write clean, production-ready code, collaborate with data architects and analysts, and contribute to reliable data solutions in a hybrid contract setting.