Senior Data Engineer (Python, PySpark, Databricks) - Remote

GSquared Group; United States; On-site; USD 120,000 - 160,000; Full time

Job summary

GSquared Group is seeking a hands-on Data Engineer with 5+ years of experience to design, develop, and support production-grade data pipelines in a Databricks environment. The role is Long-Term Contract with options for Atlanta, GA or remote work.

You will build scalable, reliable data solutions across ingestion, transformation, and orchestration, collaborating with architects and analytics teams to deliver impactful data platforms.

Qualifications

• 5+ years of professional Data Engineering experience.
• Strong hands-on experience developing data pipelines using Python and PySpark.
• Experience with Databricks.
• Experience building and supporting production-grade data pipelines processing large datasets.
• Knowledge of cloud platforms such as AWS, Azure, or Google Cloud Platform.
• Experience with data quality, validation, monitoring, and logging practices.
• Experience with orchestration tools such as Databricks Workflows, Apache Airflow, or similar.
• Experience with Git and modern CI/CD deployment practices.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Build and optimize batch and streaming data pipelines in a Databricks environment.
• Develop reusable, production-ready code to ingest, transform, and load large-scale datasets.
• Implement data quality validation, error handling, monitoring, and alerting to ensure reliable data processing.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Collaborate with data architects, analytics teams, and business stakeholders to deliver high-quality data solutions.
• Participate in code reviews and contribute to engineering best practices.
• Deploy and support data pipelines through CI/CD processes and automated release pipelines.
• Troubleshoot and resolve production issues while continuously improving pipeline reliability and performance.
• Contribute to the design and evolution of modern cloud-based data platforms.

Skills

Python

PySpark

SQL

Troubleshooting

Problem solving

Collaboration

Analytical thinking

Tools

Databricks

Airflow

Git

CI/CD

Delta Lake

Kafka

Spark

Job description

GSquared Group is seeking a hands-on Data Engineer with 5+ years of experience to design, develop, and support production-grade data pipelines in a Databricks environment. The role is Long-Term Contract with options for Atlanta, GA or remote work.

You will build scalable, reliable data solutions across ingestion, transformation, and orchestration, collaborating with architects and analytics teams to deliver impactful data platforms.