Senior PySpark Data Engineer - Hybrid, TX, W2

Highbrow Technology Inc

Raleigh (NC)

Hybrid

USD 120,000 - 150,000

Full time

3 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Highbrow Technology Inc. is seeking an experienced Data Engineer with strong PySpark expertise to design, develop, and maintain scalable data processing solutions.

The role requires extensive experience with big data technologies, distributed computing, ETL/ELT development, and cloud-based data platforms. The ideal candidate will handle large datasets, build efficient data pipelines, optimize Spark workloads, and collaborate with cross-functional teams to deliver reliable data solutions.

Qualifications

  • 8+ years of experience in Data Engineering or related roles.
  • Strong hands-on experience with Python and PySpark.
  • Extensive experience with Apache Spark, Spark SQL, and distributed data processing.
  • Strong proficiency in SQL, including complex queries, joins, aggregations, and query optimization.
  • Experience developing and maintaining large-scale ETL/ELT data pipelines. Strong understanding of data warehousing concepts, data modeling, and data lake architectures.
  • Experience working with cloud data platforms such as AWS, Azure, or GCP.
  • Experience with technologies such as Databricks, Delta Lake, Hive, or similar big data frameworks.
  • Knowledge of workflow orchestration tools such as Apache Airflow, Azure Data Factory, or AWS Glue.
  • Experience with Git, CI/CD pipelines, and Agile development methodologies.
  • Strong analytical, troubleshooting, and problem-solving skills.

Responsibilities

  • Design, develop, and maintain scalable data processing solutions.
  • Build and optimize large-scale data pipelines and Spark workloads.
  • Collaborate with cross-functional teams to deliver reliable data solutions.
  • Ensure data quality, governance, and performance across platforms.

Skills

Python
PySpark
Apache Spark
Spark SQL
SQL
ETL/ELT
Data Warehousing
Data Modeling
Cloud Platforms
Databricks
Delta Lake
Hive
Airflow
CI/CD
Git
Agile
Troubleshooting

Tools

Databricks
Delta Lake
Hive
Docker
Kubernetes

Job description

Highbrow Technology Inc. is seeking an experienced Data Engineer with strong PySpark expertise to design, develop, and maintain scalable data processing solutions.

The role requires extensive experience with big data technologies, distributed computing, ETL/ELT development, and cloud-based data platforms. The ideal candidate will handle large datasets, build efficient data pipelines, optimize Spark workloads, and collaborate with cross-functional teams to deliver reliable data solutions.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior PySpark Data Engineer - Cloud & ETL Specialist
Senior PySpark Data Engineer - Cloud & ETL Specialist

Highbrow Technology Inc • Dallas (TX)

On-site
USD 140,000 - 200,000
Senior Data Engineer | Python & PySpark Cloud Pipelines
Senior Data Engineer | Python & PySpark Cloud Pipelines

MACHINE LEARNING TECHNOLOGIES LLC • Atlanta (GA)

Hybrid
USD 93,000 - 133,000
Data Engineer – PySpark
Data Engineer – PySpark

Highbrow Technology Inc • Raleigh (NC)

Hybrid
USD 120,000 - 150,000
Data Engineer (PySpark & Python)
Data Engineer (PySpark & Python)

Highbrow Technology Inc • Dallas (TX)

On-site
USD 140,000 - 200,000
Senior PySpark Data Engineer
Senior PySpark Data Engineer

Covetus • Irving (TX)

On-site
USD 100,000 - 130,000
Senior PySpark Data Engineer: Build Robust ETL Pipelines
Senior PySpark Data Engineer: Build Robust ETL Pipelines

MTK Technologies • Irving (TX)

On-site
USD 90,000 - 120,000
Senior Data Engineer
Senior Data Engineer

MACHINE LEARNING TECHNOLOGIES LLC • Atlanta (GA)

Hybrid
USD 93,000 - 133,000
Senior PySpark Data Engineer | ETL & Data Quality
Senior PySpark Data Engineer | ETL & Data Quality

Tech Mirrors • Irving (TX)

On-site
USD 96,000 - 152,000
Data Engineer
Data Engineer

The Value Maximizer • United States

On-site
USD 90,000 - 120,000
Senior PySpark Data Engineer: ETL & Scalable Pipelines
Senior PySpark Data Engineer: ETL & Scalable Pipelines

Covetus • Irving (TX)

On-site
USD 100,000 - 130,000