Senior Data Engineer - Python, PySpark & AWS (CTH)

BCforward; Plano (TX); In loco; USD 90.000 - 96.000; Part-time

Descrizione del lavoro

BCforward is seeking a hands-on Senior Data Engineer in Plano, TX to design, build, and operate scalable big-data pipelines and cloud-native data solutions.

The role emphasizes Python and PySpark with production experience on AWS (S3/EMR/Lambda), infrastructure as code (Terraform), and modern data orchestration. You'll implement ETL/ELT pipelines, Kafka or MQ integrations, and work with Cassandra.

Competenze

• Hands-on experience with Python and PySpark for data engineering.
• Deep experience with Big Data technologies and distributed data processing.
• Production experience with AWS services (S3, EMR, Lambda).
• Experience with infrastructure as code (Terraform).
• Experience with workflow orchestration and scheduling (Airflow, Control-M, native schedulers).
• Experience with Cassandra for high-volume data storage.
• Experience building ETL/ELT pipelines and data integration.
• Knowledge of REST APIs and MQ messaging.
• Java experience is a plus.

Mansioni

• Design, develop, test, deploy, and optimize large-scale batch and streaming data pipelines.
• Build and maintain ETL/ELT workflows ingesting, transforming, validating, and delivering data.
• Develop and support AWS-based data solutions using S3, EMR, and Lambda.
• Provision, manage, and version cloud infrastructure using Terraform.
• Build event-driven integrations using Kafka, MQ, and REST APIs.
• Develop data services using Java where required.
• Orchestrate production workloads with Airflow, Control-M, and native schedulers.
• Work with Cassandra for high-volume data storage and retrieval.
• Implement data-quality checks, logging, monitoring, and performance tuning.
• Collaborate with data engineers, architects, software engineers, and stakeholders.

Conoscenze

Python

PySpark

Big Data

Data pipelines

Strumenti

AWS

S3

EMR

Lambda

Terraform

Airflow

Control-M

Kafka

MQ messaging

REST APIs

Java

Cassandra

Descrizione del lavoro

BCforward is seeking a hands-on Senior Data Engineer in Plano, TX to design, build, and operate scalable big-data pipelines and cloud-native data solutions.

The role emphasizes Python and PySpark with production experience on AWS (S3/EMR/Lambda), infrastructure as code (Terraform), and modern data orchestration. You'll implement ETL/ELT pipelines, Kafka or MQ integrations, and work with Cassandra.