Onsite Databricks Data Engineer - Python & PyTorch

Capgemini; Dallas (TX); On-site; USD 54,000 - 85,000; Full time

Benefits offered by this job

Medical benefits

Retirement benefits

Job summary

Capgemini is seeking an on-site Data Engineer in Dallas to design, develop, and optimize scalable data and ML pipelines on cloud-native platforms, with Databricks-based engineering practices for analytics and PyTorch-driven AI initiatives.

You will deliver batch and real-time ingestion, ETL/ELT solutions, and data quality and observability. Collaborating with data scientists, you’ll enable model training, feature stores, and ML workflows within CI/CD and MLOps ecosystems.

Qualifications

• Experience with Databricks Platform and cloud-native data platforms.
• Strong Python and PySpark development skills.
• Hands-on PyTorch for ML workloads and model training.
• Experience with Delta Lake, Unity Catalog and Databricks Workflows.
• Proven ETL/ELT pipeline development experience.
• Strong SQL and data modeling abilities.
• Experience with distributed computing and big data architectures.
• Knowledge of REST APIs and data integration patterns.
• Git version control and CI/CD tools experience.
• Experience with Azure, AWS, or GCP clouds.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Databricks and PySpark.
• Build batch and real-time data ingestion frameworks from diverse sources.
• Create ETL/ELT solutions for warehousing, analytics, and AI workloads.
• Implement data quality checks, monitoring, and observability.
• Optimize Databricks jobs, notebooks, clusters, and workflows.
• Design Delta Lake architectures, data models, and metadata management.
• Collaborate with Data Scientists to prepare datasets and feature stores.
• Support ML model training and inference infrastructure on Databricks.
• Integrate ML workflows with MLOps pipelines and CI/CD processes.
• Apply DataOps and MLOps best practices and automate deployments.

Skills

Databricks Platform

Python

PySpark

PyTorch

Delta Lake

Unity Catalog

Databricks Workflows

ETL/ELT

SQL

Distributed computing

REST APIs

Git

CI/CD

Azure

AWS

GCP

Spark

Kafka

Data Lake

MLflow

MLOps

Docker

Kubernetes

Infrastructure-as-code

Feature stores

Education

Bachelor’s or Master’s degree

Tools

Docker

Kubernetes

Job description

Capgemini is seeking an on-site Data Engineer in Dallas to design, develop, and optimize scalable data and ML pipelines on cloud-native platforms, with Databricks-based engineering practices for analytics and PyTorch-driven AI initiatives.

You will deliver batch and real-time ingestion, ETL/ELT solutions, and data quality and observability. Collaborating with data scientists, you’ll enable model training, feature stores, and ML workflows within CI/CD and MLOps ecosystems.