Onshore Data Engineer (Python, Pytorch)

Capgemini; Dallas (TX); On-site; USD 54,000 - 85,000; Full time

Benefits offered by this job

Medical benefits

Retirement benefits

Job summary

Capgemini is seeking an on-site Data Engineer in Dallas to design, develop, and optimize scalable data and ML pipelines on cloud-native platforms, with Databricks-based engineering practices for analytics and PyTorch-driven AI initiatives.

You will deliver batch and real-time ingestion, ETL/ELT solutions, and data quality and observability. Collaborating with data scientists, you’ll enable model training, feature stores, and ML workflows within CI/CD and MLOps ecosystems.

Qualifications

• Experience with Databricks Platform and cloud-native data platforms.
• Strong Python and PySpark development skills.
• Hands-on PyTorch for ML workloads and model training.
• Experience with Delta Lake, Unity Catalog and Databricks Workflows.
• Proven ETL/ELT pipeline development experience.
• Strong SQL and data modeling abilities.
• Experience with distributed computing and big data architectures.
• Knowledge of REST APIs and data integration patterns.
• Git version control and CI/CD tools experience.
• Experience with Azure, AWS, or GCP clouds.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Databricks and PySpark.
• Build batch and real-time data ingestion frameworks from diverse sources.
• Create ETL/ELT solutions for warehousing, analytics, and AI workloads.
• Implement data quality checks, monitoring, and observability.
• Optimize Databricks jobs, notebooks, clusters, and workflows.
• Design Delta Lake architectures, data models, and metadata management.
• Collaborate with Data Scientists to prepare datasets and feature stores.
• Support ML model training and inference infrastructure on Databricks.
• Integrate ML workflows with MLOps pipelines and CI/CD processes.
• Apply DataOps and MLOps best practices and automate deployments.

Skills

Databricks Platform

Python

PySpark

PyTorch

Delta Lake

Unity Catalog

Databricks Workflows

ETL/ELT

SQL

Distributed computing

REST APIs

Git

CI/CD

Azure

AWS

GCP

Spark

Kafka

Data Lake

MLflow

MLOps

Docker

Kubernetes

Infrastructure-as-code

Feature stores

Education

Bachelor’s or Master’s degree

Tools

Docker

Kubernetes

Job description

Capgemini is hiring an on-site contract Data Engineer to design, develop, and optimize scalable data and machine learning pipelines. The role focuses on cloud-native data platforms and Databricks-based engineering practices that support analytics and PyTorch-driven AI/ML initiatives.

Role Overview

This position delivers scalable data pipeline development and operational support for batch and real-time ingestion, ETL/ELT solutions, and data quality and observability. You will also enable machine learning data preparation, feature engineering, and infrastructure for training, experimentation, and inference on Databricks, with integration into MLOps and CI/CD workflows.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Python.
• Build and optimize batch and real-time data ingestion frameworks from diverse data sources.
• Develop ETL/ELT solutions supporting enterprise data warehousing, analytics, and AI/ML workloads.
• Implement data quality checks, validation frameworks, monitoring, and observability solutions.
• Optimize Databricks jobs, notebooks, clusters, and workflows for performance and cost efficiency.
• Design and maintain Delta Lake architectures, data models, and metadata management solutions.
• Collaborate with Data Scientists and ML Engineers to prepare training datasets and feature stores.
• Support deep learning solution development using PyTorch.
• Develop scalable feature engineering pipelines and data transformation frameworks.
• Enable model training, experimentation, and inference infrastructure on Databricks.
• Integrate ML workflows with MLOps pipelines and CI/CD processes.
• Implement DataOps and MLOps best practices.
• Automate deployments using CI/CD pipelines.
• Work with cloud-native services for storage, security, monitoring, and orchestration.
• Support infrastructure-as-code and environment automation initiatives.
• Partner with business, analytics, and data science teams to understand requirements and translate them into technical solutions.
• Ensure compliance with data governance, privacy, and security standards.
• Participate in architecture reviews and technology evaluations.

Required Qualifications

• Strong experience with the Databricks Platform.
• Expertise in Python.
• Strong hands‑on experience with PySpark.
• Experience with PyTorch for machine learning and deep learning workloads.
• Knowledge of Delta Lake, Unity Catalog, and Databricks Workflows.
• Experience with ETL/ELT pipeline development.
• Strong SQL and data modeling skills.
• Experience with distributed computing and big data architectures.
• Knowledge of REST APIs and data integration patterns.
• Git version control and CI/CD tools experience.
• Experience with one or more: Microsoft Azure (preferred), AWS, Google Cloud Platform (GCP).
• Experience supporting machine learning and AI use cases using PyTorch.
• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• 5+ years of experience in Data Engineering.
• 3+ years of experience with Databricks and cloud‑based data platforms.

Technologies

• Databricks, Python, PyTorch, PySpark
• Delta Lake, Unity Catalog, Databricks Workflows
• ETL, ELT, SQL
• REST APIs, Git, CI/CD pipelines
• Azure, AWS, Google Cloud Platform (GCP)
• Spark, Kafka/Event Streaming, Azure Data Factory
• Data Lake architectures, Data Warehousing solutions, Feature Stores
• MLflow, MLOps pipelines
• Docker, Kubernetes
• DataOps tools, MLOps tools
• Infrastructure‑as‑code
• Feature engineering pipelines, CI/CD processes

Preferred Skills

• Experience with GenAI and LLM data pipelines.
• Knowledge of MLflow and MLOps frameworks.
• Hands‑on experience with Docker and Kubernetes.
• Experience with real‑time streaming architectures.
• Familiarity with vector databases and AI platforms.
• Certifications in Databricks or Cloud platforms.

Nice‑to‑Have Experience

• Retail, QSR, Restaurant Technology, Digital Commerce, or Customer Analytics domains.
• Large‑scale enterprise data modernization initiatives.
• AI/ML and advanced analytics implementations.

Location, Employment Type, and Pay

• Location: Dallas, TX, United States (onsite)
• Job Type: Contract
• Pay Range: $39.30/hour - $61.40/hour

Education

Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Systems, or related field.

Benefits

• Medical, dental, vision, and retirement benefits