Senior Data Engineer – Real-Time Pipelines & Data Solutions

American Express Company

Hoschton (GA)

Remote

USD 42,000 - 52,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

American Express is seeking a Senior Data Engineer to own design, development, testing and deployment of high‑performance data pipelines and analytics solutions for HR data services. You will collaborate with product, analytics and data engineering colleagues to accelerate digital adoption, experimentation and customer insights across platforms.

The role focuses on data architecture, scalable workflows, CI/CD deployments and governance, with cross‑functional collaboration across teams.

Qualifications

  • At least 5 years of software development experience in big-data technologies.
  • Proficiency with Python, Spark (PySpark, Spark SQL), Hive, and Shell scripting.
  • Hands-on experience building data ingestion pipelines and data consumption services (e.g., data platforms).
  • Strong knowledge of large-scale data processing frameworks such as Spark, MapReduce, YARN, Hive.
  • Experience with NoSQL databases (HBase, Cassandra, Redis, Memcached).
  • Familiarity with cloud platforms (GCP preferred) and distributed data services.
  • Experience with CI/CD pipelines using GitHub, Jenkins and XLR.

Responsibilities

  • Define data architecture, design and integration solutions for enterprise data services.
  • Write, test, deploy and maintain scalable data engineering workflows including batch, stream and real-time pipelines.
  • Lead and mentor an agile engineering team through code reviews, CI/CD, documentation and best practices.
  • Debug, troubleshoot and optimize data application performance.
  • Automate deployments to development, QA and production environments using CI/CD pipelines.
  • Implement data governance, security and compliance controls to meet regulatory requirements.
  • Collaborate with analytics, product and data scientists to build dashboards and BI solutions.
  • Participate in coding standards and technical design reviews for data services.

Skills

Python
Spark PySpark
Hive
Shell scripting
Big-data
Data ingestion pipelines
CI/CD
Distributed data processing
NoSQL databases
Cloud platforms
Spark streaming
MapReduce
YARN
HBase
Cassandra
Redis
Memcached
Kubernetes

Tools

GitHub
Jenkins
XLR

Job description

American Express is seeking a Senior Data Engineer to own design, development, testing and deployment of high‑performance data pipelines and analytics solutions for HR data services. You will collaborate with product, analytics and data engineering colleagues to accelerate digital adoption, experimentation and customer insights across platforms.

The role focuses on data architecture, scalable workflows, CI/CD deployments and governance, with cross‑functional collaboration across teams.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer – Pipelines & Architecture
Senior Data Engineer – Pipelines & Architecture

American Express Company • United States

Remote
USD 90,000 - 150,000
Senior Data Engineer — Remote Data Pipelines & Architecture
Senior Data Engineer — Remote Data Pipelines & Architecture

American Exspress • Juneau (AK)

Hybrid
USD 27,552 - 41,328
Competitive compensation
Performance incentives
6% retirement match
+6
Senior Data Engineer - Remote/Hybrid, OLAP & ETL Expert
Senior Data Engineer - Remote/Hybrid, OLAP & ETL Expert

American Express Company • Phoenix (AZ)

On-site
USD 27,552 - 41,328
Remote work 3 days/week
6% retirement plan match
Comprehensive health benefits
Senior Data Engineer: Real-Time Data Pipelines & AI
Senior Data Engineer: Real-Time Data Pipelines & AI

AMH • Houston (TX)

On-site
USD 121,000 - 151,000
Medical insurance
Dental & Vision insurance
401(k) with company match
+4
Senior Data Engineer — Real-Time Data Pipelines & AI
Senior Data Engineer — Real-Time Data Pipelines & AI

AMH • Irving (TX)

On-site
USD 121,000 - 151,000
Medical, dental, vision insurance
401(k) with company matching
Employee stock purchase plan
+2
Remote Data Engineer — Pipelines & APIs
Remote Data Engineer — Pipelines & APIs

American Express Company • Montgomery (AL)

Remote
USD 27,552 - 41,328
Competitive compensation
Company retirement plan
Wealth-management services
+4
Senior Data Engineer — Remote, Big Data & ETL
Senior Data Engineer — Remote, Big Data & ETL

American Exspress • Nashville (TN)

Hybrid
USD 28,000 - 41,000
6% 401(k) match
Comprehensive health/vision/dental
Paid parental leave
Head of Analytics Engineering & Data Strategy
Head of Analytics Engineering & Data Strategy

Egug • New York (NY)

On-site
USD 180,000 - 230,000
Senior Big Data Engineer — Flexible/Remote
Senior Big Data Engineer — Flexible/Remote

American Express Company • Phoenix (AZ)

Remote
USD 27,552 - 41,328
Competitive pay
Retirement match
Wellbeing support
+5
Remote Hadoop Big Data Engineer – Infrastructure Lead
Remote Hadoop Big Data Engineer – Infrastructure Lead

American Express Company • City of Albany (NY)

Remote
USD 28,000 - 41,000