Senior Data Engineer - AWS Data Platforms & Pipelines

International Business Machines Corporation; United States; Remote; USD 120,000 - 180,000; Full time

Job summary

IBM Consulting is seeking a Data Engineer specializing in Data Platforms on AWS. You will design, build, and maintain batch and real-time data pipelines, create data layers with Redshift/Aurora/DynamoDB, and migrate data using DMS.

You’ll work within IBM Consulting FutureNow Centers to deliver scalable data engineering solutions for global clients. You will leverage AWS services and open source tools like Airflow and dbt to support data pipelines for data warehousing and datalake initiatives,

Qualifications

• Experience with AWS EMR, Glue, Glue Catalog and Kinesis to build batch and real-time data pipelines.
• Experience creating data layers with Redshift, Aurora and DynamoDB and migrating data with DMS.
• Experience scheduling and managing data services on AWS Platform.
• Experience using Apache Airflow and dbt, Spark with Python/Scala on AWS for data engineering.
• Experience with data warehousing and datalake data pipelines via AWS services.

Responsibilities

• Design, build, and operate batch and real-time data pipelines on AWS.
• Create data layers on Redshift, Aurora, and DynamoDB and migrate data with DMS.
• Manage data services on the AWS Platform for reliable operations.
• Develop data pipelines for Data Warehouse and Data Lake using Kinesis and Kafka.
• Leverage open source tools like Airflow and dbt to support data engineering tasks.

Skills

AWS data engineering

Data pipeline design

Batch & real-time processing

Open source tooling

Data modeling

Education

Bachelor's Degree

Master's Degree

Tools

AWS EMR

AWS Glue

Glue Catalog

Kinesis

Redshift

Aurora

DynamoDB

DMS

Apache Airflow

dbt

Spark

Job description

IBM Consulting is seeking a Data Engineer specializing in Data Platforms on AWS. You will design, build, and maintain batch and real-time data pipelines, create data layers with Redshift/Aurora/DynamoDB, and migrate data using DMS.

You’ll work within IBM Consulting FutureNow Centers to deliver scalable data engineering solutions for global clients. You will leverage AWS services and open source tools like Airflow and dbt to support data pipelines for data warehousing and datalake initiatives,