Cloud Data Engineer – AWS/Databricks & AI

International Business Machines Corporation; Minneapolis (MN); Remote; USD 120,000 - 180,000; Full time

Job summary

IBM Consulting seeks a Data Engineer embedded with client teams to design and deliver hands-on data solutions on the AWS/Databricks platform. You will re-engineer legacy workloads, migrate data from MySQL to Databricks Lakebase, and integrate SAP and non-SAP data for domain-specific data products, with production-grade prototypes and client-facing communication.

The role involves traveling up to four days per week and is performable from anywhere in the US, collaborating across AWS and GCP for

Qualifications

• Experience designing and delivering complex data solutions on AWS and Databricks platforms.
• Strong experience with data migration, ETL/ELT, and data pipeline architectures.

Responsibilities

• Design and build data pipelines and integrations using Fivetran, Databricks, and AWS services.
• Lead architecture and delivery for migrating a Tier-1 MySQL application to Databricks Lakebase with schema redesign and data migration.
• Collaborate with clients to understand needs and deliver data solutions across AWS and GCP for ML/GenAI use cases.
• Establish engineering standards: CI/CD, testing, observability, security, and cost control.

Education

Databricks Certification

Tools

Databricks

AWS

SQL

Python

Fivetran

Databricks Lakebase

Delta Lake

Unity Catalog

Apache Iceberg

Kafka

EventBridge

SQS

Terraform

CI/CD

Job description

IBM Consulting seeks a Data Engineer embedded with client teams to design and deliver hands-on data solutions on the AWS/Databricks platform. You will re-engineer legacy workloads, migrate data from MySQL to Databricks Lakebase, and integrate SAP and non-SAP data for domain-specific data products, with production-grade prototypes and client-facing communication.

The role involves traveling up to four days per week and is performable from anywhere in the US, collaborating across AWS and GCP for