Remote Data Engineer: AWS/Databricks Platform Architect

International Business Machines Corporation; Paramus (NJ); A distancia; USD 110.000 - 170.000; Jornada completa

Descripción de la vacante

IBM Consulting seeks a Data Engineer embedded with client teams to design and deliver hands-on solutions on the AWS/Databricks platform. You will migrate legacy workloads, re-engineer data flows, and partner with SAP, GCP, and other cloud services to provide scalable data products.

You will design end-to-end data pipelines, lead architecture initiatives, ensure CI/CD, security, and cost controls, and mentor delivery teams.

Formación

• Must have hands-on data engineering experience with modern cloud platforms.
• Strong architecture and design skills for complex data solutions.
• Experience migrating data workloads and building scalable data pipelines.

Responsabilidades

• Work onsite or embedded with client teams to understand business problems and turn them into working technical solutions.
• Design and build data pipelines and integrations using Fivetran, Databricks, and AWS services.
• Lead technical design and delivery for migrating a Tier-1 application from MySQL to Databricks Lakebase, including schema re-design, data migration, cutover, and validation.
• Integrate SAP data through SAP BTP and BDC Connect, non-SAP data, and design domain centric data products.
• Build prototypes and reference implementations, then work with the client to harden them for production.
• Support data and AI use cases, including integration with GCP-based ML/GenAI components.
• Establish engineering standards: CI/CD, testing, observability, security, and cost control.
• Coach client and delivery team members, and communicate status, risks, and design decisions to stakeholders.
• Design end-to-end data solutions across multiple platforms, applying industry standards and best practices to meet client needs.
• Lead the development of data platform architectures, including cognitive components and unstructured data technologies, annotation, and related data and analytics.
• Manage and maintain large data repositories, ensuring scalability and performance.
• Create data solutions on cloud-native platforms (AWS, with GCP for ML/GenAI), tailored to client requirements.
• Collaborate with clients to understand their needs and deliver customized data solutions that meet their business objectives.

Conocimientos

Databricks

AWS

Python

SQL

Data engineering

Solution architecture

CI/CD

Kafka

Herramientas

Databricks

AWS

SQL

Python

Terraform

Fivetran

Descripción del empleo

IBM Consulting seeks a Data Engineer embedded with client teams to design and deliver hands-on solutions on the AWS/Databricks platform. You will migrate legacy workloads, re-engineer data flows, and partner with SAP, GCP, and other cloud services to provide scalable data products.

You will design end-to-end data pipelines, lead architecture initiatives, ensure CI/CD, security, and cost controls, and mentor delivery teams.