Senior Data Engineer – Data Platform

OEConnection LLC; Town of Poland (NY); On-site; USD 180,000 - 230,000; Full time

Job summary

OEConnection LLC seeks a Senior Data Platform Engineer to build cloud-native data infrastructure on AWS. You will design reusable, self-service components for ingestion, curation, and governance, enabling scalable, observable data pipelines.

The role emphasizes production-grade platforms, data contracts, and observability, with opportunities to influence streaming integrations and AI-assisted features.

Qualifications

• 5+ years of experience as a Data Engineer or Platform Engineer with AWS focus.
• Proven track record on production-grade data platforms or complex data infra.
• Strong data modeling and ETL best practices.
• Proficient in Python for pipelines and automation.
• Experience with APIs, SFTP, or message queues.
• Hands-on with event-driven architectures and cloud-native pipelines.
• Experience with Kafka or AWS Kinesis for streaming data.
• Implement data quality rules and use DataZone catalog concepts.
• Terraform for IaC and observability tooling for monitoring.
• Clear communicator and cross-team collaboration.

Responsibilities

• Contribute to design and implementation of reusable ETL pipeline templates.
• Build self-service platform components that abstract infra complexity.
• Define data contracts and pipeline configuration schemas for onboarding.
• Implement data lineage tracking and observability tooling.
• Maintain CI/CD pipelines and infrastructure as code with Terraform.
• Shape future platform capabilities including streaming integrations and AI-assisted features.

Skills

AWS

Python

SQL

Terraform

CI/CD

Kafka

AWS Kinesis

S3

Glue

Athena

Lambda

Step Functions

DataZone

Soda Core

Observability tools

ETL best practices

Tools

Terraform

Jenkins

GitHub Actions

Kafka

AWS Kinesis

Job description

Since the year 2000 , OEC has grown globally to more than 1,000 employees with a double-digit revenue increase nearly every year. We provide a lively culture, employee rewards and recognition, and the opportunity to develop and implement innovative technology solutions.

About the Role

We're looking for a Senior Data Platform Engineer to join the team building our cloud-native data infrastructure on AWS. You will be an active contributor to the design and evolution of a platform that serves both data producers and consumers, providing reusable, self-service capabilities for ingestion, curation, aggregation, governance, access interfaces, and observability. This is a platform engineering role: the solutions you build are generic, shared infrastructure, not one-off pipelines.

Responsibilities

• Contribute to the design and implementation ofreusable ETL pipeline templates,scalable, serverless, and event-driven for processing and transforming large datasets from diverse sources
• Help build reusable, self-service platform components that abstract infrastructure complexity for data producers and consumers.
• Define and evolvestandards of data contractsand pipeline configuration schemas that standardize how teams onboard to the platform.
• Implementdata lineage tracking, and observability tooling to give producers and operators full visibility into platform health.
• Implement and maintainCI/CD pipelinesand infrastructure as code usingTerraform.
• Contribute to the technical direction of future platform capabilities, includingstreaming integrations, consumption interfaces, data democratization, andAI-assisted featuressuch as failure remediation and automated contract suggestions.

You Will Be a Great Fit If You

• Have5+ yearsof professional experience as a Data Engineer or Platform Engineer with strong focus onAWS services
• Have a proven track record of contributing toproduction-grade data platformsor complex data infrastructure.
• Understanddata modelingand have a solid grasp ofETL best practices.
• Are proficient inPythonfor pipeline development, automation, and scripting.
• Are familiar with data ingestion mechanisms such asAPIs,SFTP, ormessage queues.
• Have experience buildingevent-driven architecturesand scalable, cloud-native data pipelines.
• Have hands‑on experience withstreaming platformssuch as Kafka or AWS Kinesis.
• Implement and maintain data quality rules using Soda Core, integrate quality check results with theDataZonecatalog, and define the platform's approach to quality failure handling within the pipeline execution flow.
• Are comfortable building and managing infrastructure usingTerraform.
• Use observability tools to monitor logs, performance, and system health.
• Can communicate technical concepts clearly and contribute to cross‑team alignment.
• Cloud AWS:S3, Iceberg, DynamoDB, RDS, Glue, Athena, Lambda, Step Functions, EventBridge, Transfer Family, SageMaker, Lake Formation, DataZone, KMS, IAM Identity Center, SES
• Languages:Python, SQL
• Infrastructure as Code:Terraform
• CI/CD:Jenkins, GitHub Actions

Nice to Have

• Experience withDatadogfor cloud observability and alerting.
• Familiarity withAI-assisted developer toolssuch as Claude Code or similar.

If you have a passion for building robust, reusable data infrastructure and want to help shape a platform that will power data-driven products across the organization, we'd love to hear from you!

We have updated our Privacy Policy to comply with the new European Union data protection law. Please note that OEC is a US-based company, as such, your personal information supplied during the job application process by using this website may be sent to the US. By applying for this role with one of the companies in the OEC Group, you are consenting to the processing and retention of your personal data in accordance with the company's data protection policies. Please refer to our Candidate Privacy Policy for more information.