Senior Data Pipeline Engineer - AWS, Spark & DataOps

VedaPointe, Inc.; United States; On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Restaurant d'entreprise

Indemnités de stage/alternance

Job summary

VedaPointe, Inc. seeks a senior data pipeline engineer to join our Data Engineering and Analytics team in the United States.

You will help mature our data operations, design, build, and optimize data pipelines that handle healthcare data with care and security. Reporting to an architect and team lead, you’ll work across cross-functional groups to deliver robust data solutions, integrate upstream applications, and ensure scalable, reliable performance on AWS.

Qualifications

• Bachelor’s or Master’s degree in computer science, Information Technology, Statistics, or a related field.
• 5+ years of experience in data engineering with a modern tech stack.
• Strong proficiency in Python, Java, or Scala.
• Advanced SQL skills and experience with PostgreSQL and MS SQL Server; dbt preferred.
• Hands-on experience with AWS and cloud data warehouses like Redshift.
• Experience with batch and streaming data integration (Kafka, Spark Streaming).
• Experience with data orchestration tools such as Airflow, Prefect, or AWS Step Functions.
• Solid understanding of data modeling, architecture, and governance.
• Excellent problem-solving, communication, and collaboration.
• Experience designing data pipeline architectures including remote ingestion and governance.

Responsibilities

• Design, develop and maintain scalable data pipelines and ETL processes to support data acquisition, integration and delivery for upstream applications, analytics, and other data-driven initiatives
• Design and implement advanced data storage solutions (databases, data warehouses, and data lakes) and efficient data transport layers (REST API’s, message queues, message buses)
• Collaborate with executive leadership, product owners, technical architects, development teams, governance stakeholders, and other key personnel to gather, define, and document technical requirements for data pipelines that align with business objectives
• Create and maintain analytics dashboards providing actionable insights into customer acquisition, operational efficiency, and other critical business performance metrics
• Continuously optimize data architectures, data transformations, and data pipelines to enhance performance, reliability and scalability of the system.
• Apply software engineering best practices to ensure data quality, integrity, validity, availability, and compliance
• Proactively identify, troubleshoot, and resolve production data-related issues
• Create high-impact data tools for analytics and data science teams that help others move faster
• Participate in design, code development, code reviews, testing, data quality monitoring, deployment activities, and operations support
• Contribute to your overall team growth by staying current with and evaluating emerging data technologies and industry trends while sharing with colleagues

Skills

Python/Java/Scala

SQL & relational DBs

Data pipelines & ETL

Data governance & modeling

Communication & collaboration

Education

Bachelor’s or Master’s degree in computer science/IT/statistics

Tools

dbt

AWS

Kafka

Spark Streaming

Airflow

Prefect

Airbyte

DMS

Debezium

Kubernetes

Terraform

CloudFormation

CI/CD tooling

HL7/FHIR data healthcare

Job description

VedaPointe, Inc. seeks a senior data pipeline engineer to join our Data Engineering and Analytics team in the United States.

You will help mature our data operations, design, build, and optimize data pipelines that handle healthcare data with care and security. Reporting to an architect and team lead, you’ll work across cross-functional groups to deliver robust data solutions, integrate upstream applications, and ensure scalable, reliable performance on AWS.