Senior Data Engineer: Databricks, Spark & Data Platform Expert

Rapsys Technologies Pte Ltd.; Pittsburgh (Allegheny County); Presencial; USD 140.000 - 190.000; Jornada completa

Descripción de la vacante

Rapsys Technologies Pte Ltd. in Pittsburgh is seeking an experienced Senior Data Engineer to design, develop, and maintain secure data platforms for enterprise analytics.

You will build end-to-end data pipelines using Databricks, Spark, Python, SQL, and Azure Data Factory, while ensuring governance and security. You will contribute to BI and AI initiatives, implement streaming architectures with Kafka, and collaborate with cross-functional teams to deliver scalable data solutions in a fast-paced

Formación

• Bachelor's degree or higher in a technical field or equivalent experience.
• 6+ years of data engineering experience in enterprise platforms.
• Hands-on with Databricks, Spark, Python, SQL, and Azure Data Factory.
• Experience with Kafka or similar event streaming and data integration tools.

Responsabilidades

• Design, develop, test, deploy, and maintain enterprise-scale data solutions.
• Build and optimize ETL/ELT pipelines with Databricks, Spark, Delta Lake, and ADF.
• Develop scalable ingestion, transformation, and integration frameworks for analytics.
• Write and optimize SQL queries, stored procedures, and transformations.
• Design scalable data models for BI, analytics and AI.
• Develop and integrate REST APIs, event-driven architectures, and SOAP services.
• Build and maintain streaming and messaging solutions with Kafka.
• Monitor production pipelines, perform root cause analysis, and implement solutions.
• Implement data quality, governance, security, and performance optimization.
• Manage source code with Git and follow CI/CD practices.
• Collaborate with architects, product owners, and cross-functional teams.
• Participate in Agile ceremonies and maintain runbooks and docs.
• Mentor junior engineers and contribute to engineering standards.

Conocimientos

Databricks

Spark PySpark

Python

SQL

Azure Data Factory

Delta Lake

Kafka

Data governance

DevOps

Educación

Bachelor's degree in Computer Science, Information Systems, Engineering, or related field

Herramientas

Delta Live Tables

Unity Catalog

MLflow

Databricks SQL

Git

Linux

Descripción del empleo

Rapsys Technologies Pte Ltd. in Pittsburgh is seeking an experienced Senior Data Engineer to design, develop, and maintain secure data platforms for enterprise analytics.

You will build end-to-end data pipelines using Databricks, Spark, Python, SQL, and Azure Data Factory, while ensuring governance and security. You will contribute to BI and AI initiatives, implement streaming architectures with Kafka, and collaborate with cross-functional teams to deliver scalable data solutions in a fast-paced