Senior Data Engineer - Build Scalable Data Platforms (Remote)

Prima; London (KY); Hybrid; USD 90,000 - 120,000; Full time

Benefits offered by this job

Flexible work arrangement

Learning resources

Private healthcare

Gym discounts

Wellbeing programs

Mental health support

Job summary

A leading motor insurance provider in Kentucky is seeking a Senior Data Engineer to shape the architecture of data products and lead data transformation efforts. The successful candidate will be an expert in batch processing, distributed data processing, and Python programming, working with technologies like Kafka and AWS. This role also emphasizes engagement with data scientists, a passion for learning, and offers flexible work arrangements including the option to work from home or anywhere for 30 days a year.

Qualifications

• Expert in batch, distributed data processing and near real-time streaming data pipelines.
• Experience with Data Lake/Big Data Analytics platforms, preferably cloud-based solutions.
• Proficient in Python programming and software engineering best practices.
• Expertise in RDBMS, Data Warehousing, and NoSQL databases.
• Proficiency in CI/CD pipeline management and infrastructure as Code (IaC).

Responsibilities

• Shape the architecture of data products for analytics and data science.
• Lead data transformation by establishing best practices.
• Build reusable technology for data ingestion, storage, transformation, and serving.
• Engage with data scientists to refine data requirements.
• Stay updated on emerging technologies and engage in community-building activities.

Skills

Batch processing

Distributed data processing

Near real-time streaming

Python programming

Data modeling

DevOps

SQL

Data Governance

Tools

Kafka

Flink

Spark

Databricks

AWS

Redshift

PostgreSQL

Apache Oozie

Apache Airflow

Job description

A leading motor insurance provider in Kentucky is seeking a Senior Data Engineer to shape the architecture of data products and lead data transformation efforts. The successful candidate will be an expert in batch processing, distributed data processing, and Python programming, working with technologies like Kafka and AWS. This role also emphasizes engagement with data scientists, a passion for learning, and offers flexible work arrangements including the option to work from home or anywhere for 30 days a year.