Data Engineer

Prolific; London; Remote; GBP 40,000 - 80,000; Full time

Job summary

An established industry player is seeking a Data Engineer to enhance data workflows and drive impactful insights. This role involves building and optimizing scalable data pipelines, ensuring high-quality data access for teams. You will collaborate with data scientists and analysts, implementing best practices in data engineering. Join a mission-driven culture that values innovation and ethical data sourcing, while enjoying competitive compensation and remote working options. Be a part of a transformative journey in AI development, where your contributions will shape the future of technology.

Benefits

Competitive salary

Remote working

Benefits package

Opportunities for groundbreaking research

Qualifications

• 2+ years of experience in deploying production quality code with Python.
• Strong understanding of SQL and data warehouse best practices.

Responsibilities

• Build and maintain robust data pipelines for accurate data delivery.
• Create and maintain APIs for seamless integration between data systems.

Skills

Python

SQL

Data Pipeline Tools (Airflow, dbt)

API Development

Data Modelling

Analytical Skills

Problem-Solving

Education

Bachelor's Degree in Computer Science or related field

Tools

Snowflake

Redshift

Job description

Prolific is not just another player in the AI space – we are the architects of the human data infrastructure that's reshaping the landscape of AI development. In a world where foundational AI technologies are increasingly commoditized, it's the quality and diversity of human-generated data that truly differentiates products and models.

The role

We're looking for a Data Engineer to build and optimise scalable data pipelines, ensuring our teams have access to high quality, reliable data. You will work closely with data scientists, analysts and engineers to enhance data workflows and drive impact insights across the business.

What you’ll bring to the role

• Technical Expertise: 2+ years of hands on experience deploying production quality code with proficiency in Python for data processing and related packages.
• Data Infrastructure Knowledge: Deep understanding of SQL and analytical data warehouses (Snowflake, Redshift preferred) with proven experience implementing ETL/ELT best practices at scale.
• Pipeline Management: Hands on experience with data pipeline tools (Airflow, dbt) and strong ability to optimise for performance and reliability.
• API Development Experience: Ability to design and develop robust data APIs and services that expose data to applications, bridging analytical and operational systems.
• Data Modelling: Strong data modelling skills and familiarity with the Kimball methodology to create efficient, scalable data structures.
• Quality Focus: Commitment to continuously improving product quality, security, and performance through rigorous testing and code reviews.
• Documentation: Meticulous approach to creating and maintaining architecture and systems documentation.
• Collaborative Mindset: Ability to work across teams to understand and address diverse data needs while maintaining data integrity.
• Growth Orientation: Desire to continually keep up with advancements in data engineering practices and technologies.
• Problem-Solving: Exceptional analytical skills to troubleshoot complex data issues and implement effective solutions.
• Independence: Capability to ship medium features independently while contributing to the team's overall objectives.

What you’ll be doing in the role

• Data Pipeline Development: Build and maintain robust data pipelines from internal databases and SaaS applications ensuring timely and accurate data delivery.
• Data Warehouse Management: Maintain our data warehouse with high-quality, well-structured data that supports analytics and business operations.
• System Architecture: Design and implement scalable data infrastructure that accommodates our growing data volume and complexity.
• Data Service Development: Create and maintain APIs and micro services that expose data to applications, enabling seamless integration between data systems and business applications.
• Data Quality Assurance: Establish processes and tools to monitor data quality, identify issues, and implement fixes promptly.
• Technical Documentation: Create and maintain comprehensive documentation of data flows, models, and systems for knowledge sharing.
• Cross-functional Collaboration: Work closely with analytics, research, and product teams to ensure their data needs are addressed effectively.
• Best Practices Implementation: Implement and advocate for data engineering best practices across the organisation.
• System Scaling: Plan and execute system expansion as needed to support the company's growth and evolving analytical needs.
• Performance Optimisation: Continuously optimise data pipelines and warehouse performance to improve efficiency and reduce costs.
• Security and Compliance: Ensure all data systems adhere to security best practices and compliance requirements.

Why Prolific is a great place to work

We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.

We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breadth and the best of humanity.

Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research. Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture.