Senior Data Engineer

What you will do

As a Senior Data Engineer, you'll be instrumental in building and optimizing the data pipelines that power Lighthouse's core products and services. You'll collaborate with product teams to ensure seamless data delivery, enabling the development of innovative features and insights. You'll work on strategic data projects, ensuring high data quality and reliability, and play a crucial role in establishing Lighthouse as a leader in data-driven hospitality solutions.

Where you will have impact

• Design and develop scalable, reliable data pipelines using the Google Cloud stack.
• Optimise data pipelines for performance and scalability.
• Implement and maintain data governance frameworks, ensuring data accuracy, consistency, and compliance.
• Monitor and troubleshoot data pipeline issues, implementing proactive measures for reliability and performance.
• Collaborate with the DevOps team to automate deployments and improve developer experience on the data front.
• Work with data science and analytics teams to enable them to bring their research to production grade data solutions, using technologies like airflow, dbt or MLflow (but not limited to).
• As a part of a platform team, you will communicate effectively with teams across the entire engineering organisation, to provide them with reliable foundational data models and data tools.
• Mentor and provide technical guidance to other engineers working with data.

About our team

Lighthouse is not only a data-driven company, we are a data company. The heart of all our products is data. It enables hotels to make the right decisions and fuel our analytical solutions. Being a growth company enables us to regularly attract new and interesting datasets, which can unlock new product directions. Today we process billions of data points and +100TB of data on a daily basis, containing hotels' pricing information, search data, hotel bookings, etc.

The Data Platform team is part of the Data Organization (within Engineering). We manage state-of-the-art infrastructure, services and products and are constantly innovating to support all our business needs and products at scale.

Lighthouse's Data Platform team has the following responsibilities and goals:

• Scalability, reliability, and security: We make sure that our platform can handle increasing volumes of data and user demands, provide reliable access to data, and protect data from unauthorised access or loss.
• Enabling data-driven decisions and innovation: The ultimate goal of our team is to support the organisation's strategic objectives, whether that's making better business decisions based on data, developing new data-driven products or services, or improving operational efficiency.
• User empowerment: A data platform should not just be a tool for data experts, but for all users in the organisation. This includes providing user-friendly tools, clear documentation, and training to help users access and understand the data they need.
• Data governance and compliance: With the increasing importance and sensitivity of data, it's crucial to have clear policies and procedures for managing data, as well as ensuring compliance with data protection regulations.

What's in it for you?

• Flexible working environment: Work from home or at one of our global offices.
• Flexible time off: Autonomy to manage your work-life balance.
• Collaborative team: High-bar, friendly, creative, and passionate colleagues.
• Career development: Workshops, frameworks, tools, training, and processes to realize your full potential.
• Impactful work: Shape products relied on by 85,000+ users worldwide.
• Mobility options: Mobility budget or company car tailored to your needs.
• Comprehensive health insurance: Extensive coverage for you and your dependents.
• Private pension funding: Group insurance to secure your future.
• Competitive compensation: Proactively maintained to value your work.
• Referral bonuses: Earn rewards for bringing in new talent.

Who you are

• 4+ years of professional experience using Python, Java, or Scala for data processing (Python preferred).
• You stay up-to-date with industry trends, emerging technologies, and best practices in data engineering.
• Improve, manage, and teach standards for code maintainability and performance in code submitted and reviewed.
• Ship large features independently, generate architecture recommendations and have the ability to implement them.
• Great communication: Regularly achieve consensus amongst teams.

We welcome

• Familiarity with GCP, Kubernetes (GKE preferred), CI / CD tools (Gitlab CI preferred), familiarity with the concept of Lambda Architecture.
• Experience with Apache Beam or Apache Spark for distributed data processing or event sourcing technologies like Apache Kafka.
• Familiarity with monitoring tools like Grafana & Prometheus.

Technologies you will work with

Python, BigQuery, Airflow, Terraform, GCP, dbt, Kubernetes, Kafka, Atlan (feel free to ask us why these are the best fit for us).