Lead Data Engineer - Scalable Analytics & Pipelines

eositsolutions; Burlingame (CA); On-site; USD 135,000 - 150,000; Full time

Job summary

EOS IT Solutions is seeking a Senior Data Engineer in Burlingame, CA to lead data initiatives and build scalable analytics pipelines. You will organize and curate large datasets, design robust data architectures, and ensure secure, production-ready data services.

You will collaborate with the RL Data Engineering team, onboard new datasets end-to-end, maintain batch jobs and dashboards, and contribute to runbooks, CI/CD, and data quality improvements while partnering with stakeholders to meet

Qualifications

• Degree in Computer Science, Engineering, Mathematics or 7+ years industry experience.
• Demonstrated strength in data modelling, ETL development, and data warehousing experience with Oracle, Redshift, PostgreSQL, etc.
• Advanced SQL and query performance tuning skills.
• Experience building data products incrementally and integrating and managing data sets from multiple sources.
• Coding proficiency in Python programming language.
• Experience with scheduled batch orchestration (e.g. Airflow) and partitioned lake/warehouse tables.
• Experience with CI/CD, version control, and task tracking; ability to own production issues and write clear runbooks.
• Experience building operational dashboards and reporting on delivery KPIs (freshness, completeness, on-time delivery, accuracy).
• Exceptional communication and stakeholder management skills; self-motivated in an autonomous, fast-paced environment.
• Ability to work on-site in Burlingame, CA per assignment

Responsibilities

• Design, implement and operate large-scale, centralized high-volume, high-performance data structures for analytics and data science.
• Implement data ingestion routines both real time and batch using best practices in data modeling, ETL/ELT processes.
• Work with Project Manager to understand business and functional requirements and translate these requirements into robust, scalable, operable solutions with a flexible and adaptable data architecture.
• Assist SWE team with identifying in code key events and data to be logged to populate the data structures used for dashboarding.
• Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service modeling and production support for customers.
• Onboard new datasets end to end: requirements, access provisioning, packaging and CI/CD configuration, partition/snapshot validation, go-live checks.
• Own recurring batch jobs - monitoring, triage, test plans, and partition recovery - including backfills via notebooks and SQL.
• Build and maintain flow/funnel, quality-check, and operations-health dashboards in the team's BI platform; maintain internal web tools (bug fixes, duplicate-record cleanup, prod/test tagging).
• Author and maintain runbooks, end-to-end documentation, release notes, and user guides; keep them current with a scheduled audit cadence.

Skills

Data modelling

ETL development

Data warehousing

Advanced SQL

Python

Airflow

CI/CD

Runbooks

Dashboards

Stakeholder management

On-site Burlingame

Education

Degree in Computer Science, Engineering, Mathematics

Tools

Oracle

Redshift

PostgreSQL

Airflow

CI/CD tooling

Git

Job description

EOS IT Solutions is seeking a Senior Data Engineer in Burlingame, CA to lead data initiatives and build scalable analytics pipelines. You will organize and curate large datasets, design robust data architectures, and ensure secure, production-ready data services.

You will collaborate with the RL Data Engineering team, onboard new datasets end-to-end, maintain batch jobs and dashboards, and contribute to runbooks, CI/CD, and data quality improvements while partnering with stakeholders to meet