Lead Data Engineer

Inizio Partners Corp; New York (NY); Presencial; USD 110.000 - 150.000; Jornada completa

Descripción de la vacante

Inizio Partners Corp. is seeking a data engineering/analytics professional to interface with stakeholders, discover data landscapes, and design robust data pipelines. You will translate requirements into scalable data models, implement ETL/ELT processes, and ensure data quality through automated tests and monitoring.

The role emphasizes end-to-end pipeline ownership, collaboration in an Agile environment, and clear documentation for governance and onboarding across global teams.

Formación

• Bachelor’s or Master’s degree in Computer Science, Engineering, OR related analytics field.
• Strong Python and SQL proficiency with complex queries and tuning.
• Hands-on experience building end-to-end data pipelines (batch and near real time).
• Proven data transformation/ETL/ELT skills: incremental loads, partitioning, CDC.

Responsabilidades

• Interface with business stakeholders to understand data needs and align delivery.
• Lead discovery of current reporting and data landscape, mapping sources and metrics.
• Design target data pipelines with data flow diagrams, lineage, and models.
• Develop and optimize Python/SQL code for ETL/ELT processes and data integration.
• Build reusable Python libraries and parameterized pipelines for consistency.
• Implement data quality checks, automated tests, and monitoring for data trust.

Conocimientos

Python

SQL

Data pipelines

ETL/ELT

Data quality checks

Agile planning

Communication

Cross-cultural collaboration

Educación

Bachelor's/Master's in CS/Engineering

Herramientas

Microsoft Word

Excel

PowerPoint

Descripción del empleo

Role and Responsibilities:

• Interface with business stakeholders, understand their data and analytics needs, establish requirement in alignment with data product owners and align on delivery plan.
• Lead discovery of the current reporting and data landscape: inventory sources, ETL processes, upstream/downstream dependencies, and existing metrics/definition
• Translate stakeholder requirements and current-state findings into a target pipeline design (batch and near-real time where applicable), including data flow diagrams, lineage, and logical/physical model
• Develop and maintain end to end data pipelines: author, tune, and optimize Python and SQL code for extraction, transformation, and loading (ETL/ELT), including incremental loads, partitioning, and CDC patterns
• Build reusable Python libraries and parameterized pipelines to accelerate delivery and ensure consistency
• Implement data quality, validation, and reconciliation processes: automated tests, reconciliation scripts, anomaly detection, and monitoring/alerting to ensure trust in the data
• Define and codify metric logic, mapping, and lineage; implement validation tests and documentation for each metric to support governance and auditability
• Own UAT for new/changed data assets: create test plans, run tests, document findings, and provide pass/fail recommendations to IT and engineering partners
• Prepare rollout materials and lead change adoption: runbooks, operational runbooks, deployment guides, knowledge transfer sessions, and training for analysts and business users
• Support table retirement and technical debt reduction by coordinating deprecation plans and ensuring no regression in reporting
• Collaborate in an Agile environment with peers and stakeholders; participate in planning, standups, and retrospectives to deliver iterative value
• Rigorously create and maintain document such that anyone can understand the design, logic, code, acceptance testing, data quality testing, and outcomes. Document prototypes in detail, including learnings of what doesn't work.
• Effectively communicate analysis, reports, project status, milestones etc. in coordination with data product owners, program managers and scrum maters.
• Implement straight through, exception-based data product designs meeting data governance standards, including reporting. Perform necessary data quality checking tasks, prepare data quality reports, establish necessary data security measures to prepare model ready data.

Candidate Profile:

• Bachelor's/master's degree in computer science/engineering, operations research, or related analytics areas
• Deep expertise in Python and SQL, including complex query authoring and performance tuning.
• Hands on experience building and maintaining end to end data pipelines (batch and near real time).
• Proficient in data transformation/ETL/ELT: incremental loads, partitioning, CDC.
• Metric development: define logic, lineage, and validation tests.
• Implement data quality checks, automated tests, and reconciliation scripts.
• Knowledge of data warehousing and logical modeling
• Highly proficient in Microsoft Word, Excel, PowerPoint
• Exposure to Agile planning concepts: Kanban, Sprint.
• Able to work in a fast paced and continuously evolving environment with the ability to take-up challenging tasks.
• Can understand cross cultural differences and can collaborate with clients across the globe.
• Outstanding written and verbal communication skills