Middle Data Engineer (6 months' engagement)

N-iX; United States; Hybrid; USD 90,000 - 130,000; Full time

Benefits offered by this job

Flexible working format

Competitive salary

Mentorship & tech trainings

Active tech communities

Job summary

N-iX is seeking a Middle Data Engineer for a 6-month engagement to own data readiness for Fortune 100-scale datasets. You will design and run end-to-end data pipelines, validate sources and mappings, and deliver ready datasets to downstream consumers.

You will work with Snowflake/Databricks environments, write complex SQL, and collaborate with product teams across MDFA, CDF, WPP, and market stakeholders to ensure reliable data delivery and performance.

Qualifications

• 5–6 years of professional engineering experience with independent production ownership.
• Strong Python, PySpark, and SQL with complex transformations and data quality checks.
• ETL/ELT and batch-pipeline engineering across warehouses, APIs, and object storage.
• Experience with a cloud data lake/lakehouse, Medallion patterns, schema contracts, and observability.
• Solid Git practices: PRs, reviews, tests, and deployment controls.

Responsibilities

• Design, develop, test, deploy, and optimize ingestion and data-processing workflows.
• Develop and tune Python/PySpark packages, SQL queries, Databricks jobs, and ADF pipelines.
• Own technical readiness for new data cells and refreshes; validate mappings and outputs.
• Investigate data quality, taxonomy, hierarchy, and performance issues across multiple sources.
• Coordinate technical validation with Product, markets, and data providers; drive issue resolution.

Skills

Python

PySpark

SQL

ETL/ELT

Data pipelines

Git

Problem solving

Stakeholder communication

Tools

Databricks

Azure Data Factory

Delta tables

Power BI

Job description

Poland, Ukraine

Work type:

Office/Remote

Technical Level:

Middle

Job Category:

Software Development

Project:

Fortune 100 multinational conglomerate

N-iX is looking for a Middle Data engineer for 6 months' engagement.

You will act as the independent hands-on engineer and the technical owner of data readiness. The role combines pipeline and query engineering with investigation of source, mapping, hierarchy, and validation issues; coordination of market and provider validation cycles; reconciliation of outputs; and delivery of technically ready datasets to downstream consumers.

The Data Engineering team builds and maintains pipelines, while mappings and configurations determine cell/model scope; Snowflake extraction jobs run SQL through Databricks, with Blob Storage, validation, Medallion transformations, Power BI review, and Gold delivery to Ekimetrics. The operating model also requires end-to-end triage across MDFA, CDF, WPP, Data Foundation, Redmill, and market stakeholders rather than treating every discrepancy as a coding defect.

Key responsibilities:

• Independently design, develop, maintain, test, deploy, and optimize ingestion, validation, transformation, aggregation, data-quality, anomaly-detection, and extraction workflows
• Develop and tune Python/PySpark packages, SQL extraction queries, Databricks jobs, ADF pipelines, Delta tables, and source-to-target contracts
• Own technical readiness for new cells and refreshes: clarify filters and expected scope, implement or update queries, review mappings/configuration, reconcile outputs, and confirm readiness for business validation
• Investigate source, NCID, taxonomy, hierarchy, mapping, naming-convention, aggregation, missing-week, outlier, and performance issues across CDF, MDFA/PFME, APIs, manual files, and specific sources
• Coordinate technical validation cycles with Product, CMIA/markets, WPP/MDFA, CDF, other data providers, and Ekimetrics; convert reported discrepancies into actionable owners and technical evidence
• Drive complex incident resolution across pipelines, data contracts, access, service principals, secrets, networking, Power BI refreshes, and downstream delivery
• Review pull requests and test evidence; guide the Junior engineer and delegate scoped engineering/support work without becoming a people manager
• Maintain architecture documentation, interface contracts, repository documentation, runbooks, deployment procedures, and support/escalation guidance
• Recommend practical automation, reliability, performance, and maintainability improvements while respecting platform standards and business-validation ownership.

Must-have technical competencies:

• 5-6 years' professional engineering experience with independent production ownership
• Strong Python, PySpark, and SQL, including complex transformations, query optimization, reusable packages, debugging, tests, and reconciliation
• Strong ETL/ELT and batch-pipeline engineering across relational warehouses, APIs, object storage, and file-based ingestion
• Experience with a cloud data lake/lakehouse, Medallion patterns, schema/interface contracts, data-quality controls, orchestration, observability, and incident recovery
• Solid Git engineering practices: branching, pull requests, reviews, automated tests, deployment controls, and documentation
• Proven ability to translate business/data requirements into filters, mappings, transformations, validation rules, and operational workflows
• Stakeholder-facing problem solving: explain discrepancies, challenge incomplete requirements, establish technical owners, and drive issues to closure.

Nice-to-have:

• Valuable experience with Pandera or equivalent schema-validation frameworks, Streamlit, Managed Identity/service principals, Azure Communication Services, Managed VNet/Private Endpoints, and Dev/Prod release practices
• Experience with PFME/media data, syndicated sales, marketing hierarchies, MMM inputs, or multi-market data onboarding is preferred but can be learned.

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing

Project: Global biopharmaceutical company

Project: Global stock photography provider