Korean Bilingual Data Engineer

BTI Solutions; Plano (TX); Sur place; USD 110 000 - 150 000; Plein temps

Résumé du poste

BTI Solutions seeks a mid-level Data Platform Engineer to build a new greenfield data platform integrating internal systems using a Medallion Architecture. You will migrate Alteryx workflows to code-based dbt and Python, and develop an MCP server for AI agents to query data marts, supporting AI-driven analytics.

You will own end-to-end pipelines from ingestion through Bronze/Silver/Gold, manage metadata with OpenMetadata, and automate Excel reporting with Python.

Qualifications

• 3+ years of hands-on data engineering in production environments.
• Advanced SQL skills with window functions, CTEs, and query tuning.
• Hands-on dbt experience: model design, incremental strategies, tests.
• Python proficiency for data processing and automation (pandas).
• Experience with Trino/Presto, Spark, or DuckDB.
• Experience with Iceberg, Delta Lake, or Hudi.
• Experience with Medallion Architecture and star schema modeling.
• Experience integrating SQL Server and Oracle using CDC or batch processing.

Responsabilités

• Ingest data from SQL Server and Oracle into Bronze with CDC-based ingestion.
• Develop dbt models for Silver: cleansing, standardization, deduplication, quality tests.
• Build Gold data marts and migrate Alteryx transformations to SQL/dbt.
• Configure Apache Iceberg tables and REST Catalog with schema evolution and time travel.
• Create and maintain Airflow DAGs for dependencies, scheduling, and alerts.
• Develop MCP servers to support AI agents querying Gold data marts.
• Manage metadata and lineage using OpenMetadata.
• Automate Excel reporting using Python and openpyxl.

Connaissances

SQL

dbt

Python

Trino/Presto

Apache Spark

DuckDB

Apache Iceberg

Delta Lake

Medallion Architecture

CDC ingestion

Airflow

OpenMetadata

Outils

Airflow

OpenMetadata

Iceberg REST Catalog

Description du poste

We are seeking a mid-level Data Platform Engineer to build a new greenfield data platform integrating internal systems (GSCM, SEASI, SMART, and BOSS) using a Medallion Architecture. This role owns the end-to-end data pipeline—from ingestion through Bronze/Silver/Gold transformation—while migrating existing Alteryx workflows to code-based dbt and Python implementations. The engineer will also develop an MCP server enabling AI agents to query business-specific data marts, directly supporting the organization's AI-driven analytics initiatives. Success in this role will deliver a scalable, governed data foundation that accelerates business insights and AI adoption.

Responsibilities

• Ingest data from sources such as Microsoft SQL Server (BOSS) and Oracle into the Bronze layer, implementing CDC-based incremental ingestion
• Develop dbt models for the Silver layer including cleansing, standardization, deduplication, and data quality testing
• Develop Gold layer business-specific data marts and migrate existing Alteryx transformation logic
• Configure Apache Iceberg tables and a REST Catalog, including schema evolution and time travel support
• Develop and operate Apache Airflow DAGs, managing dependencies, scheduling, and failure notifications
• Develop an MCP server enabling AI agents to query Gold-layer data marts, including catalog/metadata access, ontology integration, and query execution
• Manage metadata and data lineage using OpenMetadata
• Automate Excel reporting using Python and openpyxl

Required Qualifications

• 3+ years of hands-on data engineering experience in production environments
• Advanced SQL skills, including window functions, CTEs, and query tuning
• Hands-on dbt experience, including model design, incremental strategies, macros, and tests
• Python proficiency for data processing and automation, including libraries such as pandas
• Experience with at least one analytical engine: Trino/Presto, Apache Spark, or DuckDB
• Experience with at least one lakehouse table format: Apache Iceberg, Delta Lake, or Apache Hudi
• Experience with Medallion Architecture and star schema/dimensional modeling
• Experience integrating relational database sources, such as Microsoft SQL Server and Oracle, using CDC or batch processing

Preferred Qualifications

• Experience with EL tools such as Airbyte or Fivetran, and Iceberg REST Catalog implementations such as Apache Polaris
• Experience with governance tools such as OpenMetadata or DataHub, and semantic layers such as Cube.dev
• Ability to interpret Alteryx workflows (.yxmd) and migrate their logic to SQL/dbt
• Experience developing MCP servers or building LLM, RAG, or AI agent pipelines
• Experience developing applications, APIs, or backend services using Java and/or Python
• Experience building Generative AI (GenAI) applications, including LLM integration, retrieval-augmented generation (RAG), and AI agent workflows
• Hands-on experience with LangChain and/or LangGraph for developing tool-enabled AI agents and orchestrating multi-step workflows
• Understanding of retail and home appliance business domains, including PSI (Production, Sales, and Inventory) and sales analytics
• Understanding of retail and home appliance business domains, including PSI (Production, Sales, and Inventory) and sales analytics

Required Technical Skills

• SQL
• dbt
• Python
• Trino/Presto or Apache Spark or DuckDB
• Apache Iceberg or Delta Lake or Apache Hudi
• Microsoft SQL Server
• Oracle
• MinIO (S3-compatible storage)

Preferred Technical Skills

• Java or Python
• Generative AI (GenAI) and LLM integration