Senior Data Engineer

Ciklum; Town of Poland (NY); On-site; USD 48,000 - 81,000; Full time

Benefits offered by this job

Remote work option

Premium medical with Luxmed

Udemy access

Language courses

Company-paid certifications

MyBenefit platform

Conferences and workshops

Internal mobility

Job summary

Ciklum in Poland is seeking a Senior Data Engineer to join a cross-functional team on a 26-week enterprise initiative. You will engineer an AI-assisted pipeline to generate structured documentation from a large codebase, integrating Tree-sitter AST outputs with Neo4j graphs and Qdrant vector stores.

Collaborate with AI/ML engineers, ensure data quality, security, and traceability in a regulated environment, and deliver outputs aligned with strict KPIs.

Qualifications

• 5+ years of Data Engineering experience building production-grade pipelines.
• Experience with graphs, vector storage, and data integration.
• Proficiency in Python and modern data engineering toolkits.

Responsibilities

• Build and optimize high-throughput data pipelines and ETL/ELT workflows.
• Design and maintain Neo4j graph schemas for code dependencies.
• Integrate Tree-sitter AST outputs with vector embeddings (Qdrant).
• Ensure data lineage, integrity, and cross-reference tracking.
• Collaborate with AI/ML engineers to improve retrieval quality.

Skills

Python

ETL/ELT pipelines

Data engineering

CI/CD

English (C1+)

Tools

Docker

Kubernetes

Neo4j

Qdrant

Job description

Job Description

Ciklum is looking for a Senior Data Engineer to join our team full-time in Poland. We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer technology that redefines industries and shapes the way people live.

About The Role

As a Senior Data Engineer, become a part of a cross-functional development team engineering experiences of tomorrow. The Legacy Code Semantic Documentation Project is a 26-week enterprise initiative for a global industrial automation leader. The primary objective is to engineer an automated, AI-assisted pipeline to generate structured, system-level Markdown documentation directly from an undocumented ~400K LOC codebase (spanning IEC 61131-3 languages and ANSI C/C++). Operating within a dedicated, zero-data-egress secure tenant, the technical architecture combines Tree-sitter AST parsing, Neo4j knowledge graphs, Qdrant vector search, and self-hosted open-weight LLMs (Llama 3.1 / Mixtral family) with NLI-based validation. Delivery is structured across 7 work packages executing over a 6-month period, governed by strict contractual KPIs: ≥95% code coverage, ≥92% NLI-verified factual precision, and ≥90% SME validation acceptance. All outputs align with EU Cyber Resilience Act (EU CRA) requirements for SBOM and source-level traceability.

Responsibilities

• Data Pipeline Architecture: Build and optimize high-throughput data processing pipelines linking static code analysis outputs, graph databases, vector indexes, and LLM orchestration layers
• Knowledge Graph Engineering: Design and maintain Neo4j graph schemas to map code dependencies, function calls, and module hierarchies across 400K+ LOC
• AST Integration: Integrate Abstract Syntax Tree (AST) parser outputs (Tree-sitter) into downstream knowledge graphs and vector embeddings (Qdrant)
• Data Lineage & Integrity: Implement bidirectional cross-reference tracking and graph snapshot versioning to ensure data consistency across batch documentation runs
• Pipeline Testing & Monitoring: Establish data validation protocols and automated testing for pipeline health, throughput, and state persistence
• Technical Collaboration: Partner closely with AI/ML Engineers and Architects to optimize retrieval quality and context assembly for RAG pipelines

Requirements

• Professional Experience: 5+ years of Data Engineering experience building complex, production-grade data pipelines and ETL/ELT workflows
• Domain Specialization (Must cover at least ONE key domain):
  • Option 1 (Static Analysis Focus): Hands-on experience with Abstract Syntax Tree (AST) parsing, static code analysis (e.g., Tree-sitter, language grammars), and code structure modeling
  • Option 2 (Knowledge Graph Focus): Direct experience with graph database design and optimization (Neo4j/Cypher), documentation processing pipelines, and vector search integration (Qdrant)
• Core Technical Stack: High proficiency in Python, graph query languages, vector storage, and modern data engineering toolkits
• Infrastructure & Deployment: Familiarity with containerized execution (Docker, Kubernetes) in secure, private, or air-gapped environments
• Quality & Engineering Standards: Strong focus on code quality, automated pipeline testing, CI/CD, and robust versioning
• Language: Full professional proficiency in English (C1+)

What’s in it for you?

• Strong community: Work alongside top professionals in a friendly, open-door environment
• Growth focus: Take on large-scale projects with a global impact and expand your expertise
• Tailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications
• Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands‑on experience with cutting‑edge technologies
• Flexibility: Enjoy flexibility – full remote working possibilities
• Care: We've got you covered with company-paid premium medical package through Luxmed
• Benefits: Access the MyBenefit cafeteria platform, allowing you to choose perks that best suit your lifestyle and needs

About Us

At Ciklum, we are always exploring innovations, empowering each other to achieve more, and engineering solutions that matter. With us, you’ll work with cutting-edge technologies, contribute to impactful projects, and be part of a One Team culture that values collaboration and progress. With delivery centers in Wrocław and Gdańsk, our 300+ professionals in Poland drive forward-thinking solutions for global clients. Join a community where collaboration sparks innovation—and your impact reaches millions.

Explore, empower, engineer with Ciklum!