Senior DataOps Engineer (#5858)

N-iX; Georgia; Hybrid; USD 120,000 - 180,000; Full time

Job summary

N-iX is seeking a Senior DataOps Engineer to accelerate Data & AI initiatives on a secure, hybrid AWS platform. You will design and operate a data foundation, build scalable pipelines, and enforce governance across the data estate.

The role emphasizes tokenization, real-time streaming, and a medallion lakehouse approach within a cross-functional team. Ideal candidates have 4+ years in Data Engineering/DataOps, strong AWS analytics skills, and hands-on experience with Iceberg, Spark, and Kafka.

Qualifications

• 4+ years of hands-on experience as a Data Engineer or DataOps Engineer building enterprise-grade data platforms and pipelines.
• Strong expertise with AWS Data Analytics stack: S3, Glue Data Catalog, Lake Formation, EMR, Athena, DataSync, MSK.
• Deep experience with Iceberg table format, cataloging, compaction, and schema evolution.
• Proficiency in Spark/PySpark for batch, micro-batch, and real-time processing.
• Experience with Medallion Lakehouse Architecture (Bronze/Silver/Gold).
• Tokenization and encryption at scale (e.g., Protegrity, Thales, FPE, or Spark UDF).
• Event-driven architectures & streaming: Kafka/MSK, Kafka Connect, Schema Registry, MirrorMaker2.

Responsibilities

• Design, implement, and maintain the AWS Data Platform foundation (S3 lake, Iceberg, Glue Catalog).
• Build and optimize scalable batch and streaming data pipelines using EMR Serverless, Spark/PySpark, Flink, and Athena.
• Implement data tokenization/de-identification pipelines prior to cloud transit.
• Manage real-time streaming architectures with MSK, Replicator, and Schema Registry.
• Apply Medallion Architecture for lakehouse data modeling and automate Iceberg registration.
• Coordinate data migration waves with DataSync and streaming configs.
• Configure data access control and governance with Lake Formation, Macie, and Informatica Axon/EDC.
• Develop CI/CD for data pipelines, schema-drift detection, and data contracts.
• Implement DataOps observability with CloudWatch and FinOps cost monitoring.
• Create runbooks, DR procedures, and cutover/rollback playbooks for platform hardening.

Skills

AWS Data Analytics

Apache Spark

Data Lakehouse

Kafka/MSK

Data Governance

Terraform/CDK

ETL Pipelines

Tokenization

Security & Compliance

PySpark

Schema Registry

GitLab CI/CD

Iceberg

Azure Data Services

Tools

Apache Iceberg

Apache Flink

AWS Glue Data Catalog

AWS Lake Formation

Amazon EMR Serverless

Amazon MSK

DataSync

Protegrity

Kafka Connect

Schema Registry

Terraform

AWS CDK

GitLab CI/CD

Job description

Work type:

Office/Remote

Technical Level:

Senior

Job Category:

Software Development

N-iX is looking for Senior DataOps Engineer to join the team

Client Overview:

Our client is an Azerbaijani telecommunications company, the largest mobile network operator in Azerbaijan. The main products are: Fixed telephony, Mobile telephony, Internet services, Wireless broadband, and Value-added services.

Project Objectives:

The primary goal is to accelerate the client's Data & AI initiatives via a secure, hybrid cloud foundation on AWS while systematically modernizing the IT estate as part of the cloud migration.

Key Project Objectives include:

• Cloud Foundation & Landing Zone: Deploy target hybrid network architectures, establishing a secure Landing Zone and hybrid Data/AI platforms on AWS.
• Security, Compliance & Governance: Operationalize on-prem tokenization (achieving zero raw PII in the cloud), resolve policy blockers to include AWS in the ISMS, and establish a Cloud Center of Excellence (CCoE) to govern Cloud adoption.
• AI Chatbot & Voicebot Design & Implementation: Develop and operationalize a flagship Customer Care Chatbot and Voicebot as the first hybrid-setup consumer.

Key Responsibilities:

• Design, implement, and maintain the AWS Data Platform foundation, including Amazon S3 lake layout, Apache Iceberg table format standardization, and AWS Glue Data Catalog integration.
• Build and optimize scalable batch and streaming data pipelines using Amazon EMR (Serverless and EMR-on-EKS), Apache Spark/PySpark, Apache Flink, and Amazon Athena with workgroup cost caps.
• Implement data tokenization and de-identification pipelines on the data side (PA-T) using Protegrity/Spark UDFs for batch, Spark Streaming for micro-batch, and Kafka Connect SMT for streaming PII masking prior to cloud transit.
• Build and manage real-time streaming architectures with Amazon MSK (Managed Streaming for Apache Kafka), MSK Replicator/MirrorMaker2, and Schema Registry integration.
• Implement Medallion Architecture (Bronze, Silver, Gold layers) for lakehouse data modeling, automating Iceberg table registration and backfill frameworks.
• Execute data migration waves across non-PII and PII datasets using AWS DataSync, automated register steps, and streaming migration configs.
• Configure data access control and governance models using AWS Lake Formation, cross-engine authorization, AWS Macie for PII detection, and Informatica Data Catalog (Axon, EDC, BDQ) integration.
• Build automated schema-drift identification components, GitLab CI/CD pipeline integration for dataset synchronization, and automated data contracts/circuit breakers.
• Implement DataOps observability, centralizing logging via Amazon CloudWatch, configuring FinOps cost & anomaly monitoring, and setting up automated alerts.
• Author technical documentation, operational runbooks, disaster recovery (DR) procedures, and cutover/rollback playbooks for data platform hardening.

Requirements:

Mandatory Technical Skills:

• 4+ years of hands-on experience as a Data Engineer or DataOps Engineer building enterprise-grade data platforms and pipelines.
• Strong expertise with AWS Data Analytics stack: Amazon S3, AWS Glue Data Catalog, AWS Lake Formation, Amazon EMR (EMR-on-EKS / Serverless), Amazon Athena, AWS DataSync, and Amazon MSK.
• Deep experience with Apache Iceberg table format, cataloging, compaction, and schema evolution.
• Proficient in Apache Spark / PySpark and Spark Streaming for batch, micro-batch, and real-time data processing.
• Strong experience in Medallion Lakehouse Architecture design and implementation (Bronze, Silver, Gold layers).
• Practical experience in implementing data tokenization and encryption at scale (e.g., Protegrity, Thales, FPE, or Spark UDF-based de-identification pipelines).
• Solid knowledge of event-driven architectures & streaming: Apache Kafka / Amazon MSK, Kafka Connect (SMT), Schema Registry, and MirrorMaker2 / MSK Replicator.
• Expertise with Relational Databases (Amazon RDS, PostgreSQL, Oracle) and data synchronization techniques.
• Hands-on experience with DataOps CI/CD & Automation: GitLab CI/CD, Infrastructure-as-Code (Terraform / AWS CDK), schema-drift detection, and data contract validation.
• Familiarity with data governance tools and enterprise data catalogs (e.g., Informatica Axon/EDC, AWS Lake Formation).

Strong Plus (Nice-to-Have Skills):

• AWS Certified Data Analytics - Specialty or AWS Certified Data Engineer - Associate.
• Experience in telecom domain data models, CDR processing, and high-throughput real-time telemetry.
• Experience with cloud-side tokenization/detokenization via Athena UDFs / AWS Lambda.
• Familiarity with containerization (Docker, EKS, Kubernetes) for big data runtimes.
• Experience with AWS Macie and FinOps cost-allocation/anomaly-detection frameworks.

Soft Skills & Team Fit:

• Strong critical thinking, problem-solving, and analytical skills.
• Excellent communication and collaboration skills to work closely with cross-functional teams (Data Science, Cloud/Platform, Security, Governance).
• Results-oriented, proactive mindset with strong ownership of deliverables within an Agile / Scrum framework.
• Upper-Intermediate+ English level (written and spoken).

We offer*:

• Flexible working format - remote, office-based or flexible
• A competitive salary and good compensation package
• Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
• Active tech communities with regular knowledge sharing

Project: Global biopharmaceutical company