Senior Data Engineer

Jobtailor; Atlanta (MO); On-site; USD 150,000 - 200,000; Full time

Job summary

Jobtailor is seeking a senior data engineer to architect and build our core data platform. You will drive ideation, design, and development of platform components and connectors across on-prem and cloud sources.

You will create SDKs and libraries to enable scalable data services and ensure data quality, governance, and observability. The role emphasizes end-to-end data platforms, multi-cloud experience (GCP/Azure), and hands-on Databricks/Delta Lake, with a focus on delivering robust, scalable

Qualifications

• 5+ years of proven experience in modern cloud data engineering and software engineering.
• Strong ability to build end-to-end data platforms and data services (beyond ETL).
• Experience with multi-cloud environments (GCP / Azure).
• Hands-on experience with Databricks (Delta Lake, Spark).
• Excellent experience with data modeling, governance, and observability.

Responsibilities

• Ideate, architect, design and develop key data platform components.
• Create and maintain data platform SDKs and libraries.
• Design connectors to source data from disparate systems (on-prem and cloud).
• Optimize storage, processing, and querying for large data sets.
• Develop data quality, governance, and observability frameworks.

Job description

Responsibilities

• As a senior data engineer, you will be responsible for ideation, architecture, design and development of our key data platform components
• Create and maintain essential data platform SDKs and libraries, adhering to industry best practices
• Design and develop connector frameworks and modern connectors to source data from disparate systems both on-prem and cloud
• Design and optimize data storage, processing, and querying performance for large-scale datasets using industry best practices while keeping costs in check
• Design and develop data quality frameworks and processes to ensure the accuracy and reliability of data
• Collaborate with data scientists, analysts, and cross functional teams to design data models, database schemas and data storage solutions
• Proactively identify and contribute towards platform resiliency
• Design and develop observability and data governance frameworks and practices
• Stay up to date with the latest data on engineering trends, technologies, and best practices
• Drive the deployment and release cycles, ensuring a robust and scalable platform
• Partner with AI enablement teams across the organization as well as KION IT Cloud Infrastructure, AI Platform, and security teams to ensure AI/ML capabilities align with IT framework and guidelines within KION
• Partner with Business Transformation Data Management teams to align on master data

Requirements

• 5+ years of proven experience in modern cloud data engineering and software engineering
• Proven ability to build end-to-end data platforms and data services (beyond ETL)
• Strong cloud experience, preferably GCP; Azure (ADLS) and Databricks are strong pluses
• Hands‑on experience with Databricks (Delta Lake, Spark, ML/ETL workflows)
• Experience working in multi-cloud environments (GCP / Azure)
• Proficiency with platforms such as BigQuery, Dataflow, Dataform, Cloud Run, DBT, Dataproc, SQL, Python, Airflow, Pub/Sub, and equivalent Azure tooling such as ADLS, Azure Functions, and Databricks
• Experience with microservices architectures (Kubernetes, Docker)
• Deep experience with batch and streaming data infrastructures
• Strong hands‑on experience with metadata management, data catalogs, data lineage, data quality, and data observability frameworks
• Strong understanding of data modeling, data architecture, and data governance
• Solid experience with DataOps, CI/CD, and test automation
• Excellent experience with observability tooling
• Experience building data platforms supporting AI use cases and machine learning
• Production level experience with universal semantic layers
• Production level experience with implementing either 3rd party or open-source metadata management platforms
• Production level experience with data platform resiliency
• Experience building large-scale data platforms on Azure, including Azure Data Lake and Databricks