Principal Data Engineer -Python / AI

Arka Innovate; Atlanta, Northern (GA, KY); Hybrid; USD 180,000 - 240,000; Full time

Job summary

ARKA INNOVATE is seeking a Principal Data Engineer with Python and AI experience to design scalable, production-grade data services that leverage LLMs and modern backend architectures. You will collaborate with cross-functional teams to deliver high-performance solutions, implement robust data contracts, and ensure secure, observable systems across cloud-based infrastructure and NoSQL/SQL stores.

The role emphasizes hands-on leadership, advanced data modeling (Pydantic), and building

Qualifications

• 15+ years of IT industry experience.
• Strong backend engineering in Python including OOP, typing, and async programming.
• Hands-on experience with LLMs or AI-powered systems (information extraction, semantic search, classification).
• Knowledge of agentic architectures, tool/API calling agents, and orchestration patterns.
• Data modeling with type-safe models (e.g., Pydantic) and robust data contracts.
• Experience building FastAPI-based services with event-driven integrations and async pipelines.
• Exposure to high-scale data systems, performance optimization, and efficient I/O handling.
• Solid API security concepts (AuthN/AuthZ, OAuth2/OpenID Connect).
• Experience with SQL databases (SQL Server, Azure SQL, Oracle, PostgreSQL).

Responsibilities

• Design scalable production data services leveraging Python and AI.
• Collaborate with cross-functional teams to deliver impactful solutions.
• Develop and maintain robust data contracts and high-quality APIs.
• Implement monitoring, logging, and testing to ensure reliability.
• Lead by example in debugging, metrics interpretation, and troubleshooting complex systems.

Skills

Python

Backend engineering

LLMs/AI

Data modeling

FastAPI

Async programming

Cloud infrastructure

SQL databases

Debugging

API security

Tools

Pydantic

CI/CD

Webhooks

Message queues

Job description

ARKA INNOVATE is hiring on behalf of our client for a Principal Data Engineer with Python/AI.

Role:

We are seeking a highly skilled Principal Data Engineer with strong experience in Python and AI-driven application development. The ideal candidate will have hands‑on experience building scalable production systems that leverage LLMs and modern backend architectures. This role involves designing intelligent, high-performance services and collaborating closely with cross‑functional teams to deliver impactful solution.

Qualifications & Requirements

Minimum 15+ years of experience in IT Industry.

Strong backend engineering expertise in Python, including:

• Object-Oriented Programming (OOP)
• Type hinting and modern typing practices
• Asynchronous programming (async/await)
• Proven experience working on complex, production-grade systems

Hands‑on experience building applications powered by LLMs or advanced AI, such as:

• Information extraction
• Semantic search
• Classification and matching systems

Solid understanding of agentic architectures and orchestration patterns, including:

• Tool/API calling agents
• Background processing workflows
• Job runners and schedulers

Strong experience in data modeling and schema design, including:

• Use of type‑safe models (e.g., Pydantic or similar)
• Designing robust data contracts

Experience building and consuming FastAPI-based services, including:

• Event-driven integrations (webhooks, message queues)
• Asynchronous processing pipelines

Ability to work with high-scale data systems, including:

• Performance optimization
• Efficient I/O handling
• Batching and vectorized operations

Strong debugging and problem‑solving skills:

• Log analysis
• Metrics interpretation
• Experiment‑driven troubleshooting in complex systems

Experience with cloud‑based infrastructure and data stores, such as:

• Object storage systems
• NoSQL/document databases
• Search/indexing systems
• Exposure to any major cloud provider

Proficiency with modern Python development tools:

• Dependency management
• Virtual environments
• Testing frameworks
• Linters/formatters
• CI/CD integration

Strong understanding of API security, including:

• Authentication and Authorization (AuthN/AuthZ)
• OAuth2 / OpenID Connect flows

Solid experience with SQL databases, including:

• SQL Server
• Azure SQL
• Oracle
• PostgreSQL

Excellent communication skills:

Translate business needs into technical solutionskills.

Ability to ask the right questions

Challenge ambiguous requirements

At Arka Innovate, we are proud to be an Equal Opportunity Employer. We are committed to creating an inclusive environment for all employees and do not discriminate on the basis of race, color, religion, gender, sexual orientation, national origin, age, disability, veteran status, or any other legally protected status.