Principal, Solution Architecture (Kafka & Flink)

New York Technology Partners; Chicago (IL); Vor Ort; USD 140.000 - 190.000; Vollzeit

Zusammenfassung

New York Technology Partners is seeking a Solution Architect to design and govern large-scale distributed systems across cloud and on-prem environments. You will drive event-driven and microservices patterns, define standards, and collaborate with DevOps, security, and data teams.

You’ll translate business requirements into robust architectures, review designs, mentor engineers, and ensure performance, reliability, and regulatory alignment across the capital markets domain.

Qualifikationen

• BS degree in Computer Science or a related field.
• Experience designing architectures across distributed/enterprise platforms; microservices and streaming/batch apps.
• Kubernetes deployment topology for stateless services with multi-AZ resilience.

Aufgaben

• Design applications’ architecture across distributed platforms using event-driven and microservices patterns.
• Define and enforce architectural standards across engineering teams.
• Oversee data architecture for streaming, batch, and message-based processing.

Kenntnisse

Event-driven design

Distributed systems

Java/Python/C#

Kafka

Cloud (AWS)

Infrastructure as Code (Terraform)

DevOps tooling

Architecture principles

Ausbildung

BS in Computer Science or equivalent

Tools

Terraform

Kubernetes

Git

Jenkins

Docker

ArgoCD

Protocol Buffers

Jobbeschreibung

We are building a culture that empowers continued learning, authentic innovation, and constant creativity. We are looking for an individual to join our team as a Solution Architect with the following skills and abilities:

Primary Duties and Responsibilities:

To perform this job successfully, an individual must be able to perform each primary duty satisfactorily.

What You’ll Do:

You’ll be working alongside experts that are building next generation clearing system; and paving the future of digital transformation in the capital markets industry by:

• Design the applications’ architecture across distributed and enterprise technology platforms while leveraging patterns such as event-driven architecture, publish/subscribe messaging, and microservices while using infrastructure-as-code (IaC) practices to enable consistent, repeatable, and automated provisioning across environments.
• Define and enforce architectural standards, patterns, and best practices across engineering teams.
• Design streaming and batch-based systems that support both real-time and scheduled data flow across the organization.
• Shape and guide the architecture of message-based systems utilizing Kafka or similar platforms to enable reliable, decoupled, event-driven communication between services.
• Develop and oversee data architecture and integration strategies spanning streaming, batch, and message-based processing to ensure reliable, secure, and efficient data flow across systems.
• Own end-to-end solution architecture from ideation through implementation, including documentation of architectural decisions and system diagrams.
• Guide the development and deployment of distributed, cloud-native services, ensuring resilience, scalability, and cost-efficiency.
• Collaborate with DevOps, platform engineering, and security teams to ensure architectures are operable, resilient, and aligned with organizational standards.
• Review designs and code from engineering teams to ensure alignment with architectural standards, security requirements, and regulatory/compliance needs.
• Troubleshoot and resolve complex, cross-system issues spanning application, infrastructure, messaging, and data layers.
• Partner with engineering, data, and business teams to translate business requirements into robust, scalable technical solutions.
• Translate complex architectural decisions and trade-offs into clear, actionable insights for business stakeholders and non-technical audiences to support informed decision-making.
• Evaluate emerging tools, frameworks, and technologies, and provide recommendations for adoption.
• Mentor engineering teams on architectural best practices, distributed systems, event-driven and batch processing design.
• Ability to explain complex technical concepts to non-technical audiences.

Supervisory Responsibilities:

• None

Qualifications:

The requirements listed are representative of the knowledge, skill, and/or ability required. Reasonable accommodation may be made to enable individuals with disabilities to perform the primary functions.

• Demonstrated experience in one or more of the following domains: event-driven and message-based systems, transaction processing systems, distributed and parallel computing.
• Solid experience designing application architectures across distributed and enterprise technology platforms, including microservices and streaming or batch-based applications
• Experience with projects involving complex integration of disparate types of technologies/platforms and data.
• Architect Kubernetes-based deployment topology for stateless services and on-demand jobs, including horizontal pod autoscaling, multi-AZ resilience, and cross-region/on-prem failover and disaster recovery strategies (hot-hot, S3 replication).
• Fluent in object-oriented design, industry best practices, software patterns, and architecture principles.
• Experience defining and documenting architecture strategies, designs, and requirements across all enterprise architecture domains – business, information systems (applications & data), and technology.
• Experience defining non-functional requirements (NFRs) including performance, scalability, availability, and security and others as well as ensuring applications’ architecture are designed to satisfy them.
• Highly motivated with a strong sense of ownership of their work/projects.
• Excellent oral and written communication skills.

Technical Skills:

• Software development experience in Java, Python, or C#, including building high-throughput, low-latency services for data-intensive or transaction-oriented systems
• Strong knowledge of Kafka or other messaging/streaming systems, including topic design, partitioning strategies, and producer/consumer patterns for reliable, event-driven communication.
• Fluent in object-oriented design principles, software design patterns, and modern architecture practices (e.g., microservices, event-driven architecture, domain-driven design), with the ability to apply them to real-world, large-scale systems.
• Knowledge of identity and access management using OAuth2/OIDC flows (client credentials, authorization code with PKCE), JWT-based token validation, and integration with identity providers (e.g., ForgeRock, SailPoint).
• Knowledge of schema management and data serialization formats, preferably Protocol Buffers (Protobuf), including schema evolution
• Understanding of data lake-house architecture and table formats such as Apache Iceberg
• Familiarity with Infrastructure as Code principles and tooling preferably Terraform to enable repeatable, automated environment design.
• Working knowledge of DevOps tooling and practices such as Git, Jenkins, Docker, Artifactory, ArgoCD, and others to support automated build, test, and deployment of distributed, containerized applications.
• Experience working with Cloud ecosystems (AWS, PaaS, K8s)

Education and/or Experience:

Minimum Qualifications:

• BS degree in Computer Science, similar technical field, or equivalent practical experience.
• 10+ years of relevant work experience

Preferred Qualifications:

• 6+ years of work experience in the capital markets industry.