Azure Data Architect

Quantum Technologies. LLC; Atlanta (GA); In loco; USD 140.000 - 190.000; Tempo pieno

Descrizione del lavoro

Quantum Technologies. LLC seeks a senior data engineering professional to partner with business units, designing data pipelines and governance models. You will evaluate buy vs.

build options, implement scalable Databricks solutions, and drive automation across CI/CD and cloud services while aligning with IT strategy. Responsibilities include translating requirements, building robust data architectures, and mentoring peers to raise delivery maturity and operational excellence across multiple

Competenze

• 10+ years of experience in data engineering, including design, development and operations.
• Advanced proficiency in Parquet, Iceberg, PySpark, Lambda, Databricks and Azure data services.
• Proficiency in Python, Java or Scala for data processing and apps.
• Extensive Databricks experience with large-scale data pipelines and governance.
• CI/CD pipelines and version control with Git, Bitbucket and Jenkins.
• Databricks Data Engineering Associate or Professional certification.
• Azure certifications and strong SQL/DB knowledge (Oracle SQL Server).
• Experience with data modeling and ER concepts, DDL/DML.

Mansioni

• Serve as expert for business unit IT systems and automate processes.
• Translate requirements into functional specs and manage traceability.
• Redesign business processes and ensure compliance with standards.
• Facilitate buy vs build decisions and coordinate with stakeholders.
• Define division processes to improve efficiency and governance.
• Develop reports using SQL-driven tools and Business Objects.
• Analyze incidents and work with product support to implement fixes.
• Lead multiple projects and assess complexity and dependencies.
• Mentor team members and maintain knowledge of tech trends.

Conoscenze

Data engineering

Parquet

Iceberg

PySpark

Databricks

Azure data services

Python

Java

Scala

CI/CD

Git

Bitbucket

Jenkins

Data governance

Oracle SQL

SQL

Formazione

Bachelor's degree in Computer Science or related field

Strumenti

Git

Bitbucket

Jenkins

DAB deployments

Databricks

Azure data services

Descrizione del lavoro

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Work Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, Other

Job Summary :
Works with business partners as the primary interface between Information Technology and specific business areas, identifying business unit requirements, creating project and process specifications, coordinating with project teams, and ensuring adherence to schedules and budgets and alignment with Information Technology strategy. Determines if solutions already exist or if new solutions are needed and provides input regarding buy vs build decisions. Ensures proper integration of solutions that impact work processes. Identifies opportunities for improvement or enhancement and negotiates agreements and commitments between business partners and Information Technology from requirements gathering to final implementation.

Minimum Education : Bachelor's degree in computer science, information systems, or a related field or an equivalent combination of education and experience.

Minimum Experience : 12 years of experience developing, implementing and maintaining enterprise policies, procedures and operating standards including preparing agendas, meeting minutes, and follow-up actions for meetings with management. Regular office environment;

Required Skills, & Abilities :

• 10+ years of experience in data engineering, including design, application development, testing, and operational stability.
• Advanced proficiency in data processing frameworks and tools, including Parquet, Iceberg, PySpark, Lambda, Databricks and Azure data services.
• Proficiency in programming languages such as Python, Java, or Scala for data processing and application development.
• Extensive hands-on experience with Databricks, including architecting and managing large-scale data pipelines, optimizing Spark jobs, implementing security and governance controls, and automating workflows. Proven ability to leverage Databricks for advanced analytics, data lakehouse architectures, and seamless integration with AWS or other cloud services.
• Proficiency in automation and continuous delivery methods, utilizing CI/CD pipelines with tools like Git/Bitbucket, Jenkins, automated DAB deployments and version control.
• Certification on Databricks Data Engineering Associate or Professional.
• Azure Certifications.
• Proficiency in relational databases (Oracle or SQL Server and ER Data Modeling.
• Skilled in writing Oracle SQL queries utilizing DML, DDL, and

Job Duties and Responsibilities :

• Serves as an expert regarding the business units supported and has expert understanding of the Information Technology organization's systems and capabilities. Analyzes business unit operations to understand strengths and weaknesses and to identify process and function automation opportunities. Develops appropriate business models to represent business requirements and meet business needs.
• Obtains and validates information requirements from stakeholders through various analysis techniques. Translates high level business requirements into functional specifications for the Information Technology organization. Manages requirements, approvals, changes, and traceability.
• Assists with business process redesign and documentation as needed for new technology. Ensures technology solutions are in compliance with and uphold organizational standards. Develops business models to represent requirements, such as use cases, business process flows, and data flow diagrams.
• Negotiates agreements and commitments. Facilitates communication between business units and Information Technology from initial requirements to final implementation. Provides input regarding buy versus build decisions. Participates in planning sessions with business units to improve business processes.
• Participates in defining and streamlining division level processes, procedures, and templates as needed to support the organization's process maturity goals and to ensure effective and efficient methods in accordance with best practices.
• Provides business reporting and analytical support, particularly pertaining to departmental governance and performance, utilizing various data sources and analytical tools. Performs analysis and provides reporting views with improvement recommendations. Designs, develops, tests, and maintains various reports using SQL driven application table portlets and Business Objects.
• Conducts systems analysis with a focus on business process models and use cases. Performs system administration and maintenance as needed in support of departmental activities. Performs audits to ensure compliance with division processes, procedures and standards.
• Analyzes incidents related to the system/application to identify problem areas. Works closely with the application business partner's product support team to identify and implement solutions.
• Works on multiple projects evaluating and identifying project complexity, assumptions, constraints, and dependencies. Conducts timely surveys to gauge effectiveness of delivery of technology solutions.
• Performs a variety of tasks to improve on the existing operating processes, methodologies and standards. Participates in the development of processes, procedures, and protocols for the department to ensure effective and efficient methods of providing support. Coordinates the maintenance of all tools used to administer the processes as needed to ensure system currency.
• Builds, maintains, and manages stakeholder relationships. Educates the Information Technology organization on the direction of the business. Communicates complex technical concepts simply and effectively to nontechnical team members. Guides team members through the established operating process and elevate non-conformance as needed.
• Maintains a current knowledge base and understanding of technological trends. Uses Information Technology knowledge base to bring solutions to business units supported and enhances the enterprise's competitive edge.
• Assists less experienced team members as needed.
• Each team member is expected to be aware of risk within their functional area. This includes observing all policies, procedures, laws, regulations and risk limits specific to their role. Additionally, they should raise and report known or suspected violations to the appropriate Firm authority in a timely fashion.
• Performs other related duties as required.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.