Lead Data Engineer - ETL & SQL

J.P. Morgan; Columbus (OH); On-site; USD 90,000 - 150,000; Full time

Job summary

An established industry player is seeking a Lead Data Engineer to join their innovative team. In this pivotal role, you'll design and implement robust data solutions that enhance business operations and drive decision-making. Your expertise in data management, governance, and containerization will be crucial as you collaborate with cross-functional teams to deliver scalable data architectures. This position offers the opportunity to work with cutting-edge technologies and contribute to a culture of diversity and inclusion. If you're passionate about data engineering and eager to make a meaningful impact, this is the perfect opportunity for you.

Qualifications

• 5+ years of experience in data and software engineering.
• Strong experience with relational and NoSQL databases.
• Expertise in scalable data architectures and pipelines.

Responsibilities

• Designs and delivers data platforms ensuring security and scalability.
• Constructs and optimizes data pipelines for ETL/ELT operations.
• Collaborates with teams across multiple locations for seamless integration.

Skills

Data Management

Data Governance

SQL

Python

Java

Collaboration

Containerization

ETL Processes

Data Quality

Communication

Education

Formal training in Data Engineering

Tools

Kubernetes

Docker

Git

Airflow

Job description

Join us as we embark on a journey of collaboration and innovation, where your unique skills and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference.

As a Lead Data Engineer at JPMorgan Chase within the Corporate Sector Enterprise Technology and Tech Reference Data, you are an integral part of an agile team that works to enhance, build, and deliver data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. As a core technical contributor, you are responsible for maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm’s business objectives.

Job responsibilities

• Provides recommendations and insight on data management, governance procedures, and intricacies applicable to the acquisition, maintenance, validation, and utilization of data.
• Designs and delivers trusted data collection, storage, access, and analytics data platform solutions in a secure, stable, and scalable way.
• Collaborates with team members across Columbus, Argentina, and Jersey City to ensure seamless integration and operation of the Fabric service within the Technology Reference Data (TRD) framework.
• Develops and implements data federation strategies using APIs, RDMS, and Object Stores to facilitate unified data access for Business Analysts, Data Scientists, Data Engineers, and Software Developers.
• Utilizes ANSI SQL to enable efficient data aggregation and manipulation across multiple data sources, enhancing analytical capabilities and decision-making processes.
• Designs and maintains robust software solutions for authorization and key management, ensuring secure connectivity to both private and public cloud environments.
• Constructs and optimizes data pipelines for ETL/ELT operations, ensuring efficient data processing and transformation to meet business needs.
• Builds and manages container images for custom services and applications, as well as third-party open-source applications, to support scalable and reliable deployment in cloud environments.
• Evaluates and reports on access control processes to determine the effectiveness of data asset security with minimal supervision.
• Adds to team culture of diversity, equity, inclusion, and respect.

Required qualifications, capabilities, and skills

• Formal training or certification on data and software engineering concepts and 5+ years of applied experience.
• Strong experience with both relational and NoSQL databases.
• Experience and proficiency across the data lifecycle.
• Experience in building and deploying applications within containerized environments utilizing tools such as Kustomize, Kubernetes, and Docker.
• Expertise in designing and implementing scalable data architectures and pipelines.
• Strong proficiency in programming languages such as Python, JAVA, and SQL.
• In-depth knowledge of data warehousing solutions and ETL processes.
• Ability to work collaboratively with cross-functional teams, including data scientists, analysts, and business stakeholders.
• Experience with data governance and ensuring data quality and integrity.
• Excellent communication skills, with the ability to convey complex technical concepts to non-technical audiences.
• Experience with version control systems like Git and CI/CD pipelines for data engineering workflows.

Preferred qualifications, capabilities, and skills

• Experience with Trino / Presto and other SQL query engines.
• Proficiency in Big Data technologies, with a strong focus on Performance Optimization using best practices.
• Solid understanding of the Parquet, ORC, and AVRO file formats.
• Solid understanding of open table formats particularly iceberg.
• Experience in building, maintaining, and optimizing iceberg tables.
• Proficiency in building and maintaining efficient container images.
• Experience with orchestration tools such as Airflow.