Senior Associate, Data Engineer

General Atlantic; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

An established industry player is on the lookout for a Senior Associate Data Engineer to join their Data Strategy team. This pivotal role involves architecting and developing data solutions that enhance business processes and provide a seamless user experience. The ideal candidate will have a robust background in data engineering, particularly in cloud-based technologies and integration tools. You will collaborate with various teams to build a self-service data platform and contribute to the overall data architecture. If you are passionate about leveraging data to drive business success and thrive in a dynamic environment, this opportunity is perfect for you.

Benefits

Medical Insurance

Retirement Savings Contributions

Mental Health Resources

Physical Health Resources

Annual Discretionary Bonuses

Long-term Incentive Programs

Qualifications

• 5-8 years of experience as a data engineer with architecture skills.
• Strong background in cloud-based data integration tools and technologies.

Responsibilities

• Architect and develop data ingestion layers and solutions using Python and SQL.
• Create and maintain complex data processing pipelines and integration jobs.

Skills

Python

SQL

Data Integration

Cloud-based Architecture

ETL/ELT

Data Engineering

API Management

Data Analysis

Problem Solving

Communication Skills

Education

Bachelor’s degree in Computer Science

Equivalent working experience in IT

Tools

Databricks

Azure

Tableau

PowerBI

Talend

SSIS

Job description

General Atlantic is a leading global investor in the private markets with more than four decades of experience providing capital and strategic support for over 540 growth companies throughout its history. Established in 1980, General Atlantic continues to be the dedicated partner to visionary founders and investors seeking to build dynamic businesses and create long-term value. The firm leverages its patient capital, operational expertise, and global platform to support a diversified investment platform spanning Growth Equity, Credit, Climate, and Infrastructure strategies. General Atlantic manages approximately $100 billion in assets under management, inclusive of all strategies, as of October 1, 2024, with more than 900 professionals in 20 countries across five regions. For more information on General Atlantic, please visit: www.generalatlantic.com.

Position Summary

General Atlantic is looking to hire a Senior Associate Data Engineer as a key member in the Data Strategy team. This position will report directly to the VP, Head of Data Engineering. The Data Engineer will work as a developer and architect with primary focus on data integration & preparation, data as a service using API Management Services, and self-service data platform to provide a distinctive user experience for GA’s global team of professionals, investors, and other stakeholders. As a member of the data team, this person will work with our PMO’s, Infrastructure & Security, and App Dev teams to assist in the design and development of custom solutions that streamline business processes. The ideal candidate must have a solid background in building data products with a strong intellectual curiosity and desire to learn the latest technology.

Responsibilities

Data Integration & Preparation

• Responsible for architecture, design, administration, and development of Data Ingestion Layer as part of Data Strategy, especially using cloud-based architecture.
• Implement data solutions using python, SQL, PySpark and Databricks technologies like DLT, Autoloader & DLT Streaming.
• Develop solutions in a cloud environment leveraging open-source languages.
• Leverage Infrastructure-as-Code to build and deploy infrastructure and data pipelines.
• Define and develop standards, configurations, and operational/administration guidelines for data integration layer to provide it as a service to Data Science and App Dev teams.
• Create integration jobs connecting to heterogeneous data sources like SaaS applications, hosted applications, private cloud applications, databases etc. using APIs or varying data connectors.
• Develop ETL, ELT and other service-based integration jobs as part of overall Data application delivery.
• Develop Reporting services using tools like Tableau, PowerBI or SSRS.
• Provide Level 3 support for all data applications, especially for integration layer.

Data as a Service

• Define and develop processes and controls around the data service layer so that Data Science or App Dev teams can provide or ingest data via the centralized service layer.
• Work with data team members in designing and developing the data classes within the data service layer.
• Build and maintain a repository of web endpoints for the centralized logical data service layer as part of forward-looking integration capabilities.
• Build complex data processing pipelines using control flow and data flow components for real-time or batch jobs using python, Azure services, Talend, Databricks or SSIS.
• Modernize legacy applications from on-prem to the cloud as well as off legacy ETL tools.

Self-Service Data Platform

• Work with data team members to prepare data or provide real-time data to de-centralized Data Lake, an Enterprise Data Hub, or Data Warehouse, using latest cloud-based technologies like Databricks and/or Azure Data Services.
• Collaborate with data team members in design and development of Enterprise Data Hub that can store, compute and query relational data, columnar data and unstructured blobs.

Data Design & Development

• Contribute towards all development of SQL and NOSQL scripts, tables, Views, stored procedures, extended functions in python etc. in data layer.
• Interact with data team members in design and development on overall data application delivery, including data ingestion & preparation, data design & development, and data analytics & reporting.

Qualifications

Background and Training

• Bachelor’s degree in Computer Science, management information systems, or equivalent working experience in information technology.
• At least 5-8 years as a data engineer, with at least 2 years of architecture and design experience.
• Preferred experience working in the Finance industry.

Technology

• Demonstrated experience in learning, working with, and adopting new technologies.
• Experience and competency with design and development of cloud-based data integration tools, especially python, spark, Databricks and Talend or Alteryx.
• Proficient in working with heterogeneous data sources, cloud providers like Salesforce, Workday, Box etc., data aggregators, web sources or relational/analytical data sources providing structured, semi-structured or unstructured data.
• Exposure to design & development using API services on Databricks or Azure to provide data as a service, and BI & Reporting tools like PowerBI and Tableau.
• Extensive experience in coding tools and their libraries like python, .NET or Java in building complex custom processing or calculation sub jobs.
• Ability to factor usability, performance, security, and maintainability requirements into software solutions.
• Familiarity with agile software engineering processes and ability to contribute to planning and reporting.

Leadership/Professionalism

• Demonstrated self-motivation to plan and execute tasks with minimal direction and drive for exceptional, high-quality results.
• Ability to work in a fast-paced, dynamic environment and retain professionalism and accuracy.
• A strong work ethic and ‘can do’ attitude: motivated, flexible nature, team-player spirit, ability to maintain composure at all times.

Communication

• Strong written and verbal communication and interpersonal skills.
• Ability to communicate complex technical concepts to product owners, business partners, and IT management.
• Demonstrated ability to structure communication to promote a proposed idea or solution.

Business Analysis/Problem Solving

• Strong attention to detail with the ability to be proactive, solve problems, think ahead, and multi-task daily.
• Experience reviewing business requirements and creating technical design documentation.
• Ability to identify areas for improvement and present and implement viable solutions.
• Strong comfort and experience liaising with internal and external contacts.

General Atlantic offers a robust reward program to all employees that will support you and your family, maintaining fulfilling, secure and healthy lives now and into the future, which includes but is not limited to medical insurance, retirement savings contributions, mental and physical health resources and an equal pay program that allows you to earn a base salary of $140,000 – 190,000 USD of which the exact amount offered will be driven by the degree to which you meet role requirements, additional special skills and experience. Additional reward programs, such as annual discretionary bonuses and long-term incentive programs, are available for eligible employees and are offered as recognition for performance and one’s contributions towards the organization’s success.

General Atlantic is an equal opportunity employer and does not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, veteran status, or any other status protected under federal, state, or local law.

By completing this application, you are consenting to General Atlantic’s processing and use of your personal data. Information on how General Atlantic will use and process your data can be found here: https://www.generalatlantic.com/privacy-notice/.