Data Engineer Principal

FedEx Dataworks; Coraopolis (Allegheny County); Sur place; USD 140 000 - 190 000; Plein temps

Résumé du poste

FedEx Dataworks is seeking a Data Engineer Principal to lead enterprise-grade data engineering initiatives, shaping standards and reusable assets, and guiding multi-disciplinary teams across IT, business, software, and data science domains.

You will own end-to-end data ingestion, transformation, and pipelining, mentor engineers, and drive scalable, secure data solutions for FedEx and external data products.

Qualifications

• Bachelor's degree required or equivalent.
• 7+ years of data engineering experience.
• Proven leadership of data engineering projects and teams.

Responsabilités

• Understand business and technical problems Dataworks aims to solve.
• Build tools, platforms and pipelines for data analysis and modeling.
• Scale from laptop-scale to cluster-scale data problems and infrastructure.
• Deliver tangible value rapidly with cross-functional teams.
• Codify best practices into reusable patterns and codebases.
• Collaborate with FedEx leaders and partner ecosystems for deployments.

Connaissances

Data engineering

Big data pipelines

Cloud environments

Distributed systems

Azure

Machine learning

Formation

Bachelor's degree in Information Systems/CS/Math/Engineering

Outils

DataBricks

BigQuery

GCP

Description du poste

Job Profile SummaryThe Data Engineer Principal plays a pivotal role within Dataworks and provides dynamic leadership at the enterprise level. S/he is focused on fundamentally creating and driving engineering innovation within Dataworks, helping to define and build the Dataworks organization and overseeing the delivery of key business initiatives. S/he acts as a “universal translator” between IT, business, software engineers and data scientists, collaborating with these multi-disciplinary teams. The Data Engineer Principal will champion the creation and adherence of technical standards for data engineering, including the selection and refinements of foundational technical components. S/he will work on those aspects of the Dataworks platform that govern the ingestion, transformation, and pipelining of data assets, both to end users within FedEx and into data products and services that may be externally facing. Day-to-day, s/he will be deeply involved in code reviews, large-scale deployments and general oversight. S/he will also mentor junior engineers to support the continued up-skilling of the Data Engineering team.

Job Profile SummaryThe Data Engineer Principal plays a pivotal role within Dataworks and provides dynamic leadership at the enterprise level. S/he is focused on fundamentally creating and driving engineering innovation within Dataworks, helping to define and build the Dataworks organization and overseeing the delivery of key business initiatives. S/he acts as a “universal translator” between IT, business, software engineers and data scientists, collaborating with these multi-disciplinary teams. The Data Engineer Principal will champion the creation and adherence of technical standards for data engineering, including the selection and refinements of foundational technical components. S/he will work on those aspects of the Dataworks platform that govern the ingestion, transformation, and pipelining of data assets, both to end users within FedEx and into data products and services that may be externally facing. Day-to-day, s/he will be deeply involved in code reviews, large-scale deployments and general oversight. S/he will also mentor junior engineers to support the continued up-skilling of the Data Engineering team. Takes end-to-end ownership and responsibility over critical data engineering initiatives. Advances Dataworks’ broad capabilities to use and deploy cutting edge data engineering tools and methods in Dataworks projects, platforms and products. Anchors current best practices by championing the design and build of reusable data engineering assets. Simultaneously works to keep Dataworks on the bleeding edge by disseminating the very latest and most sophisticated methods and tools for grappling with extremely large scale and complex data engineering problems. Oversees the design, build, test and maintenance of data pipelines at big data scale. Leads batch processing of collected data to ensure data is ready for analysis. Instantiates processes for keeping the ecosystem and pipeline optimized and efficient, troubleshooting standard performance, data related problems and providing expert guidance. Independently implements parsers, validators, transformers and correlators to reformat, update and enhance the data. Combines knowledge of data engineering methods, CI/CD, statistics, and machine learning practices to provide recommendations on the most organizationally critical and complex data-related problems. Advises junior data engineers, managers, and those in less senior positions.

Additional Summary And Preferred Skills Responsible for designing and implementing data pipelines, ensuring data quality and accessibility, creating data models, managing ETL processes, optimizing databases, collaborating with stakeholders, utilizing big data tools, implementing real-time data processing, ensuring data governance and security, documenting data architecture, and delivering data-driven insights.

• Design and implement data pipelines to collect, process, and store data efficiently.
• Ensure data quality, integrity, and accessibility.
• Create data models to support business analytics and reporting.
• Work with data scientists, analysts, and other stakeholders to understand data needs, and design schemas.
• Build and manage ETL (Extract, Transform, Load) processes for data integration and automate data ingestion and transformation tasks.
• Administer and optimize databases (both SQL and NoSQL). Monitor database performance and troubleshoot issues.
• Collaborate with software engineers to integrate data systems with applications.
• Utilize big data tools for processing large datasets, such as Hadoop or Spark.
• Implement distributed systems for real-time data processing.
• Ensure data governance and security through compliance with data governance policies and data security standards.
• Implement data access controls and encryption techniques.
• Document data architecture, pipelines, and processes.
• Provide reports and insights based on data analysis.
• Working knowledge of DataBricks, BigQuery, and GCP

Job Description

• Understanding in depth both the business and technical problems Dataworks aims to solve
• Building tools, platforms and pipelines to enable teams to clearly and cleanly analyze data, build models and drive decisions Inventing new approaches and processes for tackling data intensive problems
• Scaling up from “laptop-scale” to “cluster scale” problems, in terms of both infrastructure and problem structure and technique
• Delivering tangible value very rapidly, collaborating with diverse teams of varying backgrounds and disciplines
• Codifying best practices for future reuse in the form of accessible, reusable patterns, templates, and code bases
• Interacting with leaders from the broader enterprise and outside of FedEx (partner ecosystems and customers) to create synergies and ensure smooth deployments to downstream systems

Minimum Education Bachelor’s Degree in Information Systems, Computer Science, or a quantitative discipline such as Mathematics or Engineering and/or equivalent formal training or work experience. Directly related PhD preferred.

Minimum Experience Seven plus (7+) years equivalent work experience in measurement and analysis, quantitative business problem solving, simulation development and/or predictive analytics. Extensive knowledge in data engineering and machine learning frameworks including design, development and implementation of highly complex systems and data pipelines. Extensive knowledge in Information Systems including design, development and implementation of large batch or online transaction-based systems. Strong understanding of the transportation industry, competitors, and evolving technologies. Experience providing expert leadership in a general planning or consulting setting. Experience as a leader or a senior member of multi-functional project teams. Excellent interpersonal skills and the ability to communicate effectively to senior audiences. A related advanced degree may offset related experience requirements.

Knowledge, Skills And Abilities

• Technical background in computer science, software engineering, database/distributed systems
• Deep fluency with distributed and cloud environments and a deep understanding of how to balance computational considerations with theoretical properties
• Detailed knowledge of the Microsoft Azure tooling for large-scale data engineering efforts and deployments is highly preferred
• A highly compelling track record of designing and deploying large scale technical solutions, which deliver tangible, ongoing value
• Direct experience having built and deployed robust, complex production systems that implement modern, data scientific methods at scale
• Ability to context-switch, to provide support to dispersed teams which may need an “expert hacker” to unblock an especially challenging technical obstacle, and to work through problems as they are still being defined
• Demonstrated ability to deliver technical projects with a team, often working under tight time constraints to deliver value
• An ‘engineering’ mindset, willing to make rapid, pragmatic decisions to improve performance, accelerate progress or magnify impact
• Comfort with working with distributed teams on code-based deliverables, using version control systems and code reviews
• Demonstrated ability to conduct data analysis, investigation, and