Staff Data Engineer

Walmart Inc.; Bentonville (AR); Hybrid; USD 127,000 - 220,000; Full time

Benefits offered by this job

Health benefits

401(k)

Stock purchase

Paid time off

Education assistance with 100% company

Company discounts

Adoption expense reimbursement

Job summary

Walmart Inc. in Bentonville, AR seeks a Staff Data Engineer to design and implement scalable data pipelines and data models across multiple data platforms. You will translate business problems into data-driven solutions, collaborate with stakeholders, and drive data governance and quality initiatives.

The role requires deep experience with Hadoop/Spark ecosystems, cloud platforms, and modern orchestration tools. You will mentor junior engineers and contribute to architecture decisions.

Qualifications

• Experience designing scalable big data pipelines and ETLs using Hadoop, Hive, Spark, and Databricks.
• Experience with Scala, Python, Spark, and SQL for ETL development.
• Strong SQL and Spark SQL querying for data analysis and validation.
• Experience in cloud ecosystems (GCP, AWS, Azure).

Responsibilities

• Translate business problems into data-driven solutions and select appropriate methods.
• Recommend business insights and build business cases for ROI.
• Identify and source data fit for purpose; ensure data quality.
• Build data infrastructure for transformation and integration from multiple sources.
• Develop and maintain complex data models and documentation.
• Develop and test code, review engineering designs, and ensure scalable deployments.
• Establish data governance practices and educate stakeholders.

Skills

Big data pipelines

ETL development

Python

SQL

Spark

Hadoop/Hive

Databricks

Cloud platforms (GCP/AWS/Azure)

Airflow/Oozie/Autosys

Data modeling

Education

Bachelor's degree in Computer Science or related field

Master’s degree in Computer Science or related field

Tools

Apache Airflow

Oozie

AutoSys

Confluence

Jira

Azure DevOps

Databricks

Hadoop

Hive

Snowflake

Parquet/AVRO/JSON/CSV

S3/GCS/Azure Blob Storage

Job description

Position: Staff Data Engineer

Job Location: 811 Excellence Dr, Bentonville, AR 72716

Duties:

• Tech. Problem Formulation: Translates/co-owns business problems within one's discipline to data related or mathematical solutions. Identifies appropriate methods/tools to be leveraged to provide a solution for the problem. Shares use cases and gives examples to demonstrate how the method would solve the business problem.
• Understanding Business Context: Provides recommendations to business stakeholders to solve complex business issues. Develops business cases for projects with a projected return on investment or cost savings. Translates business requirements into projects, activities, and tasks and aligns to overall business strategy and develops domain specific artifact. Serves as an interpreter and conduit to connect business needs with tangible solutions and results. Identifies and recommends relevant business insights pertaining to their area of work.
• Data Source Identification: Understands the priority order of requirements and service level agreements. Defines and identifies the most suitable sources for required data that is fit for purpose, referring to external sources as required. Performs initial data quality checks on the extracted data. Reviews the deliverables of junior associates and provides guidance on data source and quality.
• Data Transformation and Integration: Builds the infrastructure required for optimal transformation and integration from a wide variety of data sources using appropriate data integration technologies. Uses modern tools, techniques, and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks. Deploys pipelines using scheduling and orchestration frameworks. Evaluates impacts of data issues and risks at an early stage. Identifies needs and creates methods to fuse and reshape complex, multi-source data and make it usable for modeling. Updates knowledge of current and emerging big data analytics and data science trends and techniques.
• Data Modeling: Assembles large, complex data across all data platforms and data tools. Builds complex logical and conceptual models and provides guidance to team on physical data models. Identifies and defines the appropriate techniques for exposing data to other systems. Reviews and provides guidance and inputs on all data modeling activities to team members. Creates and maintains critical data documentation and metadata that allows data to be understood and leveraged as a shared asset. Assists in defining data modeling standards and foundational best practices. Provides inputs to the architectural design to make best use of the available resources, given goals, and expected loads.
• Code Development and Testing: Reviews the solution and application design to ensure it meets business, technical, and data requirements. Identifies language and libraries to use in the development process. Maps test cases to business and functional requirements. Creates proof of concepts. Reviews and troubleshoots code in line with final designs. Identifies and recommends the appropriate testing methodology. Identifies the environment(s) for deployment. Identifies and recommends modifications of application based on different environmental requirements. Identifies modifications needed for scalability and drives the change. Monitors applications in production and leads development of patches where required. Reviews and ensures all code documentation is complete and updated periodically.
• Data Governance: Establishes, modifies, and documents data governance projects and recommendations. Implements data governance practices in partnership with business stakeholders and peers. Interprets company and regulatory policies on data. Educates others on data governance processes, practices, policies, and guidelines. Provides recommendations on needed updates or inputs into data governance policies, practices, or guidelines.

Minimum education and experience required:

• Bachelor's degree or the equivalent in Computer Science or related field plus 4 years of experience in software engineering or related experience;
• OR Master’s degree or the equivalent in Computer Science or related field plus 2 years of experience in software engineering or related experience.

Skills Required:

• Must have experience with: Designing and implementing highly scalable, optimizing, and distributing big data pipelines and ETLs using frameworks like Hadoop, Hive, Spark, and Databricks;
• Developing ETL pipelines using Scala, Python, Spark, and SQL;
• SQL and Spark SQL querying for data analysis and validation;
• Spark shell scripting for automated data validation;
• Documenting data pipelines and workflows like Confluence, Jira, and Azure DevOps;
• Orchestrating and automating implemented pipeline solutions using tools and frameworks like Automic, AutoSys, Oozie, and Apache Airflow;
• Writing SQLs for data querying from RDBMS and Data warehousing frameworks like Apache Hive and Snowflake;
• Developing and maintaining modules in cloud ecosystem in Tech like Google Cloud, AWS, and Azure;
• Working with Parquet, AVRO, CSV, ORC, and JSON;
• Working in different file-based storage systems, including HDFS, GCS, Azure Blob Storage, and S3;
• Creating and automating dashboards by using Power BI, Tableau, and Looker;
• Leading large-scale projects and providing guidance to team members;
• System architecture, HLD to detailed design, and disaster recovery;
• Resource modeling, optimization, and forecasting;
• Applying incident management, change management, defect-reduction, and system-reliability processes to improve operational performance.

Employer will accept any amount of experience with the required skills.

Rate of pay: $127,171.00 - 220,000.00/year.

Additional compensation includes annual or quarterly performance incentives.

Benefits:

• At Walmart, we offer competitive pay as well as performance-based incentive awards and other great benefits for a happier mind, body, and wallet.
• Health benefits include medical, vision and dental coverage.
• Financial benefits include 401(k), stock purchase and company-paid life insurance.
• Paid time off benefits include PTO (including sick leave), parental leave, family care leave, bereavement, jury duty and voting.
• Other benefits include short-term and long-term disability, education assistance with 100% company paid college degrees, company discounts, military service pay, adoption expense reimbursement, and more.
• Eligibility requirements apply to some benefits and may depend on your job classification and length of employment.
• Benefits are subject to change and may be subject to a specific plan or program terms.
• For information about benefits and eligibility, see One.Walmart.com.

Wal‑Mart is an Equal Opportunity Employer.

Wal‑Mart and its subsidiaries are committed to maintaining a drug‑free workplace and has a no tolerance policy regarding the use of illegal drugs and alcohol on the job. This policy applies to all employees and aims to create a safe and productive work environment.

Fifty years ago, Sam Walton started a single mom‑and‑pop shop and transformed it into the world's biggest retailer. Since those founding days, one thing has remained consistent: our commitment to helping our customers save money so they can live better. Today, we're reinventing the shopping experience and our associates are at the heart of it. You'll play a crucial role in shaping the future of retail, improving millions of lives around the world. This is that place where your passions meet purpose. Join our family and create a career you're proud of.