Associate Data Engineer – Cloud Data Lake & Security

BeyondTrust; United States; On-site; USD 90,000 - 120,000; Full time

Job summary

BeyondTrust invites a talented Associate Data Engineer to join our team, helping build and optimize our data lake that processes billions of events daily. You will work with state-of-the-art data engineering tools to deliver cost-effective cloud solutions that enable cybersecurity research and rapid business value.

You will develop data pipelines, contribute to data modelling, and collaborate with senior engineers to drive improvements, with a strong emphasis on observability and ML Operations

Qualifications

• Proficient in Python or an OO language.
• Spark experience required; Databricks knowledge is a plus.
• Experience with distributed data processing.
• Knowledge of data warehousing for analytics.
• Experience with cloud data processing pipelines.

Responsibilities

• Contribute to data modelling efforts that support identity security use cases and analytical requirements.
• Develop and maintain data pipelines that bring together complex information from disparate systems to support security analysis in the data lake.
• Optimize data workloads at a software level by improving processing efficiency.
• Work with senior engineers to identify areas for process improvement and help implement solutions.
• Use monitoring and observability best practices to ensure optimal pipeline performance.
• Assist with ML Operations to ensure optimal model efficiency.
• Leverage CI/CD best practices to effectively develop and release source code.
• Participate in code reviews and collaborate with team members to improve engineering practices.

Skills

Cloud data processing

Python

Distributed processing

Learning agility

Tools

Spark

Databricks

CI/CD

Data warehousing

Graph data stores

Job description

BeyondTrust invites a talented Associate Data Engineer to join our team, helping build and optimize our data lake that processes billions of events daily. You will work with state-of-the-art data engineering tools to deliver cost-effective cloud solutions that enable cybersecurity research and rapid business value.

You will develop data pipelines, contribute to data modelling, and collaborate with senior engineers to drive improvements, with a strong emphasis on observability and ML Operations