Senior Python Data Engineer — Energy Trading Pipelines

Ampstek; England; On-site; GBP 90,000 - 130,000; Full time

Job summary

Ampstek is seeking a senior data engineer with 8+ years of Python experience to lead the design and development of scalable data pipelines and enterprise Python applications.

You will work with NumPy, Pandas, BeautifulSoup, Selenium, SQL (PostgreSQL), Spark, and cloud platforms such as AWS S3 and Azure Databricks, in an Agile environment across multiple time zones.

You will also mentor team members, drive best practices in automation and data processing, and coordinate with stakeholders globally.

Qualifications

• 8+ years of experience in python scripting.
• Experience with Python packages: NumPy, pandas, polars, BeautifulSoup, Selenium, Requests.
• Experience in building native python ETL pipelines for scraping/processing.
• Good to have experience in processing big data.
• Proficient in SQL, PostgreSQL; Spark knowledge is a plus.
• Experience with AWS (S3, Lambda) and Azure Databricks.
• Experience delivering projects with Agile/Scrum.
• Ability to coordinate with teams across locations/time zones.
• Strong interpersonal and communication skills with leadership ability.

Responsibilities

• Lead the design, development, and optimization of scalable data pipelines and Python applications.
• Provide technical leadership in Python development and automation best practices.
• Work with Python data libraries such as NumPy, Pandas, BeautifulSoup, Selenium, pdfplumber, and Requests.
• Architect and optimize SQL queries and PostgreSQL solutions; leverage other relational stores.
• Integrate DevOps with CI/CD pipelines, Jenkins, and Git for smooth deployments.
• Develop cloud-based data workflows on AWS S3 and Azure Databricks.
• Drive agile delivery by collaborating with cross-functional teams under Agile/Scrum.

Skills

Python scripting

Team leadership

Agile/Scrum

Communication

Tools

NumPy

pandas

polars

BeautifulSoup

Selenium

Requests

SQL

PostgreSQL

Spark

AWS

Azure Databricks

Jenkins

Git

pdfplumber

Job description

Ampstek is seeking a senior data engineer with 8+ years of Python experience to lead the design and development of scalable data pipelines and enterprise Python applications.

You will work with NumPy, Pandas, BeautifulSoup, Selenium, SQL (PostgreSQL), Spark, and cloud platforms such as AWS S3 and Azure Databricks, in an Agile environment across multiple time zones.

You will also mentor team members, drive best practices in automation and data processing, and coordinate with stakeholders globally.