AI security engineer

Tata Consultancy Services Limited; Warszawa; In loco; PLN 180.000 - 320.000; Tempo pieno

Descrizione del lavoro

Tata Consultancy Services Limited is seeking an AI Security Engineer to evaluate cybersecurity capabilities of frontier AI models and develop internal benchmarks. You will design test suites to measure refusal rates for cyber-related prompts and work with risk governance teams to inform safe AI adoption.

You will build datasets, scoring methods, and tooling to ensure repeatable evaluation across model versions, while communicating findings through clear reports and dashboards.

Competenze

• Bachelor's or Master's degree in Computer Science, Software Engineering, Cybersecurity, Data Science, AI, or related field.
• Hands-on cybersecurity experience in offensive security, pentesting, red teaming, or vulnerability research is a plus.
• Proficiency in Python or another language for test harnesses, task automation, and results analysis.
• Experience using and prompting large language models, AI safety testing, or curiosity to learn how models work.
• Experience working with APIs and building automation around them.

Mansioni

• Design, build, and maintain internal benchmarks that measure frontier models’ cyber capabilities.
• Develop and run test suites that measure refusal rates for policy-violating cyber requests, including adversarial prompts.
• Maintain evaluation datasets, scoring methods, and tooling for repeatable tests across model versions and vendors.
• Partner with Model Risk Management, governance, and technology teams to understand evaluation needs.
• Communicate results to technical and non-technical audiences via reports, dashboards, and presentations with actionable recommendations.

Conoscenze

Python

LLM evaluation

APIs & automation

Communication skills

Formazione

Bachelor's or Master's in CS/SE/Cybersecurity/Data Science/AI

Descrizione del lavoro

Описание:

We are seeking an AI Security Engineer to evaluate the cybersecurity capabilities and safety of frontier AI models. The role will focus on developing internal cyber benchmarks and test how well model guardrails refuse cyber-related requests. The work will inform how the firm assesses and safely adopts AI models across the enterprise.

Задачи:

Design, build, and maintain internal benchmarks that measure frontier models’ cyber capabilities. Develop and run test suites that measure refusal rates for policy-violating cyber requests, including adversarial and jailbreak-style prompts. Maintain evaluation datasets, scoring methods, and tooling so tests are repeatable across model versions and vendors. Partner with internal teams and stakeholders, such as Model Risk Management, governance, and technology teams to understand evaluation needs and support safe AI adoption. Communicate results clearly to both technical and non-technical audiences through reports, dashboards, and presentations, including actionable recommendations. A suite of internal cyber benchmarks that measure frontier model capabilities across defined security tasks, with documented methodology and scoring criteria. A repeatable guardrail testing framework that measures how models respond to cyber tasks across defined risk categories, confirming they refuse or comply appropriately for each. A maintained library of evaluation datasets and test prompts mapped to each risk category, versioned so results can be compared over time. Supporting tooling and documentation that allow teams to run evaluations independently.

Требования:

Bachelor's or Master's degree in Computer Science, Software Engineering, Cybersecurity, Data Science, Artificial Intelligence, or a related field. Будет плюсом: Hands-on cybersecurity experience in areas such as offensive security, penetration testing, red teaming or vulnerability research; Proficiency in Python or another programming language for building test harnesses, automating tasks, and analyzing results; Strong written communication skills for documenting methods and findings; Experience using and prompting large language models, exposure to model evaluation or AI safety testing, or a strong curiosity and ability to quickly learn how these models work and how to assess them; Experience working with APIs and building automation around them

Условия:

Cooperation on a B2B contract basis. Opportunity to use foreign languages and the newest technologies on a daily basis.