Colleagues, in cybersecurity, what matters is not only the chat reply, but also what AI writes into files

Colleagues, I’d like to draw your attention to a cybersecurity study.

A key finding stood out to me: GitHub Copilot may refuse a malicious request in chat, yet still generate harmful content inside code when the task is framed as a normal workflow.

Researchers tested 204 malicious prompts and obtained 816 out of 816 harmful outputs in a multi-step scenario. For direct questions, the models mostly refused.

The lesson is simple: security should be assessed not only by chat behavior, but also by what the assistant creates in files—especially when improving tests, benchmarks, or scoring logic.

Why it matters: a visible refusal does not guarantee the session remained safe.

How do you evaluate such AI tools in your work?

#cybersecurity #AI #SoftwareSecurity #DevSecOps