Implementing Trojan neural networks on state-of-the-art language models for AI safety
Proposal in one sentence:
Trojan neural networks are maliciously perturbed networks that have hidden triggers for harmful behaviour and are incredibly important to study for the safety of future AI.
Description of the project and what problem it is solving:
As part of the third Alignment Jam, AI testing hackathon, a team took the first steps towards novel implementations of vision Trojan neural network methods into Transformers for language modeling. With this grant, we ask for support to help these budding researchers publish their work in reputable conferences and start their journey towards working within AI safety.
All grant money will go to the hackathon team to finish their publication. Team lead and squad member here are supported separately.
- Extending the work
- Finishing the paper with the three team members
- Submitting to workshop at ICML or to the main track of ICLR
- Twitter: esbenkc
- Discord: Arvino Bibulus#9302
- Twitter: fazlbarez
- Discord: Fazl#3700
- Along with the three team members for the project from Ecole Nationale Superiore Ulm: Clément Dumas, Charbel-Raphaël Segerie, Liam Imadache
Additional notes for proposals