Transformer Trojans: Implementing Trojan neural networks on state-of-the-art language models for AI safety

Proposals Round 9
#1

Transformer Trojans

Implementing Trojan neural networks on state-of-the-art language models for AI safety

Proposal in one sentence:

Trojan neural networks are maliciously perturbed networks that have hidden triggers for harmful behaviour and are incredibly important to study for the safety of future AI.

Description of the project and what problem it is solving:

As part of the third Alignment Jam, AI testing hackathon, a team took the first steps towards novel implementations of vision Trojan neural network methods into Transformers for language modeling. With this grant, we ask for support to help these budding researchers publish their work in reputable conferences and start their journey towards working within AI safety.

image
image1296×802 162 KB

All grant money will go to the hackathon team to finish their publication. Team lead and squad member here are supported separately.

Grant Deliverables:

  • Extending the work
  • Finishing the paper with the three team members
  • Submitting to workshop at ICML or to the main track of ICLR

Squad

Squad Lead:

  • Twitter: esbenkc
  • Discord: Arvino Bibulus#9302

Squad members:

  • Twitter: fazlbarez
  • Discord: Fazl#3700
  • Along with the three team members for the project from Ecole Nationale Superiore Ulm: Clément Dumas, Charbel-Raphaël Segerie, Liam Imadache

Additional notes for proposals

3 Likes