IA3

  • IA3 방식이 peft에 추가되었길래 colab 환경(T4, 14GB)에서 간단하게 50steps만 학습 진행

  • What is IA3?

    • IA3 method that scales activations by learned vectors, attaining stronger performance while only introducing a relatively tiny amount of new parameters.

GPT-neoX example of training

peft_IA3.ipynb

Ref