Replies: 1 comment
-
这两个应该需要补上。 COSINE_WITH_RESTARTS = "cosine_with_restarts"
POLYNOMIAL = "polynomial" 这两个可以考虑合并到 Trainer 里面一起支持 "CosineAnnealingWithWarmupDecay",
"LinearAnnealingWithWarmupDecay", |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
1. LR Scheduler 升级更新
Huggingface的 https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_utils.py
而当前的PaddleNLP的Trainer中只支持下面几个,建议更新。
2. 统一使用Trainer里面的LR Scheduler(与HF对齐)
https://github.com/PaddlePaddle/PaddleNLP/blob/develop/paddlenlp/transformers/optimization.py 有旧的LR Scheduler,建议废弃,未来的新代码需要升级成最新的 Trainer 里面的 Scheduler,这样在使用体验上对习惯HF的用户来说更友好。
Beta Was this translation helpful? Give feedback.
All reactions