SOCIAL CHEMISTRY 101 - Learning to Reason about Social and Moral Norms

Note

  • web site: https://maxwellforbes.com/social-chemistry/
  • paper: SOCIAL CHEMISTRY 101- Learning to Reason about Social and Moral Norms.pdf
  • 데이터셋 수집 순서
    • 상황 수집 필요 (여기선 레딧, 고민상담, 찬반좌등이 있을법한 상황설정된곳에서 가져옴)
    • → 상황마다 RoT를 1~5개 생성
    • → RoT마다 multiple annotation 작성 (같은 RoT를 다르게 볼 수 있으니..)
    • → RoT Breakdown, Action Breakdown (Action도 해야되나.. RoT안에 포함된거아닌가.. attribute 자체는 다르긴한데? 정답은! RoT는 도덕적인 관점에서 따지고, Action은 법이나 문화적인 관점에서 따짐)
    • → 추가적인 태깅도 진행 불분명한 상황이거나, 19금상황이거나, 너무불편한주제(학대나,…); 이런 경우는 스킵가능
  • RoT가이드라인등은 비슷하게 가져가면 될 것으로 보임
  • 137명이 태깅함
  • split our dataset into 80/10/10% train/dev/test partitions by situation
  • 데이터 예시
area m split rot-agree rot-categorization rot-moral-foundations rot-char-targeting rot-bad rot-judgment action action-agency action-moral-judgment action-agree action-legal action-pressure action-char-involved action-hypothetical situation situation-short-id rot rot-id rot-worker-id breakdown-worker-id n-characters characters
amitheasshole 1 train 4 advice loyalty-betrayal char-1 0 it’s bad doing something that causes other people to lose trust in you. agency -1 3 legal -2 char-1 hypothetical losing trust in my friend reddit/amitheasshole/aypvmz It’s bad to do something that causes other people to lose trust in you. rot/reddit/amitheasshole/aypvmz/3K5TEWLKGYQFYAIY0H6JQMIY5MEIVM/127/2 127 0 2 narrator|my friend
amitheasshole 1 dev 3 social-norms loyalty-betrayal char-0 0 expected people participating in the big events in their friends’ lives if asked. agency 0 3 legal 2 char-0 explicit-no saying no to being a bridesmaid at a friend’s wedding reddit/amitheasshole/9tzn0z People are expected to participate in the big events in their friends’ lives if asked. rot/reddit/amitheasshole/9tzn0z/3EG49X351XRR9FSSYVYCH4PEC656XX/89/1 89 39 3 narrator|a bridesmaid|a friend
amitheasshole 1 test 3 social-norms care-harm|loyalty-betrayal char-1 0 Partners should Listening to each other’s issues. agency 2 3 legal 2 char-1 probable telling my boyfriend I am bored and unhappy at my job reddit/amitheasshole/a1311q Partners should listen to each other’s issues. rot/reddit/amitheasshole/a1311q/3JV9LGBJWWT6CZ369HK2AIBAUGUGOV/111/2 111 145 2 narrator|my boyfriend

Author

Maxwell Forbes†‡ Jena D. Hwang‡ Vered Shwartz†‡ Maarten Sap† Yejin Choi†‡
†Paul G. Allen School of Computer Science & Engineering, University of Washington ‡Allen Institute for AI

Abstract

  • introduce SOCIAL-CHEM- 101, a large-scale corpus that catalogs 292k rules-of-thumb such as “It is rude to run a blender at 5am” as the basic conceptual units.
  • Each rule-of-thumb is further broken down with 12 different dimensions of people’s judgments, including social judgments of good and bad, moral foundations, expected cultural pressure, and assumed legality
    • which together amount to over 4.5 million annotations of categorical labels and free-text descriptions.
  • NEURAL NORM TRANSFORMER, learns and generalizes SOCIAL-CHEM-101 to successfully reason about previously unseen situations, generating relevant (and potentially novel) attribute-aware social rules-of-thumb
자세히 보기

A Contrastive Framework for Neural Text Generation (NeurIPS 2022)

발표자료

느낀점

  • 잘 쓴 논문 같다
  • 간단한 아이디어지만 효과적
  • decoding 시간이 생각보다 덜 걸려서 신기했음
  • contrastive 방법론이 결국 비슷한 토큰 안나오게 하겠다인데, simCTG는 MLE 로 보완되지만 디코딩 부분은 아예 비슷한걸 견제하는 식으로 나오는데도 결과가 좋게 나오는게 신기 (물론 기존의 확률아 있어서 보완이 되지만) -> degeneration penalty를 크게 줘도 ppl 결과가 좋길래 신기했음)

Note

Author

Yixuan Su* Tian Lan** Yan Wang** Dani Yogatama+ Lingpeng Kong++ Nigel Collier*
*Language Technology Lab, University of Cambridge
**Tencent AI Lab +DeepMind
++Department of Computer Science, The University of Hong Kong

자세히 보기

Learning rate & warmup step & LR scheduling

Background

요즘 딥러닝을 하다보면 수도없이 접하게 되는 단어 중 하나는 learning rate, warmup, LR scheduler와 같은 것들입니다.
이미 시중에 여러가지 기법들이 나와있고 한번은 정리해야겠다 생각했는데, 우연히 좋은 스레드를 발견하게 되서 공유해봅니다.

버트 논문에는 다음과 같은 그림이 있습니다. 여기서 밑줄쳐진 learning rate warmup과 linear decay에 대한 내용입니다.

Warmup

일반적으로 lr warmup은 말그대로 천천히 lr을 올리는 작업을 뜻합니다.
lr을 2e-5로 셋팅하고 warmup step을 10,000으로 셋팅한다면, (linear 일 경우) lr은 10,000 스텝동안 0에서 2e-5 까지 증가하게 됩니다.
코드 구현을 보면 다음과 같습니다
현재 스텝(global step)이 warmup 스텝 대비 어느정도이지 비율을 구하고 (warmup_percent_done = global_steps_float / warmup_steps_float)그에 맞는 warmup_lr을 구하는 방식으로 lr을 조절해갑니다. (warmup_learning_rate = init_lr * warmup_percent_done)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
def create_optimizer(loss, init_lr, num_train_steps, num_warmup_steps, use_tpu):
"""Creates an optimizer training op."""
global_step = tf.train.get_or_create_global_step()

learning_rate = tf.constant(value=init_lr, shape=[], dtype=tf.float32)

# Implements linear decay of the learning rate.
learning_rate = tf.train.polynomial_decay(
learning_rate,
global_step,
num_train_steps,
end_learning_rate=0.0,
power=1.0,
cycle=False)

# Implements linear warmup. I.e., if global_step < num_warmup_steps, the
# learning rate will be `global_step/num_warmup_steps * init_lr`.
if num_warmup_steps:
global_steps_int = tf.cast(global_step, tf.int32)
warmup_steps_int = tf.constant(num_warmup_steps, dtype=tf.int32)

global_steps_float = tf.cast(global_steps_int, tf.float32)
warmup_steps_float = tf.cast(warmup_steps_int, tf.float32)

warmup_percent_done = global_steps_float / warmup_steps_float
warmup_learning_rate = init_lr * warmup_percent_done

is_warmup = tf.cast(global_steps_int < warmup_steps_int, tf.float32)
learning_rate = (
(1.0 - is_warmup) * learning_rate + is_warmup * warmup_learning_rate)

# It is recommended that you use this optimizer for fine tuning, since this
# is how the model was trained (note that the Adam m/v variables are NOT
# loaded from init_checkpoint.)
optimizer = AdamWeightDecayOptimizer(
learning_rate=learning_rate,
weight_decay_rate=0.01,
beta_1=0.9,
beta_2=0.999,
epsilon=1e-6,
exclude_from_weight_decay=["LayerNorm", "layer_norm", "bias"])
자세히 보기

LLM(Large-Scale Language Model)을 위한 넓고 얕은 지식들

최근 LLM 관련 일을 하면서 익혀야할게 너무나 많다는 사실을 새삼스럽게 알게 되었다. 예전엔 아 그냥 하면 되지~ 정도로 생각했는데.. 디테일한게 생각보다 많구나 싶어서 이것 저것 많이 보기야 봤는데 머리에 남는게 없는 것 같아서 글로 간단하게 그리고 약간 어쩔수 없이 파편화된 상태로 정리해놓으려한다. 나 포함 누군가에게 도움이 되기를 바라며

PyTorch로 분산 어플리케이션 개발하기

  • 제일 먼저 볼 것!
  • 참고
  • Pytorch Multi-GPU 정리 중
  • node_gpu
  • 분산학습을 할때 로그를 찍으면 프로세스 개수만큼 찍힌다 -> 따로 처리가 필요해짐! if rank==0일때만 찍게 한다던지

code snippet

all-reduce

  • EleutherAI gpt-neox
    1
    2
    3
    4
    5
    6
    def reduce_losses(losses):
    """Reduce a tensor of losses across all GPUs."""
    reduced_losses = torch.cat([loss.clone().detach().view(1) for loss in losses])
    torch.distributed.all_reduce(reduced_losses)
    reduced_losses = reduced_losses / torch.distributed.get_world_size()
    return reduced_losses

Multi GPU & Node

자세히 보기

Efficient Training of Language Models to Fill in the Middle (FIM)

발표자료

(발표)Efficient Training of Language Models to Fill in the Middle.pdf

느낀점

  • 첫인상은 data augmentation 기법에 관련된 내용을 extensive하게 검증했다정도..?
  • free-form generation을 하고 싶다에 초점을 두고 논문 전개

Note

  • 50%란게 어떤걸까
    • 데이터셋에서 FIM으로 transformation하는 비율 (FIM 자체는 랜덤하게 짜르니까)
  • SPM에서 캐싱이 무슨 의미 일까

Author

자세히 보기

(ALiBi) TRAIN SHORT, TEST LONG: ATTENTION WITH LINEAR BIASES ENABLES INPUT LENGTH EXTRAPOLATION

Ref

Author

  • 저자: Ofir Press1,2 Noah A. Smith1,3 Mike Lewis2
    1Paul G. Allen School of Computer Science & Engineering, University of Washington 2Facebook AI Research 3Allen Institute for AI

요약

  • extrapolation 잘됨
  • 11% 빠름 속도, 11% 메모리 적게씀
  • 동일 길이로 학습한 모델 대비 짧은 길이로 학습해도 ppl 유지됨
  • 구현도 간단하다
  • position embedding 지우고 대신에 길이에 linear하게 비례해서 attention score 깎아버리자!

Abstract

자세히 보기

COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

논문파일

Ref

Author

  • 저자: Yu Meng1∗, Chenyan Xiong2, Payal Bajaj2, Saurabh Tiwary2, Paul Bennett2, Jiawei Han1, Xia Song2
    1 University of Illinois at Urbana-Champaign 2 Microsoft
    • NeurIPS 2021 논문
      image

요약

자세히 보기

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

용어

  • reduce -> 각 프로세스가 가진 값들을 특정한 하나의 process에 연산해서 모으는 연산 (하나의 값으로 모음)
  • all+* -> 이름 앞에 all이 붙으면 연산결과를 참여한 모든 프로세스가 동일하게 반환받음
  • all reduce -> 하나의 디바이스가 reduce한 값을 참여한 모든 프로세스가 동일하게 받을 수 있게 전달

논문파일

Megatron-LM- Training Multi-Billion Parameter Language Models Using Model Parallelism.pdf

Ref

Author

자세히 보기

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Author

  • 저자: Yinhan Liu∗§ Myle Ott∗§ Naman Goyal∗§ Jingfei Du∗§ Mandar Joshi† Danqi Chen§ Omer Levy§ Mike Lewis§ Luke Zettlemoyer†§ Veselin Stoyanov§
    • † Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA
    • § Facebook AI

느낀점

Abstract

  • hyperparameter choices have significant impact on the final results
  • carefully measures the impact of many key hyperparameters and training data size
  • find that BERT was significantly undertrained, and can match or exceed the performance of every model published after it

Introduction

  • We present a replication study of BERT pretraining (Devlin et al., 2019), which includes a careful evaluation of the effects of hyperparmeter tuning and training set size.
  • modifications
    • (1) training the model longer, with bigger batches, over more data;
    • (2) removing the next sentence prediction objective;
    • (3) training on longer sequences; and
    • (4) dynamically changing the masking pattern applied to the training data.
  • contributions
    • (1) We present a set of important BERT design choices and training strategies and introduce alternatives that lead to better downstream task performance;
    • (2) We use a novel dataset, CC-NEWS, and confirm that using more data for pretraining further improves performance on downstream tasks;
    • (3) Our training improvements show that masked language model pretraining, under the right design choices, is competitive with all other recently published methods.
자세히 보기

A Neural Network Solves and Generates Mathematics Problems by Program Synthesis: Calculus, Differential Equations, Linear Algebra, and More

Author

  • 저자:
    • Iddo Drori1,a,b, Sunny Trana, Roman Wangb, Newman Chengb, Kevin Liua, Leonard Tangc, Elizabeth Kea, Nikhil Singha, Taylor L. Pattic, Jayson Lynchd, Avi Shporera, Nakul Vermab, Eugene Wub, and Gilbert Strang(아니 그 유명한 길버트 스트랭..)a
    • aMIT; bColumbia University; cHarvard University; dUniversity of Waterloo

느낀점

  • large scale 모델이 생각보다 할줄아는게 많다는걸 알게됨.. 코드로 파인튜닝하면 수학문제 푸는 코드도 만드는구나 (그런 코드가 깃헙에 있었겠지만..!)

Abstract

  • program synthesis을 통해 PLM & code에 finetune된 모델(Codex Transformer model)이 수학문제를 풀수있음을 논함
  • university-level Mathematics course questions을 생성하는 연구(?)

Introduction

자세히 보기