Improved language models and what they mean
This article explains OpenAI's GPT-2, a large-scale unsupervised language model trained only to predict the next word. Despite no task-specific training, it generated coherent paragraphs, set language…
GPT-2
Training GPT-2 Based on Human Feedback
This article reviews OpenAI's early experiment fine-tuning a 774M-parameter GPT-2 model using human feedback. Labelers taught the model to match their preferences, but narrow instructions led to unint…