Comments on U.S. National AI Research Resource Interim Report
AI 速览 美国国家AI研究资源中期报告发布，相关方已提交评论意见。
人工智能
围绕「人工智能」分类汇集的全球近期技术动态与前沿报道。
Efficient training of language models to fill in the middle
AI 速览 OpenAI提出“填空”（Fill in the Middle）训练方法，可高效训练语言模型根据上下文补全中间内容。
A hazard analysis framework for code synthesis large language models
AI 速览 提出面向代码生成大语言模型的危害分析框架。
Deploying TensorFlow Vision Models in Hugging Face with TF Serving
AI 速览 Hugging Face与TensorFlow Serving集成，支持部署TensorFlow视觉模型。
How to train your model dynamically using adversarial data
AI 速览 利用对抗数据动态训练模型的方法。
Introducing The World's Largest Open Multilingual Language Model: BLOOM
AI 速览 BigScience发布1760亿参数开源多语言模型BLOOM，为全球最大开放多语言语言模型。
DALL·E 2 pre-training mitigations
In order to share the magic of DALL·E 2 with a broad audience, we needed to reduce the risks associated with powerful image generation models. To this end, we put various guardrails in place to prevent generated images from violating our content policy.
Accelerate Large Model Training using DeepSpeed
AI 速览 DeepSpeed可加速大模型训练。
Learning to play Minecraft with Video PreTraining
We trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled video dataset of human Minecraft play, while using only a small amount of labeled contractor data. With fine-tuning, our model can learn to craft diamond tools, a task that usually takes proficient humans over 20 minutes (24,000 actions). Our model uses the native human interface of keypresses and mouse movements, making it quite general, and represents a step towards general computer-using agents.
Evolution through large models
AI 速览 通过大型模型实现进化。
AI-written critiques help humans notice flaws
We trained “critique-writing” models to describe flaws in summaries. Human evaluators find flaws in summaries much more often when shown our model’s critiques. Larger models are better at self-critiquing, with scale improving critique-writing more than summary-writing. This shows promise for using AI systems to assist human supervision of AI systems on difficult tasks.
Techniques for training large neural networks
Large neural networks are at the core of many recent advances in AI, but training them is a difficult engineering and research challenge which requires orchestrating a cluster of GPUs to perform a single synchronized calculation.
The Annotated Diffusion Model
AI 速览 《注释版扩散模型》发布，提供扩散模型的详细注释解析。
Best practices for deploying language models
Cohere, OpenAI, and AI21 Labs have developed a preliminary set of best practices applicable to any organization developing or deploying large language models.
Teaching models to express their uncertainty in words
AI 速览 研究提出训练语言模型用自然语言表达自身不确定性，而非仅输出概率数值。
Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel
AI 速览 PyTorch Fully Sharded Data Parallel（FSDP）可加速大模型训练。
Habana Labs and Hugging Face Partner to Accelerate Transformer Model Training
AI 速览 Habana Labs与Hugging Face达成合作，共同加速Transformer模型训练。
Announcing the 🤗 AI Research Residency Program
AI 速览 Hugging Face宣布推出AI研究驻留计划（AI Research Residency Program）。
Fine-Tune a Semantic Segmentation Model with a Custom Dataset
AI 速览 使用自定义数据集微调语义分割模型。
New GPT-3 capabilities: Edit & insert
We’ve released new versions of GPT-3 and Codex which can edit or insert content into existing text, rather than just completing existing text.
Lessons learned on language model safety and misuse
We describe our latest thinking in the hope of helping other AI developers address safety and misuse of deployed models.
Economic impacts research at OpenAI
Call for expressions of interest to study the economic impacts of large language models.
A research agenda for assessing the economic impacts of code generation models
AI 速览 研究议程：评估代码生成模型的经济影响。
BERT 101 - State Of The Art NLP Model Explained
AI 速览 BERT是一种基于Transformer架构的预训练语言模型，通过双向编码器表示实现上下文感知的词嵌入，在多项NLP任务上达到当时最优性能。
Aligning language models to follow instructions
We’ve trained language models that are much better at following user intentions than GPT-3 while also making them more truthful and less toxic, using techniques developed through our alignment research. These InstructGPT models, which are trained with humans in the loop, are now deployed as the default language models on our API.
Introducing text and code embeddings
We are introducing embeddings, a new endpoint in the OpenAI API that makes it easy to perform natural language and code tasks like semantic search, clustering, topic modeling, and classification.
Deploy GPT-J 6B for inference using Hugging Face Transformers and Amazon SageMaker
AI 速览 Hugging Face Transformers与Amazon SageMaker可用于部署GPT-J 6B模型进行推理。
WebGPT: Improving the factual accuracy of language models through web browsing
We’ve fine-tuned GPT-3 to more accurately answer open-ended questions using a text-based web browser.
Perceiver IO: a scalable, fully-attentional model that works on any modality
AI 速览 DeepMind推出Perceiver IO，一种可扩展的全注意力模型，能处理任意模态数据。
Customizing GPT-3 for your application
Fine-tune with a single command.