国际清算银行-CB-LM：中央银行的语言模型（英）-33份

BIS Working PapersNo 1215 CB-LMs: language models for central banking by Leonardo Gambacorta, Byeungchun Kwon, Taejin Park, Pietro Patelli, Sonya Zhu Monetary and Economic Department October 2024 JEL classification: E58, C55, C63, G17 Keywords: large language models, gen AI, central banks, monetary policy analysis BIS Working Papers are written by members of the Monetary and Economic Department of the Bank for International Settlements, and from time to time by other economists, and are published by the Bank. The papers are on subjects of topical interest and are technical in character. The views expressed in them are those of their authors and not necessarily the views of the BIS. This publication is available on the BIS website (www.bis.org). © Bank for International Settlements 2024. All rights reserved. Brief excerpts may be reproduced or translated provided the source is stated. ISSN 1020-0959 (print) ISSN 1682-7678 (online) 1 CB-LMs: language models for central banking Leonardo Gambacorta+*, Byeungchun Kwon+, Taejin Park+, Pietro Patelli+, Sonya Zhu+ + Bank for International Settlements (BIS), * CEPRAbstract We introduce central bank language models (CB-LMs) — specialised encoder-only language models retrained on a comprehensive corpus of central bank speeches, policy documents and research papers. We show that CB-LMs outperform their foundational models in predicting masked words in central bank idioms. Some CB-LMs not only outperform their foundational models, but also surpass state-of-the-art generative Large Language Models (LLMs) in classifying monetary policy stance from Federal Open Market Committee (FOMC) statements. In more complex scenarios, requiring sentiment classification of extensive news related to the US monetary policy, we find that the largest LLMs outperform the domain-adapted encoder-only models. However, deploying such large LLMs presents substantial challenges for central banks in terms of confidentiality, transparency, replicability and cost-efficiency. JEL Classification: E58, C55, C63, G17. Keywords: large language models, gen AI, central banks, monetary policy analysis We thank Douglas Araujo, Flavio Bazzana, Claudia Biancotti, Carolina Camassa, Chris Cox, Fernando Perez-Cruz, and seminar participants at the BIS and Generative AI in Finance Paper Development Workshop at TU Dresden for useful comments and suggestions. The views expressed in this article are those of the authors and do not necessarily reflect those of the BIS. 2 1. Introduction Communication is becoming an increasingly important tool for central banks to manage public expectation. Leveraging the power of language models, there is a growing body of economic literature applying Natural Language Processing (NLP) techniques to decipher central bank communication. Words, vocal tones and body languages have all been found as effective channels for central banks to manage public expectations. While these studies have made valuable contributions, most