026-04

2021-03-03 (水) 08:23:24 (67d) | Topic path: Top / 026-04

第26回研究会

FinMegatron: Large Financial Domain Language Models

著者

Xianchao Wu (NVIDIA)

概要

General domain pretrained large-scale language models, such as BERT and GPT3, have achieved state-of-the-art results among numerous NLP classification and generation applications. These pretraining technology is also willing to be used in vertical domains, such as finance. The downstream applications include financial event extraction from news, summarization, and causal inferencing. In this paper, we propose large-scale pretrained models for financial domain in English and Japanese languages. The original datasets come from professional financial news. We empirically study the factors of sub-word vocabulary set, model size and pre-trained corpus and their impacts to the downstream financial NLP applications, such as named entity recognition, relation extraction, causal inferencing and question answering. We will release our models and codes for open-usage.

キーワード

financial language models, BERT, GPT

論文

file04_SIG-FIN-26.pdf

添付ファイル: file04_SIG-FIN-26.pdf 238件 [詳細]
トップ   編集 凍結 差分 バックアップ 添付 複製 名前変更 リロード   新規 一覧 単語検索 最終更新   ヘルプ   最終更新のRSS