亚洲AV无码一区二区乱子伦AS,国产精品水嫩水嫩,亚州熟妇视频无码

原文： https://pytorch.org/tutorials/beginner/text_sentiment_ngrams_tutorial.html

本教程介紹了如何使用torchtext中的文本分類數(shù)據(jù)集，包括

- AG_NEWS,
- SogouNews,
- DBpedia,
- YelpReviewPolarity,
- YelpReviewFull,
- YahooAnswers,
- AmazonReviewPolarity,
- AmazonReviewFull

本示例說明了如何使用這些TextClassification數(shù)據(jù)集之一訓(xùn)練用于分類的監(jiān)督學(xué)習(xí)算法。

用 ngram 加載數(shù)據(jù)

一些 ngrams 功能用于捕獲有關(guān)本地單詞順序的一些部分信息。在實踐中，應(yīng)用二元語法或三元語法作為單詞組比僅僅一個單詞提供更多的好處。一個例子：

"load data with ngrams"
Bi-grams results: "load data", "data with", "with ngrams"
Tri-grams results: "load data with", "data with ngrams"

TextClassification數(shù)據(jù)集支持 ngrams 方法。通過將 ngrams 設(shè)置為 2，數(shù)據(jù)集中的示例文本將是一個單字加 bi-grams 字符串的列表。

import torch
import torchtext
from torchtext.datasets import text_classification
NGRAMS = 2
import os
if not os.path.isdir('./.data'):
    os.mkdir('./.data')
train_dataset, test_dataset = text_classification.DATASETS['AG_NEWS'](
    root='./.data', ngrams=NGRAMS, vocab=None)
BATCH_SIZE = 16
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

定義模型

該模型由 EmbeddingBag 層和線性層組成(請參見下圖）。 nn.EmbeddingBag計算嵌入“袋”的平均值。此處的文本條目具有不同的長度。 nn.EmbeddingBag此處不需要填充，因為文本長度以偏移量保存。

另外，由于nn.EmbeddingBag會動態(tài)累積嵌入中的平均值，因此nn.EmbeddingBag可以提高性能和存儲效率，以處理張量序列。

../_images/text_sentiment_ngrams_model.png

import torch.nn as nn
import torch.nn.functional as F
class TextSentiment(nn.Module):
    def __init__(self, vocab_size, embed_dim, num_class):
        super().__init__()
        self.embedding = nn.EmbeddingBag(vocab_size, embed_dim, sparse=True)
        self.fc = nn.Linear(embed_dim, num_class)
        self.init_weights()


    def init_weights(self):
        initrange = 0.5
        self.embedding.weight.data.uniform_(-initrange, initrange)
        self.fc.weight.data.uniform_(-initrange, initrange)
        self.fc.bias.data.zero_()


    def forward(self, text, offsets):
        embedded = self.embedding(text, offsets)
        return self.fc(embedded)

啟動實例

AG_NEWS 數(shù)據(jù)集具有四個標(biāo)簽，因此類別數(shù)是四個。

1 : World
2 : Sports
3 : Business
4 : Sci/Tec

詞匯的大小等于詞匯的長度(包括單個單詞和 ngram）。類的數(shù)量等于標(biāo)簽的數(shù)量，在 AG_NEWS 情況下為 4。

VOCAB_SIZE = len(train_dataset.get_vocab())
EMBED_DIM = 32
NUN_CLASS = len(train_dataset.get_labels())
model = TextSentiment(VOCAB_SIZE, EMBED_DIM, NUN_CLASS).to(device)

用于生成批處理的函數(shù)

由于文本條目的長度不同，因此使用自定義函數(shù) generate_batch(）生成數(shù)據(jù)批和偏移量。該功能將傳遞到torch.utils.data.DataLoader中的collate_fn。 collate_fn的輸入是張量為 list_batch_size 的張量列表，collate_fn函數(shù)將它們打包成一個小批量。請注意此處，并確保將collate_fn聲明為頂級 def。這樣可以確保該功能在每個工作程序中均可用。

原始數(shù)據(jù)批處理輸入中的文本條目打包到一個列表中，并作為單個張量級聯(lián)，作為nn.EmbeddingBag的輸入。偏移量是定界符的張量，表示文本張量中各個序列的起始索引。 Label 是一個張量，用于保存單個文本條目的標(biāo)簽。

def generate_batch(batch):
    label = torch.tensor([entry[0] for entry in batch])
    text = [entry[1] for entry in batch]
    offsets = [0] + [len(entry) for entry in text]
    # torch.Tensor.cumsum returns the cumulative sum
    # of elements in the dimension dim.
    # torch.Tensor([1.0, 2.0, 3.0]).cumsum(dim=0)


    offsets = torch.tensor(offsets[:-1]).cumsum(dim=0)
    text = torch.cat(text)
    return text, offsets, label

定義功能以訓(xùn)練模型并評估結(jié)果。

建議 PyTorch 用戶使用 torch.utils.data.DataLoader ，它可以輕松地并行加載數(shù)據(jù)(教程為，此處為）。我們在此處使用DataLoader加載 AG_NEWS 數(shù)據(jù)集并將其發(fā)送到模型以進(jìn)行訓(xùn)練/驗證。

from torch.utils.data import DataLoader


def train_func(sub_train_):


    # Train the model
    train_loss = 0
    train_acc = 0
    data = DataLoader(sub_train_, batch_size=BATCH_SIZE, shuffle=True,
                      collate_fn=generate_batch)
    for i, (text, offsets, cls) in enumerate(data):
        optimizer.zero_grad()
        text, offsets, cls = text.to(device), offsets.to(device), cls.to(device)
        output = model(text, offsets)
        loss = criterion(output, cls)
        train_loss += loss.item()
        loss.backward()
        optimizer.step()
        train_acc += (output.argmax(1) == cls).sum().item()


    # Adjust the learning rate
    scheduler.step()


    return train_loss / len(sub_train_), train_acc / len(sub_train_)


def test(data_):
    loss = 0
    acc = 0
    data = DataLoader(data_, batch_size=BATCH_SIZE, collate_fn=generate_batch)
    for text, offsets, cls in data:
        text, offsets, cls = text.to(device), offsets.to(device), cls.to(device)
        with torch.no_grad():
            output = model(text, offsets)
            loss = criterion(output, cls)
            loss += loss.item()
            acc += (output.argmax(1) == cls).sum().item()


    return loss / len(data_), acc / len(data_)

分割數(shù)據(jù)集并運(yùn)行模型

由于原始 AG_NEWS 沒有有效的數(shù)據(jù)集，因此我們將訓(xùn)練數(shù)據(jù)集分為訓(xùn)練/有效集，其分割比率為 0.95(訓(xùn)練）和 0.05(有效）。在這里，我們在 PyTorch 核心庫中使用 torch.utils.data.dataset.random_split 函數(shù)。

CrossEntropyLoss 標(biāo)準(zhǔn)將 nn.LogSoftmax(）和 nn.NLLLoss(）合并到一個類中。在訓(xùn)練帶有 C 類的分類問題時很有用。 SGD 實現(xiàn)了隨機(jī)梯度下降方法作為優(yōu)化程序。初始學(xué)習(xí)率設(shè)置為 4.0。 StepLR 在此處用于通過歷時調(diào)整學(xué)習(xí)率。

import time
from torch.utils.data.dataset import random_split
N_EPOCHS = 5
min_valid_loss = float('inf')


criterion = torch.nn.CrossEntropyLoss().to(device)
optimizer = torch.optim.SGD(model.parameters(), lr=4.0)
scheduler = torch.optim.lr_scheduler.StepLR(optimizer, 1, gamma=0.9)


train_len = int(len(train_dataset) * 0.95)
sub_train_, sub_valid_ = \
    random_split(train_dataset, [train_len, len(train_dataset) - train_len])


for epoch in range(N_EPOCHS):


    start_time = time.time()
    train_loss, train_acc = train_func(sub_train_)
    valid_loss, valid_acc = test(sub_valid_)


    secs = int(time.time() - start_time)
    mins = secs / 60
    secs = secs % 60


    print('Epoch: %d' %(epoch + 1), " | time in %d minutes, %d seconds" %(mins, secs))
    print(f'\tLoss: {train_loss:.4f}(train)\t|\tAcc: {train_acc * 100:.1f}%(train)')
    print(f'\tLoss: {valid_loss:.4f}(valid)\t|\tAcc: {valid_acc * 100:.1f}%(valid)')

得出：

Epoch: 1  | time in 0 minutes, 9 seconds
        Loss: 0.0263(train)     |       Acc: 84.6%(train)
        Loss: 0.0000(valid)     |       Acc: 90.1%(valid)
Epoch: 2  | time in 0 minutes, 9 seconds
        Loss: 0.0120(train)     |       Acc: 93.6%(train)
        Loss: 0.0001(valid)     |       Acc: 91.4%(valid)
Epoch: 3  | time in 0 minutes, 9 seconds
        Loss: 0.0070(train)     |       Acc: 96.4%(train)
        Loss: 0.0001(valid)     |       Acc: 91.7%(valid)
Epoch: 4  | time in 0 minutes, 9 seconds
        Loss: 0.0039(train)     |       Acc: 98.0%(train)
        Loss: 0.0001(valid)     |       Acc: 91.4%(valid)
Epoch: 5  | time in 0 minutes, 9 seconds
        Loss: 0.0023(train)     |       Acc: 99.0%(train)
        Loss: 0.0001(valid)     |       Acc: 91.7%(valid)

使用以下信息在 GPU 上運(yùn)行模型：

紀(jì)元：1 | 時間在 0 分鐘 11 秒

Loss: 0.0263(train)     |       Acc: 84.5%(train)
Loss: 0.0001(valid)     |       Acc: 89.0%(valid)

紀(jì)元：2 | 時間在 0 分鐘 10 秒內(nèi)

Loss: 0.0119(train)     |       Acc: 93.6%(train)
Loss: 0.0000(valid)     |       Acc: 89.6%(valid)

紀(jì)元：3 | 時間在 0 分鐘 9 秒

Loss: 0.0069(train)     |       Acc: 96.4%(train)
Loss: 0.0000(valid)     |       Acc: 90.5%(valid)

紀(jì)元：4 | 時間在 0 分鐘 11 秒

Loss: 0.0038(train)     |       Acc: 98.2%(train)
Loss: 0.0000(valid)     |       Acc: 90.4%(valid)

紀(jì)元：5 | 時間在 0 分鐘 11 秒

Loss: 0.0022(train)     |       Acc: 99.0%(train)
Loss: 0.0000(valid)     |       Acc: 91.0%(valid)

使用測試數(shù)據(jù)集評估模型

print('Checking the results of test dataset...')
test_loss, test_acc = test(test_dataset)
print(f'\tLoss: {test_loss:.4f}(test)\t|\tAcc: {test_acc * 100:.1f}%(test)')

得出:

Checking the results of test dataset...
        Loss: 0.0003(test)      |       Acc: 91.1%(test)

正在檢查測試數(shù)據(jù)集的結(jié)果…

Loss: 0.0237(test)      |       Acc: 90.5%(test)

測試隨機(jī)新聞

使用到目前為止最好的模型并測試高爾夫新聞。標(biāo)簽信息在可用。

import re
from torchtext.data.utils import ngrams_iterator
from torchtext.data.utils import get_tokenizer


ag_news_label = {1 : "World",
                 2 : "Sports",
                 3 : "Business",
                 4 : "Sci/Tec"}


def predict(text, model, vocab, ngrams):
    tokenizer = get_tokenizer("basic_english")
    with torch.no_grad():
        text = torch.tensor([vocab[token]
                            for token in ngrams_iterator(tokenizer(text), ngrams)])
        output = model(text, torch.tensor([0]))
        return output.argmax(1).item() + 1


ex_text_str = "MEMPHIS, Tenn. – Four days ago, Jon Rahm was \
    enduring the season's worst weather conditions on Sunday at The \
    Open on his way to a closing 75 at Royal Portrush, which \
    considering the wind and the rain was a respectable showing. \
    Thursday's first round at the WGC-FedEx St. Jude Invitational \
    was another story. With temperatures in the mid-80s and hardly any \
    wind, the Spaniard was 13 strokes better in a flawless round. \
    Thanks to his best putting performance on the PGA Tour, Rahm \
    finished with an 8-under 62 for a three-stroke lead, which \
    was even more impressive considering he'd never played the \
    front nine at TPC Southwind."


vocab = train_dataset.get_vocab()
model = model.to("cpu")


print("This is a %s news" %ag_news_label[predict(ex_text_str, model, vocab, 2)])

得出:

This is a Sports news

這是體育新聞

您可以在此處找到本說明中顯示的代碼示例。

腳本的總運(yùn)行時間：(1 分鐘 26.276 秒）

Download Python source code: text_sentiment_ngrams_tutorial.py Download Jupyter notebook: text_sentiment_ngrams_tutorial.ipynb

由獅身人面像畫廊生成的畫廊

PyTorch 使用 TorchText 進(jìn)行文本分類

用 ngram 加載數(shù)據(jù)

定義模型

啟動實例

用于生成批處理的函數(shù)

定義功能以訓(xùn)練模型并評估結(jié)果。

分割數(shù)據(jù)集并運(yùn)行模型

使用測試數(shù)據(jù)集評估模型

測試隨機(jī)新聞

推薦文章

推薦教程

推薦課程

定義功能以訓(xùn)練模型并評估結(jié)果。