easynlp.appzoo

Data Augmentation

class easynlp.appzoo.data_augmentation.model.DataAugmentation(pretrained_model_name_or_path=None, **kwargs)

Pre-trained model for MLM data enhancement. Similar to the one used in LanguageModeling.

forward(inputs)

Feature Vectorization

class easynlp.appzoo.feature_vectorization.model.FeatureVectorization(pretrained_model_name_or_path=None, **kwargs)

forward(inputs)

Language Modeling

class easynlp.appzoo.language_modeling.model.LanguageModeling(pretrained_model_name_or_path=None, **kwargs)

forward(inputs)

compute_loss(forward_outputs, label_ids, insert_know_labels=None)

Sequence Classification

class easynlp.appzoo.sequence_classification.model.SequenceClassification(pretrained_model_name_or_path=None, **kwargs)

init_weights()

forward(inputs)

compute_loss(forward_outputs, label_ids, **kwargs)

class easynlp.appzoo.sequence_classification.model.DistillatorySequenceClassification(pretrained_model_name_or_path=None, **kwargs)

class easynlp.appzoo.sequence_classification.model.SequenceMultiLabelClassification(pretrained_model_name_or_path=None, **kwargs)

The application class for multi-label text classification

compute_loss(forward_outputs, label_ids)

The loss function for multi-label text classification. We use multi-label cross-entropy function rather than the vanilla cross-entropy function.

class easynlp.appzoo.sequence_classification.model.FewshotSequenceClassification(pretrained_model_name_or_path=None, user_defined_parameters=None, **kwargs)

class easynlp.appzoo.sequence_classification.model.CptFewshotSequenceClassification(pretrained_model_name_or_path=None, user_defined_parameters=None, **kwargs)

GEEP Classification

class easynlp.appzoo.geep_classification.model.MultiHeadedAttention(hidden_size, heads_num, dropout)

Each head is a self-attention operation. self-attention refers to https://arxiv.org/pdf/1706.03762.pdf

forward(key, value, query)

Parameters:

• key -- [batch_size x seq_length x hidden_size]
• value -- [batch_size x seq_length x hidden_size]
• query -- [batch_size x seq_length x hidden_size]
• mask -- [batch_size x 1 x seq_length x seq_length]

Returns:

[batch_size x seq_length x hidden_size]

Return type:

output

class easynlp.appzoo.geep_classification.model.GEEPClassifier(input_size, labels_num)

Classifiers for early exit.

forward(hidden)

easynlp.appzoo.geep_classification.model.attr_set(classifiers, key, val)

Load weight for classifiers.

class easynlp.appzoo.geep_classification.model.GEEPClassification(pretrained_model_name_or_path, user_defined_parameters, **kwargs)

GEEPClassification: a hybrid architechture including a BERT-ish backbone and multiple early-exit classifiers. You sholud provide following user_defined_parameters: user_defined_parameters['geep_exit_num']: Required for training. The number of early-exit classifiers, these classifiers receive each Transformer Layer output from bootom to top. user_defined_parameters['geep_threshold']: Required for inference, between 0 and 1. It is the threshold for the normalized cross entropy. Smaller value for higher accuracy and slower inference time.

classmethod from_pretrained(pretrained_model_name_or_path, user_defined_parameters={}, **kwargs)

forward(inputs)

compute_loss(forward_outputs, label_ids, **kwargs)

Sequence Labeling

class easynlp.appzoo.sequence_labeling.model.SequenceLabeling(pretrained_model_name_or_path=None, **kwargs)

init_weights()

forward(inputs)

compute_loss(forward_outputs, label_ids)

Text Matching

class easynlp.appzoo.text_match.model.TextMatch(pretrained_model_name_or_path=None, **kwargs)

init_weights()

forward(inputs)

compute_loss(forward_outputs, label_ids)

class easynlp.appzoo.text_match.model.TextMatchTwoTowerV1(pretrained_model_name_or_path=None, **kwargs)

init_weights()

forward(inputs)

compute_loss(forward_outputs, label_ids)

class easynlp.appzoo.text_match.model.TextMatchTwoTower(pretrained_model_name_or_path=None, **kwargs)

init_weights()

forward(inputs)

compute_loss(forward_outputs, label_ids)

class easynlp.appzoo.text_match.model.DistillatoryTextMatch(pretrained_model_name_or_path=None, **kwargs)

class easynlp.appzoo.text_match.model.FewshotSingleTowerTextMatch(pretrained_model_name_or_path=None, user_defined_parameters=None, **kwargs)

class easynlp.appzoo.text_match.model.CptFewshotSingleTowerTextMatch(pretrained_model_name_or_path=None, user_defined_parameters=None, **kwargs)