RBERT for Relation Extraction task for KLUE

Project Description

  • Relation Extraction task is one of the task of Korean Language Understanding Evaluation(KLUE) Benchmark.
  • Relation extraction can be defined as multiclass classification task for relationship between subject entity and object entity.
  • Classes are such as no_relation, per:employee_of, org:founded_by... totaling 30 labels.
  • This repo contains custom fine-tuning method utilizing monologg's R-BERT Implementation.
  • Custom punctuations with Pororo NER has been added to the dataset prior to the model's training.
  • If you want to refer to the experimentation note such as punctuation method of the entity, please refer to the blog post

Arguments Usage

batch_sizeint40batch size for training and inferece
num_foldsint5number of fold for Stratified KFold
num_train_epochsint5number of epochs for training
lossstrfocallossloss function
gammafloat1.0focalloss's gamma value
optimizerstradampoptimizer for training
schedulerstrget_cosine_schedule_with_warmuplearning rate scheduler
learning_ratefloat0.00005initial learning rate
weight_decayfloat0.01Loss function's weight decay, preventing overfit
debugboolfalsedebug with CPU device for better error representation
save_stepsint100number of steps for saving the model
evaluation_stepsint100number of step until the evaluation
metric_for_best_modelstreval/lossthe metric for determining which is the best model





  • GPU : Tesla V100 32GB
GitHub - snoop2head/KLUE-RBERT at pythonawesome.com
Utilizing RBERT model for KLUE Relation Extraction task - GitHub - snoop2head/KLUE-RBERT at pythonawesome.com