2/27/2023

save tokeniser from AutoTokenizer, AutoProcessor

 refer to code:

.

from transformers import AutoTokenizer, AutoProcessor

model_name_or_path = 'nielsr/layoutlmv3-finetuned-funsd'
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
processor = AutoProcessor.from_pretrained(model_name_or_path)

# Save tokenizer
tokenizer.save_pretrained('tokenizer_dir')

# Save processor
processor.save_pretrained('processor_dir')

..


πŸ™‡πŸ»‍♂️

www.marearts.com

No comments:

Post a Comment