huggingface-transformers

Suppress HuggingFace logging warning: “Setting `pad_token_id` to `eos_token_id`:{eos_token_id} for open-end generation.”

April 2, 2024 by Tarik

How to download model from huggingface?

November 27, 2023 by Tarik

what’s difference between tokenizer.encode and tokenizer.encode_plus in Hugging Face

September 1, 2023 by Tarik

How to change huggingface transformers default cache directory

July 16, 2023 by Tarik

Load a pre-trained model from disk with Huggingface Transformers

June 9, 2023 by Tarik

Where is the file located relative to your model folder? I believe it has to be a relative PATH rather than an absolute one. So if your file where you are writing the code is located in ‘my/local/’, then your code should be like so: PATH = ‘models/cased_L-12_H-768_A-12/’ tokenizer = BertTokenizer.from_pretrained(PATH, local_files_only=True) You just need … Read more

Where does hugging face’s transformers save models?

May 20, 2023 by Tarik

Update 2023-05-02: The cache location has changed again, and is now ~/.cache/huggingface/hub/, as reported by @Victor Yan. Notably, the sub folders in the hub/ directory are also named similar to the cloned model path, instead of having a SHA hash, as in previous versions. Update 2021-03-11: The cache location has now changed, and is located … Read more