Create the named tuple outside of the function: from collections import namedtuple import pickle P = namedtuple(“P”, “one two three four”) def pickle_test(): my_list = [] abe = P(“abraham”, “lincoln”, “vampire”, “hunter”) my_list.append(abe) with open(‘abe.pickle’, ‘wb’) as f: pickle.dump(abe, f) pickle_test() Now pickle can find it; it is a module global now. When unpickling, all … Read more
I got this error when running in an jupyter notebook. I think the problem was that I was using %load_ext autoreload autoreload 2. Restarting my kernel and rerunning solved the problem.
Yes, python can pickle lambda functions… but only if you have something that uses copy_reg to register how to pickle lambda functions — the package dill loads the copy_reg you need into the pickle registry for you, when you import dill. Python 2.7.8 (default, Jul 13 2014, 02:29:54) [GCC 4.2.1 Compatible Apple Clang 4.1 ((tags/Apple/clang-421.11.66))] … Read more
cPickle comes with the standard library… in python 2.x. You are on python 3.x, so if you want cPickle, you can do this: >>> import _pickle as cPickle However, in 3.x, it’s easier just to use pickle. No need to install anything. If something requires cPickle in python 3.x, then that’s probably a bug.
For pandas users who saved a dataframe to a pickle file with protocol 5 in python 3.8 and need to load it into python 3.6 which only supports protocol 4 (I’m looking at you google colab): !pip3 install pickle5 import pickle5 as pickle with open(path_to_protocol5, “rb”) as fh: data = pickle.load(fh) Could also save into … Read more
Use the latest protocol that supports the lowest Python version you want to support reading the data. Newer protocol versions support new language features and include optimisations. From the pickle module data format documentation: There are currently 6 different protocols which can be used for pickling. The higher the protocol used, the more recent the … Read more
This is what I ended up doing. Updating the __dict__ means we keep any new member variables I add to the class and just update the ones that were there when the object was last pickle’d. It seems the simplest while maintaining the saving and loading code inside the class itself so calling code just … Read more
As pickle’s docs say, in order to save and restore a class instance (actually a function, too), you must respect certain constraints: pickle can save and restore class instances transparently, however the class definition must be importable and live in the same module as when the object was stored whyteboard.tools is not the “the same … Read more
pickle is for serializing some object (or objects) as a single bytestream in a file. shelve builds on top of pickle and implements a serialization dictionary where objects are pickled, but associated with a key (some string), so you can load your shelved data file and access your pickled objects via keys. This could be … Read more
Two additions to Tim Peters’ accepted answer. First, you need not store the number of items you pickled separately if you stop loading when you hit the end of the file: def loadall(filename): with open(filename, “rb”) as f: while True: try: yield pickle.load(f) except EOFError: break items = loadall(myfilename) This assumes the file contains only … Read more