Here’s a very simple example for Python that should supplement the pickle docs. class Foo(object): def __init__(self, val=2): self.val = val def __getstate__(self): print(“I’m being pickled”) self.val *= 2 return self.__dict__ def __setstate__(self, d): print(“I’m being unpickled with these values: ” + repr(d)) self.__dict__ = d self.val *= 3 import pickle f = Foo() f_data … Read more
Some uses that I have come across: 1) saving a program’s state data to disk so that it can carry on where it left off when restarted (persistence) 2) sending python data over a TCP connection in a multi-core or distributed system (marshalling) 3) storing python objects in a database 4) converting an arbitrary python … Read more
Most of the answers here have dealt with how to mange EOFError exceptions, which is really handy if you’re unsure about whether the pickled object is empty or not. However, if you’re surprised that the pickle file is empty, it could be because you opened the filename through ‘wb’ or some other mode that could … Read more
The problem is that multiprocessing must pickle things to sling them among processes, and bound methods are not picklable. The workaround (whether you consider it “easy” or not;-) is to add the infrastructure to your program to allow such methods to be pickled, registering it with the copy_reg standard library method. For example, Steven Bethard’s … Read more
You’ll have to tell pickle.load() how to convert Python bytestring data to Python 3 strings, or you can tell pickle to leave them as bytes. The default is to try and decode all string data as ASCII, and that decoding fails. See the pickle.load() documentation: Optional keyword arguments are fix_imports, encoding and errors, which are … Read more
I prefer JSON over pickle for my serialization. Unpickling can run arbitrary code, and using pickle to transfer data between programs or store data between sessions is a security hole. JSON does not introduce a security hole and is standardized, so the data can be accessed by programs in different languages if you ever need … Read more
Pickle serializes a single object at a time, and reads back a single object – the pickled data is recorded in sequence on the file. If you simply do pickle.load you should be reading the first object serialized into the file (not the last one as you’ve written). After unserializing the first object, the file-pointer … Read more
I’ve compared performance (space and time) for a number of ways to store numpy arrays. Few of them support multiple arrays per file, but perhaps it’s useful anyway. Npy and binary files are both really fast and small for dense data. If the data is sparse or very structured, you might want to use npz … Read more
If you are getting this error in python3, then, it could be an incompatibility issue between python 2 and python 3, for me the solution was to load with latin1 encoding: pickle.load(file, encoding=’latin1′)
As for your second problem: Traceback (most recent call last): File “<stdin>”, line 1, in <module> File “C:\Python31\lib\pickle.py”, line 1365, in load encoding=encoding, errors=errors).load() EOFError After you have read the contents of the file, the file pointer will be at the end of the file – there will be no further data to read. You … Read more