r.raw (HTTPResponse) is already a file-like object (just pass stream=True): #!/usr/bin/env python import sys import requests # $ pip install requests from PIL import Image # $ pip install pillow url = sys.argv[1] r = requests.get(url, stream=True) r.raw.decode_content = True # Content-Encoding im = Image.open(r.raw) #NOTE: it requires pillow 2.8+ print(im.format, im.mode, im.size) In general … Read more
Calling mmap(2) once per memory allocation is not a viable approach for a general purpose memory allocator because the allocation granularity (the smallest individual unit which may be allocated at a time) for mmap(2) is PAGESIZE (usually 4096 bytes), and because it requires a slow and complicated syscall. The allocator fast path for small allocations … Read more
Be careful. This works at first blush, but it directly writes to a register that the GPIO controller driver believes it owns. It will cause odd and hard to track down side effects, either on this GPIO line or on a GPIO that is in the same bank. For this to work reliably you need … Read more
If you open and mmap() a regular file, data will end up in that file. If you just need to share a memory region, without the need to persist the data, which incurs extra I/O overhead, use shm_open(). Such a memory region would also allow you to store other kinds of objects such as mutexes … Read more
mmap can be used for a few things. First, a file-backed mapping. Instead of allocating memory with malloc and reading the file, you map the whole file into memory without explicitly reading it. Now when you read from (or write to) that memory area, the operations act on the file, transparently. Why would you want … Read more
If we change the program to see where the malloc‘d memory is: #include <unistd.h> #include <stdio.h> #include <stdlib.h> void program_break_test() { printf(“%10p\n”, sbrk(0)); char *bl = malloc(1024 * 1024); printf(“%10p\n”, sbrk(0)); printf(“malloc’d at: %10p\n”, bl); free(bl); printf(“%10p\n”, sbrk(0)); } int main(int argc, char **argv) { program_break_test(); return 0; } It’s perhaps a bit clearer that … Read more
2 will work but you don’t have to rely on the OS happening to have space available, you can reserve your address space beforehand so your fixed mmapings will always succeed. For instance, To reserve one gigabyte of address space. Do a mmap(NULL, 1U << 30, PROT_NONE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); Which will reserve … Read more
Anonymous mappings can be pictured as a zeroized virtual file. Anonymous mappings are simply large, zero-filled blocks of memory ready for use. These mappings reside outside of the heap, thus do not contribute to data segment fragmentation. MAP_ANONYMOUS + MAP_PRIVATE: every call creates a distinct mapping children inherit parent’s mappings childrens’ writes on the inherited … Read more
Reads back to what? What is the final destination of this data? Since it sounds like you are completely IO bound, mmap and read should make no difference. The interesting part is in how you get the data to your receiver. Assuming you’re putting this data to a pipe, I recommend you just dump the … Read more
Depends on what exactly you want to use it for. If you want to map existing files into memory, that’s supported with memory-mapped files. They can also be used to share memory between processes (use named mapping object with no underlying file). If you want to map physical memory, that’s generally not supported from user … Read more