Iterate through folders, then subfolders and print filenames with path to text file

Charles’ answer is good, but can be improved upon to increase speed and efficiency. Each item produced by os.walk() (See docs) is a tuple of three items. Those items are: The working directory A list of strings naming any sub-directories present in the working directory A list of files present in the working directory Knowing … Read more

How to get the current running module path/name

This works for me: __loader__.fullname Also if I do python -m b.c from a\ I get ‘b.c’ as expected. Not entirely sure what the __loader__ attribute is so let me know if this is no good. edit: It comes from PEP 302: http://www.python.org/dev/peps/pep-0302/ Interesting snippets from the link: The load_module() method has a few responsibilities … Read more

How to fix symbol lookup error: undefined symbol errors in a cluster environment

After two dozens of comments to understand the situation, it was found that the libhdf5.so.7 was actually a symlink (with several levels of indirection) to a file that was not shared between the queued processes and the interactive processes. This means even though the symlink itself lies on a shared filesystem, the contents of the … Read more

Python 2.6 JSON decoding performance

The new Yajl – Yet Another JSON Library is very fast. yajl serialize: 0.180 deserialize: 0.182 total: 0.362 simplejson serialize: 0.840 deserialize: 0.490 total: 1.331 stdlib json serialize: 2.812 deserialize: 8.725 total: 11.537 You can compare the libraries yourself. Update: UltraJSON is even faster.

How do you switch between python 2 and 3, and vice versa?

Using ‘virtualenv’ you can have different isolated Python environments on a single machine. Also you can switch any-time between the different python interpreter versions. What is virtualenv? A Virtual Environment is an isolated working copy of Python which allows you to work on a specific project without worry of affecting other projects. It enables multiple … Read more

List Comprehension: why is this a syntax error?

Because print is not a function, it’s a statement, and you can’t have them in expressions. This gets more obvious if you use normal Python 2 syntax: my_list=[1,2,3] [print my_item for my_item in my_list] That doesn’t look quite right. 🙂 The parenthesizes around my_item tricks you. This has changed in Python 3, btw, where print … Read more

Recursively convert python object graph to dictionary

An amalgamation of my own attempt and clues derived from Anurag Uniyal and Lennart Regebro’s answers works best for me: def todict(obj, classkey=None): if isinstance(obj, dict): data = {} for (k, v) in obj.items(): data[k] = todict(v, classkey) return data elif hasattr(obj, “_ast”): return todict(obj._ast()) elif hasattr(obj, “__iter__”) and not isinstance(obj, str): return [todict(v, classkey) … Read more

Python readlines() usage and efficient practice for reading

The short version is: The efficient way to use readlines() is to not use it. Ever. I read some doc notes on readlines(), where people has claimed that this readlines() reads whole file content into memory and hence generally consumes more memory compared to readline() or read(). The documentation for readlines() explicitly guarantees that it … Read more

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)