How do I profile a Python script?

Python includes a profiler called cProfile. It not only gives the total running time, but also times each function separately, and tells you how many times each function was called, making it easy to determine where you should make optimizations. You can call it from within your code, or from the interpreter, like this: import … Read more

How do I import a module given the full path?

For Python 3.5+ use (docs): import importlib.util import sys spec = importlib.util.spec_from_file_location(“module.name”, “/path/to/file.py”) foo = importlib.util.module_from_spec(spec) sys.modules[“module.name”] = foo spec.loader.exec_module(foo) foo.MyClass() For Python 3.3 and 3.4 use: from importlib.machinery import SourceFileLoader foo = SourceFileLoader(“module.name”, “/path/to/file.py”).load_module() foo.MyClass() (Although this has been deprecated in Python 3.4.) For Python 2 use: import imp foo = imp.load_source(‘module.name’, ‘/path/to/file.py’) foo.MyClass() … Read more

How do I get the row count of a Pandas DataFrame?

For a dataframe df, one can use any of the following: len(df.index) df.shape[0] df[df.columns[0]].count() (== number of non-NaN values in first column) Code to reproduce the plot: import numpy as np import pandas as pd import perfplot perfplot.save( “out.png”, setup=lambda n: pd.DataFrame(np.arange(n * 3).reshape(n, 3)), n_range=[2**k for k in range(25)], kernels=[ lambda df: len(df.index), lambda … Read more

fatal error: Python.h: No such file or directory

Looks like you haven’t properly installed the header files and static libraries for python dev. Use your package manager to install them system-wide. For apt (Ubuntu, Debian…): sudo apt-get install python-dev # for python2.x installs sudo apt-get install python3-dev # for python3.x installs For yum (CentOS, RHEL…): sudo yum install python-devel # for python2.x installs … Read more

Random string generation with upper case letters and digits

Answer in one line: ”.join(random.choice(string.ascii_uppercase + string.digits) for _ in range(N)) or even shorter starting with Python 3.6 using random.choices(): ”.join(random.choices(string.ascii_uppercase + string.digits, k=N)) A cryptographically more secure version: see this post ”.join(random.SystemRandom().choice(string.ascii_uppercase + string.digits) for _ in range(N)) In details, with a clean function for further reuse: >>> import string >>> import random >>> … Read more

Proper way to declare custom exceptions in modern Python?

Maybe I missed the question, but why not: class MyException(Exception): pass To override something (or pass extra args), do this: class ValidationError(Exception): def __init__(self, message, errors): # Call the base class constructor with the parameters it needs super().__init__(message) # Now for your custom code… self.errors = errors That way you could pass dict of error … Read more

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)