Read a small random sample from a big CSV file into a Pandas data frame

Assuming no header in the CSV file: import pandas import random n = 1000000 #number of records in file s = 10000 #desired sample size filename = “data.txt” skip = sorted(random.sample(range(n),n-s)) df = pandas.read_csv(filename, skiprows=skip) would be better if read_csv had a keeprows, or if skiprows took a callback func instead of a list. With … Read more

Read a small random sample from a big CSV file into a Python data frame

Assuming no header in the CSV file: import pandas import random n = 1000000 #number of records in file s = 10000 #desired sample size filename = “data.txt” skip = sorted(random.sample(range(n),n-s)) df = pandas.read_csv(filename, skiprows=skip) would be better if read_csv had a keeprows, or if skiprows took a callback func instead of a list. With … Read more

Using FileSystemWatcher to monitor a directory

The problem was the notify filters. The program was trying to open a file that was still copying. I removed all of the notify filters except for LastWrite. private void watch() { FileSystemWatcher watcher = new FileSystemWatcher(); watcher.Path = path; watcher.NotifyFilter = NotifyFilters.LastWrite; watcher.Filter = “*.*”; watcher.Changed += new FileSystemEventHandler(OnChanged); watcher.EnableRaisingEvents = true; }

Reading a file using a relative path in a Python project

Relative paths are relative to current working directory. If you do not want your path to be relative, it must be absolute. But there is an often used trick to build an absolute path from current script: use its __file__ special attribute: from pathlib import Path path = Path(__file__).parent / “../data/test.csv” with path.open() as f: … Read more

Hata!: SQLSTATE[HY000] [1045] Access denied for user 'divattrend_liink'@'localhost' (using password: YES)