fetchall – Tarik Billa

Why is the same SQLite query being 30 times slower when fetching only twice as many results?

December 28, 2023 by Tarik

The execution time geometrically proportional to the number of rows in each table rather than arithmetically e.g. 3 tables with 10 rows each => 1,000 comparision 3 tables with 10, 10 and 40 rows => 4,000 comparisons 3 tables with 20 rows each => 8,000 comparisons You could probably re-factor the query to avoid some … Read more

python postgres can I fetchall() 1 million rows?

September 15, 2023 by Tarik

The solution Burhan pointed out reduces the memory usage for large datasets by only fetching single rows: row = cursor.fetchone() However, I noticed a significant slowdown in fetching rows one-by-one. I access an external database over an internet connection, that might be a reason for it. Having a server side cursor and fetching bunches of … Read more