How to escape underscores in Postgresql

You need to use a backslash to escape the underscore. Change the example query to the following:

SELECT table_name, column_name FROM information_schema.columns
WHERE column_name LIKE '%\_by'