Found the answer. It is in this line in the ALTER DEFAULT PRIVILEGES documentation.
You can change default privileges only for objects that will be created by yourself or by roles that you are a member of.
I was using alter default privileges from a different user than the one creating the tables.
Make sure to set the role to the user creating the table before the alter default privilege statement:
SET ROLE <user_that_creates_new_tables>;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO <user_name>;