In general, your design is correct.
But, if user_id is unique in the table “users”, you don’t need the column “id” in “users”. (A single table containing a unique “id” and a unique “user_id” is pretty unusual.) You also don’t need the column “id” in the table “followers”.
Primary key in “followers” should be (user_id, follower_id), and make sure each of those columns has a foreign key referencing “user_id” in “users”.