Well I found the solution. It can be done one by passing a closure function in with() as second index of array like
Post::query()
->with(['user' => function ($query) {
$query->select('id', 'username');
}])
->get()
It will only select id and username from other table. I hope this will help others.
Remember that the primary key (id in this case) needs to be the first param in the
$query->select() to actually retrieve the necessary results.*