You need to publish the app before it becomes available for testing.
if you publish the app and the apk is only in “alpha testing” section
then it is NOT available to general public, only for activated testers in the alpha section.
EDIT:
One additional note: “normal” users will not find your app on Google Play, but also the activated tester can not find the application by using the search box.
Only the direct link to the application package will work. (only for the activated testers).