I figured it out using custom dialog theme. windowIsFloating true will get rid of the background but will add some extra space underneath the background as a background. In which case you can use windowBackground @null to erase it.
<style name="CustomDialog" parent="@android:style/Theme.Holo.Light" >
<item name="android:windowBackground">@null</item>
<item name="android:windowIsFloating">true</item>
</style>
Usage:
AlertDialog.Builder builder = new AlertDialog.Builder(getActivity(), R.style.CustomDialog);
Thank you to Raghunandan who gave me the link that includes all style attributes. It took me a while but I went through that file and found very interesting elements. Definitely have a look at the link posted below to explore theme styles.
https://android.googlesource.com/platform/frameworks/base/+/refs/heads/master/core/res/res/values/themes.xml