Different device types can have probe() functions. For example, PCI and USB devices both have probe() functions.
If you’re talking about PCI devices, I would recommend you read chapter 12 of Linux Device Drivers, which covers this part of driver initialization. USB is covered in chapter 13.
Shorter answer, assuming PCI: The driver’s init function calls pci_register_driver() which gives the kernel a list of devices it is able to service, along with a pointer to the probe() function. The kernel then calls the driver’s probe() function once for each device.
This probe function starts the per-device initialization: initializing hardware, allocating resources, and registering the device with the kernel as a block or network device or whatever it is.
That makes it easier for device drivers, because they never need to search for devices or worry about finding a device that was hot-plugged. The kernel handles that part and notifies the right driver when it has a device for you to handle.