When a DLL is registered, the DllRegisterServer method entry point in your DLL is invoked. Similarly, DllUnregisterServer is invoked when a DLL is unregistered.
As described in this MSDN article:
Instructs an in-process server to
create its registry entries for all
classes supported in this server
module. If this function fails, the
state of the registry for all its
classes is indeterminate.
For COM DLLs, you will need to implement your own DllRegisterServer and DllUnregisterServer entry point methods which do the registering/unregistering as appropriate. Example code for DllRegisterServer can be found here.
The end result of registering a DLL is that all of the CLSIDs for the components in the DLL are registered under HKEY_CLASSES_ROOT\CLSID. This allows CoCreateInstance to find the correct server when instantiating COM objects from another DLL or application.
DllUnregisterServer will do the reverse, and remove all of the CLSIDs from the registry that were put in there by DllRegisterServer.
More general information about DllRegisterServer can be found here.