It is complicated, but I’ll try to explain as best I can. I think I’ll start with Java. My discussion is geared to Java 6, I’m not sure what has changed in Java 7.
Java’ built-in cryptography is available through the Java Cryptography Extension (JCE). This extension has two parts to it, the application API and service provider API. The application API is the part you interact with. You use the getInstance() factory methods of various crypto classes. The service provider aspect is more confusing for the average programmer. They don’t care about how the crypto is implemented, they just want something that works. But under the hood there are the crypto provider classes that do the actual work. If you look at the arguments to getInstance() you’ll see that you can specify the provider if you want. Why would you ever want to? Maybe you have paid $$$ for an optimized commercial implementation of RSA, so you want to use that one. Perhaps one provider has a FIPS certificate or some other certification that you need for your app. Then you would specify that provider. Sun/Oracle ships their Java environment with several providers that together comprise the default provider set for their Java environment. Don’t look at them too carefully because they are overlapping and thus confusing due somewhat to historical artifacts. Basically, when using Oracle Java you ask for some crypto like a KeyPairGenerator through KeyPairGenerator.getInstance("RSA"); you’re going to get an appropriate class instance from one of these providers.
Next, lets look at bouncycastle. The bouncycastle library consists of two parts. One is their unique crypto library whose API you have experimented with in your #2 above. The second part is a lot of glue code to allow this library to be used as crypt provider for the JCE. This means you as a programmer have a choice as to how you use the bouncycastle crypto library. You can use their API directly as in #2 above. Or, you can use the JCE api but explicitly specify the bouncycastle implementation by something like KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA", "BC");.
If you prefer to use the unique bouncycastle API directly (they call it their “lightweight API”) then you have no need for all the glue code used to make it work as a JCE provider. For this bouncycastle does provide a download of just the lightweight API classes.
And now, at last, we look at Android’s implementation. Google didn’t license Oracle’s Java source code, so they didn’t have any of Oracle’s JCE providers. They had to provide their own providers. Since bouncycastle had all the code needed, and was open source and liberally licensed, Google/Android chose to use bouncycastle as the basis for their default JCE provider. But, Android has made no effort to make available the unique lightweight API for Android programmers. They expect you to use these classes solely through the JCE. They have modified the bouncycastle code to tune it for Android. They fact that you can find and maybe use some of the lightweight API directly on Android is simply a side-effect of the fact that it’s there under the hood. And not everything is there. Some have described this situation as “bouncycastle on Android is crippled”.
To actually provide a full featured version of the bouncycastle library on Android some developers produced something called the Spongycastle library. It is nothing more than the bouncycastle library modified so that it can work on Android. The chief modification was to change the package names from org.bouncycastle.* to org.spongycastle.* to prevent namespace conflicts.
So what should you use? That depends on what you want to do, what your portability needs are, what your style preferences are, and what you crypto skill level is. In general, when you are using these libraries you are using crypto at fairly low level. You are concentrating at how to do it (use RSA for key transport, use AES for message encryption, use HMAC-SHA256 for message integrity, etc.) versus what to do (I want to send an encrypted message to a recipient via an email-like mechanism). Obviously, if you can you should stick to higher-level libraries that directly solve your problem. These libraries already understand what PKCS#1 is and how to use it as part of larger and more complete protocols.