It's really the files that can cause issues with upgrades or installs, not folders. The Java will naturally as with any application create folders and directories where needed.
I suppose you have not the SE (simple edition) or end users version of Java and perhaps a developers version? This may explain the differences.
Perhaps your java install was with an online installer, not the download installer?
Using Add and Remove is the usual preferred method. Unless there is a proper uninstaller with the application) or if using some other methods of uninstalling (Windows Installer Clean Up Application or several others).

Oldsod.