SSL Certificate not being found.

[misterdwood]

• Ripme version: 1.95
• Java version:
  openjdk version "11.0.17" 2022-10-18
  OpenJDK Runtime Environment (build 11.0.17+8-post-Ubuntu-1ubuntu222.04)
  OpenJDK 64-Bit Server VM (build 11.0.17+8-post-Ubuntu-1ubuntu222.04, mixed mode, sharing)
• Operating system:
  Linux Kubuntu - 4 versions. Raspberry Pis, and Vms.

• Exact URL you were trying to rip when the problem occurred: any gallery on motherless.com ie: https://motherless.com/G170A6B5)
• Please include any additional information about how to reproduce the problem: It worked fine about a month ago, now this is happening.

Expected Behavior

For it to pull actual files and download them

Actual Behavior

It's getting SSL certificate failures. I can navigate the site just fine on a web browser, but unfortunately, something happened where it won't work to download from with Ripme. I should clarify that some stuff downloads, but a folder with 6000 items, 5500 errored with that error and the others downloaded. I have no idea any solution.

avax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:353)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:296)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:291)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1357)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1232)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1175)
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:183)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1416)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:456)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:427)
at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:572)
at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:201)
at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168)
at com.rarchives.ripme.ripper.DownloadFileThread.run(DownloadFileThread.java:136)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439)
at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306)
at java.base/sun.security.validator.Validator.validate(Validator.java:264)
at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222)
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1341)
... 18 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434)
... 24 more
Getting key exception.while.downloading.file in en_US value Exception while downloading file

[occipita]

I can confirm this behaviour. The site itself can be navigated, but images are hosted on a CDN and the CDN uses a different CA (GoGetSSL) which isn't supported by default in OpenJDK (at least up to version 18, which is what I'm using).

Example of a URL that fails: https://cdn5-images.motherlessmedia.com/images/F78038F.jpg

Below is the CA certificate that needs to be supported. AIUI, the best way of doing this is distributing a keystore along with the application that contains all of the appropriate CA certificates. If anyone wants, I can build a system based on the default OpenJDK keystore with this one added and add code to the application to load it during startup. Does that sound useful?

-----BEGIN CERTIFICATE-----
MIIF1zCCA7+gAwIBAgIRAJOLsI5imHtPdfmMtqUEXJYwDQYJKoZIhvcNAQEMBQAw
gYgxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpOZXcgSmVyc2V5MRQwEgYDVQQHEwtK
ZXJzZXkgQ2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMS4wLAYD
VQQDEyVVU0VSVHJ1c3QgUlNBIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTE4
MDkwNjAwMDAwMFoXDTI4MDkwNTIzNTk1OVowTDELMAkGA1UEBhMCTFYxDTALBgNV
BAcTBFJpZ2ExETAPBgNVBAoTCEdvR2V0U1NMMRswGQYDVQQDExJHb0dldFNTTCBS
U0EgRFYgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCfwF4hD6E1
kLglXs1n2fH5vMQukCGyyD4LqLsc3pSzeh8we7njU4TB85BH5YXqcfwiH1Sf78aB
hk1FgXoAZ3EQrF49We8mnTtTPFRnMwEHLJRpY9I/+peKeAZNL0MJG5zM+9gmcSpI
OTI6p7MPela72g0pBQjwcExYLqFFVsnroEPTRRlmfTBTRi9r7rYcXwIct2VUCRmj
jR1GX13op370YjYwgGv/TeYqUWkNiEjWNskFDEfxSc0YfoBwwKdPNfp6t/5+RsFn
lgQKstmFLQbbENsdUEpzWEvZUpDC4qPvRrxEKcF0uLoZhEnxhskwXSTC64BNtc+l
VEk7/g/be8svAgMBAAGjggF1MIIBcTAfBgNVHSMEGDAWgBRTeb9aqitKz1SA4dib
wJ3ysgNmyzAdBgNVHQ4EFgQU+ftQxItnu2dk/oMhpqnOP1WEk5kwDgYDVR0PAQH/
BAQDAgGGMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0lBBYwFAYIKwYBBQUHAwEG
CCsGAQUFBwMCMCIGA1UdIAQbMBkwDQYLKwYBBAGyMQECAkAwCAYGZ4EMAQIBMFAG
A1UdHwRJMEcwRaBDoEGGP2h0dHA6Ly9jcmwudXNlcnRydXN0LmNvbS9VU0VSVHJ1
c3RSU0FDZXJ0aWZpY2F0aW9uQXV0aG9yaXR5LmNybDB2BggrBgEFBQcBAQRqMGgw
PwYIKwYBBQUHMAKGM2h0dHA6Ly9jcnQudXNlcnRydXN0LmNvbS9VU0VSVHJ1c3RS
U0FBZGRUcnVzdENBLmNydDAlBggrBgEFBQcwAYYZaHR0cDovL29jc3AudXNlcnRy
dXN0LmNvbTANBgkqhkiG9w0BAQwFAAOCAgEAXXRDKHiA5DOhNKsztwayc8qtlK4q
Vt2XNdlzXn4RyZIsC9+SBi0Xd4vGDhFx6XX4N/fnxlUjdzNN/BYY1gS1xK66Uy3p
rw9qI8X12J4er9lNNhrsvOcjB8CT8FyvFu94j3Bs427uxcSukhYbERBAIN7MpWKl
VWxT3q8GIqiEYVKa/tfWAvnOMDDSKgRwMUtggr/IE77hekQm20p7e1BuJODf1Q7c
FPt7T74m3chg+qu0xheLI6HsUFuOxc7R5SQlkFvaVY5tmswfWpY+rwhyJW+FWNbT
uNXkxR4v5KOQPWrY100/QN68/j17paKuSXNcsr56snuB/Dx+MACLBdsF35HxPadx
78vkfQ37WcVmKZtHrHJQ/QUyjxdG8fezMsh0f+puUln/O+NlsFtipve8qYa9h/K5
yD0oZN93ChWve78XrV4vCpjO75Nk5B8O9CWQqGTHbhkgvjyb9v/B+sYJqB22/NLl
R4RPvbmqDJGeEI+4u6NJ5YiLIVVsX+dyfFP8zUbSsj6J34RyCYKBbQ4L+r7k8Srs
LY51WUFP292wkFDPSDmV7XsUNTDOZoQcBh2Fycf7xFfxeA+6ERx2d8MpPPND7yS2
1dkf+SY5SdpSbAKtYmbqb9q8cZUDEImNWJFUVHBLDOrnYhGwJudE3OBXRTxNhMDm
IXnjEeWrFvAZQhk=
-----END CERTIFICATE-----

[misterdwood]

I'd like a fix if you have one. I tried using your updated code. I couldn't seem to make it work, possibly my error. I pulled your files, and used maven to build it, then i launched the "ripme-1.7.95-jar-with-dependencies.jar" and it kept erroring saying it could not find the video. I tried on galleries and images and videos and none of them worked and they all said they couldn't find the video. Anyways, I'd be happy to help test anything else.

[misterdwood]

Any Update on this?

[invalidexpression]

The issue can be solved without touching the source code.

First some background:
As @occipita already mentioned, the website and the content are hosted on different domains with different server certificates.
However, when looking at the rest of the certificate chain (intermediate CA and root CA) I couldn't see a difference. Both server certificates are signed by the same intermediate CA (the one, that occipita posted the certificate above; you'll need it to fix the issue).
At least in my Java version the root CA was trusted by default, which you could also see, becuase ripme was successfully able to download the list of links from the website. It started complaining when it came to the content servers.

The actual problem is, that the content servers only send the server certificate. The trust store on the other hand only knows about the root CA certificate. The certificate of the intermediate CA is missing and hence, trust can't be established, because the middle part of the chain is missing.
The website itself sends the server certificate together with the intermediate certificate. That's the usual way. In this case the chain is complete and trust can be established.
Web browsers usually do not complain about such things, because they somehow find and download the intermediate certificate in the background or cache them, whatever...

To make a long story short:
The root cause is that the content server(s), do not send the intermediate CA certificate.

How to fix it?
You need to add the certificate of the intermediate CA (the one that occipita posted above) to your Java trust store (cacerts file). IN this block 'you' refers to the users; not the developers. The root cause is the content server, not the java app!

1. You need to copy the certificate mentioned above (including the -----BEGIN/END CERTIFICATE-----) to a new text file and save it somewhere
2. For importing this file as trusted CA into the Java trust store, the steps and locations vary depending on your OS. But there are a lot of guides on the internet.
   On Debian based linux it should be /lib/ssl/certs/java/cacerts and you may use this guide: https://stackoverflow.com/questions/373295/digital-certificate-how-to-import-cer-file-in-to-truststore-file-using
   On Windows it should be %JAVA_HOME%\jre\lib\security\cacerts and you can use this superb tool: https://keystore-explorer.org/ (that also runs on Linux)

With Keystore Explorer you can execute the steps GUI based:
Open the trust store. You can try menu File -> Open Special -> Open CA Certificates, pay attention if you downloaded the tool with its own JRE; alternatively choose menu File -> Open and navigate to the actual location. If you open it manually the password is: changeit
Then import the certificate file as Trusted CA (menu Tools -> Import Trusted Certificate)
Finally save the trust store file. (In case you don't have write access with KeyStore Explorer, save it somewhere temporarily and replace the original file with admin/root priviliges).

Hope that helps!

[zaham-ijjan]

You need to add the certificate of the intermediate CA (the one that occipita posted above) to your Java trust store (cacerts file).

Not us , the user must add this certifacte to his installed JVM cacerts in his machine wich most of end users are not familiar with or don't know how really it works to begin with .

the best solution to this probleme is changing the releasing strategy from a jar release to a binary file with a well configured embeded JRE for this type of issues

[invalidexpression]

Yes, with 'you' I meant the users; not developers. I have updated my post.
The best solution from my point of view would be, if ML content servers provided the correct CA chain ;-)