[BUG] Mobile background backup does not work with self signed certificate #5562

ktm-91 · 2023-12-08T16:54:59Z

The bug

When I take a picture on my phone and wait 5 seconds, instead of finding that photo uploaded to Immich server, I found 3 "SSL handshake failure" logs on my reverse proxy. Every other feature works as expected, through the reverse proxy, in HTTPS, including the foreground backup when I open the app.
I'm using HAProxy on pfSense with a self-signed certificate.

The OS that Immich Server is running on

Debian 12 x64 + Docker

Version of Immich Server

1.90.1

Version of Immich Mobile App

1.90.0 build.114

Platform with the issue

Server
Web
Mobile

Your docker-compose.yml content

Not relevant (anyway, the default one)

Your .env content

Not relevant (anyway, the default one)

Reproduction steps

1. take a picture with the phone
2. wait 5 seconds without opening Immich app
3. the picture is not getting uploaded automatically in the background, instead the connection attempts of the app results in 3 "SSL handshake failure" logs (3 every picture that the app tries to upload in the background)

Additional information

The feature works correctly if I use the same reverse proxy without SSL Offloading and if I point directly to Immich server.

ktm-91 · 2023-12-18T13:51:47Z

Same issue with latest Android app version 1.91.0

ktm-91 · 2023-12-23T15:13:09Z

Am I the only one with this issue? Still present in the latest version on the app

harshitandro · 2023-12-31T17:38:27Z

Having similar issue with 1.91.4 version android app and server. Haproxy as reverse proxy.

sivel27 · 2024-01-14T16:04:26Z

Same on v1.92.0 android.

wociscz · 2024-01-27T11:52:44Z

Same with 1.93.3 - created another issue before I stumbled upon this (same) issue. Not working also with valid (not self-signed) certificate.

jacob-horton · 2024-01-28T16:26:12Z

Same on v1.93.3 android:

Certificates installed on android
"Allow self-signed SSL certificates" enabled
Traefik reverse proxy for immich server
Default immich docker configuration
Issue doesn't occur if using HTTP instead of HTTPS (still going through reverse proxy)
Issue doesn't occur if using IP/port of immich server directly

ktm-91 · 2024-01-28T16:53:40Z

Yeah the bug is still there, and I also found that with the reverse proxy I cannot play any video of my libraries from the Android app. I find the same certificate errors in HAProxy logs whenever I try to start playing.

xxTBxx · 2024-02-08T02:55:30Z

Seeing this with android 1.94.1 build 121. Foreground backup works fine but background backup throw errors on self signed certificates.

Server is running in K3s with Traefik ingress in K3s, config is straight from Immich Helm charts.

Background Upload Android Steps

With android app minimized photo taken and collected logs after failure notification.

app log level shout
logs pulled in adb shell using logcat --pid=$(pidof -s app.alextran.immich)

Tested with foreground upload setting both enabled and disabled.

Logs

02-07 20:15:27.716 16377 25795 I flutter : Disconnect to Websocket Connection
02-07 20:15:39.618 16377 29665 D BackupWorker: enqueueBackupWorker: BackupWorker enqueued
02-07 20:15:39.740 16377 16377 D BackupWorker: startWork
02-07 20:15:39.744 16377 16453 I WM-Processor: Moving WorkSpec (fa0dd64a-45b3-485b-a9ca-dffe984c6fab) to the foreground
02-07 20:15:39.815 16377 16377 I WM-SystemFgDispatcher: Started foreground service Intent { act=ACTION_START_FOREGROUND cmp=app.alextran.immich/androidx.work.impl.foreground.SystemForegroundService (has extras) }
02-07 20:15:39.815 16377 16377 D FlutterGeolocator: Flutter engine connected. Connected engine count 2
02-07 20:15:40.009 16377  3185 I flutter : Error [getDeviceBackupAsset] ApiException 400: TLS/SSL communication failed: GET /asset/device/5150272af11c21dfc7a0318d53261558bb76391675299b85103d46890f3683ed (Inner exception: HandshakeException: Handshake error in client (OS Error: 
02-07 20:15:40.009 16377  3185 I flutter : 	CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate(handshake.cc:393)))
02-07 20:15:40.009 16377  3185 I flutter : 
02-07 20:15:40.009 16377  3185 I flutter : #0      _SecureFilterImpl._handshake (dart:io-patch/secure_socket_patch.dart:99)
02-07 20:15:40.009 16377  3185 I flutter : #1      _SecureFilterImpl.handshake (dart:io-patch/secure_socket_patch.dart:143)
02-07 20:15:40.009 16377  3185 I flutter : #2      _RawSecureSocket._secureHandshake (dart:io/secure_socket.dart:920)
02-07 20:15:40.009 16377  3185 I flutter : #3      _RawSecureSocket._tryFilter (dart:io/secure_socket.dart:1049)
02-07 20:15:40.010 16377  3185 I flutter : <asynchronous suspension>
02-07 20:15:40.010 16377  3185 I flutter : 
02-07 20:15:40.012 16377 16437 I WM-Processor: Moving WorkSpec (fa0dd64a-45b3-485b-a9ca-dffe984c6fab) to the foreground
02-07 20:15:40.037 16377  3185 I flutter : ERROR backupAsset: HandshakeException: Handshake error in client (OS Error: 
02-07 20:15:40.037 16377  3185 I flutter : 	CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate(handshake.cc:393))
02-07 20:15:40.067 16377  3185 I flutter : ERROR backupAsset: HandshakeException: Handshake error in client (OS Error: 
02-07 20:15:40.067 16377  3185 I flutter : 	CERTIFICATE_VERIFY_FAILED: unable to get local issuer certificate(handshake.cc:393))
02-07 20:15:40.073 16377 16377 D FlutterGeolocator: Flutter engine disconnected. Connected engine count 1
02-07 20:15:40.073 16377 16377 E FlutterGeolocator: Geolocator position updates stopped
02-07 20:15:40.073 16377 16377 E FlutterGeolocator: There is still another flutter engine connected, not stopping location service
02-07 20:15:40.106 16377 16377 D BackupWorker: stopEngine result=Success {mOutputData=Data {}}
02-07 20:15:40.107 16377 16397 I WM-WorkerWrapper: Worker result SUCCESS for Work [ id=fa0dd64a-45b3-485b-a9ca-dffe984c6fab, tags={ app.alextran.immich.BackupWorker } ]
02-07 20:15:40.110 16377 16377 I WM-SystemFgDispatcher: Stopping foreground service

Foreground Upload Logs

Given that the images have not been uploaded at this stage, if the app is opened and the foreground upload setting is enabled the pictures are successfully uploaded. Logs for the successful foreground upload are as follows:

02-07 20:46:46.739 16377 25795 I flutter : [APP STATE] hidden
02-07 20:46:46.739 16377 25795 I flutter : [APP STATE] inactive
02-07 20:46:46.805 16377 25795 I flutter : [APP STATE] resumed
02-07 20:46:46.805 16377 25795 I flutter : Attempting to connect to websocket
02-07 20:46:46.817 16377 16377 D ContentObserverWorker: enabled ContentObserverWorker
02-07 20:46:46.817 16377 16377 D ContentObserverWorker: enabled ContentObserverWorker
02-07 20:46:46.824 16377 25795 I flutter : Start backup process
02-07 20:46:46.843 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:46.865 16377 25795 I flutter : Established Websocket Connection
02-07 20:46:46.889 16377 25795 I flutter : refreshRemoteAssets full took 83ms
02-07 20:46:47.059 16377 25795 I flutter : refreshDeviceAlbums took 170ms
02-07 20:46:47.060 16377 25795 I flutter : newRemote: true, newLocal: true
02-07 20:46:47.081 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.100 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.186 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.203 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.235 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.253 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.269 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.286 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.303 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.319 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.336 16377 25796 E alextran.immich: PIXEL: ioctl err: 1
02-07 20:46:47.653 16377 16513 D ExifInterface: No image meets the size requirements of a thumbnail image.
02-07 20:46:47.869 16377 25795 I flutter : _getBackupAlbumsInfo takes 1044ms
02-07 20:46:55.102 16377 25795 I flutter : [APP STATE] inactive
02-07 20:46:55.242 16377 25795 I flutter : [APP STATE] hidden

ktm-91 · 2024-04-05T07:11:32Z

Do you also have the same issue trying to play videos from Android app through reverse proxy?

Sammy1Am · 2024-04-08T13:07:11Z

I'm having the same issue with version 1.101. Also using Traefik proxy, but using Let's Encrypt signed certificates. Even though my certs aren't self-signed, I still had to check the "ignore self-signed" (or whatever) setting before I could even login.

Sounds SSL related somehow maybe, or maybe just a Traefik thing?

ktm-91 · 2024-04-08T13:40:29Z

I'm having the same issue with version 1.101. Also using Traefik proxy, but using Let's Encrypt signed certificates. Even though my certs aren't self-signed, I still had to check the "ignore self-signed" (or whatever) setting before I could even login.

Sounds SSL related somehow maybe, or maybe just a Traefik thing?

It's not a Traefik issue, I'm using HAProxy with a Let's Encrypt certificate as well. I cannot make neither the background backup feature nor the video playback working on Android

CommanderBubble · 2024-04-22T12:59:06Z

i've got the same issue with 1.102.3, and as people have described, it seems to be a TLS issue

i spun up a new instance on a VM to test it wasn't something else i'd done as well
running a default install straight from the getting started, and then adding traefik as a reverse proxy

ip:port works
http works
https (self-signed) fails in background

the logs don't show anything that looks like an error either logs.log

sidamos · 2024-05-13T04:32:34Z

I have the same issue on iOS and tested a lot of combinations with Apache and Caddy Reverse Proxy, external subdomain, direct connection, valid and invalid certs. The foreground operations always worked fine, issue is only with the background sync. Ignore invalid certs is ON in settings.

Background sync works:

http, direct connection in local network
https, external subdomain, Reverse Proxy (Caddy, but Apache should work too), valid cert from Let's Encrypt

Background sync works not:

https, external subdomain, Reverse Proxy (Caddy or Apache), cert from base domain (= invalid for subdomain)

Basic Auth inside the URL also is not a problem, unless there are special characters in the password like "#". Does not work, even when encoded. Cannot login.

BTW, I cannot always change the "ignore invalid certs" setting. Sometimes it is disabled.

jasonmhite · 2024-05-16T20:36:28Z

Possibly related, I'm also having trouble with the background sync not working on Android. I'm using a certificate signed by my own root certificate authority, which I have also added to the Android root certificate store. It looks like Immich doesn't see this as a valid cert, perhaps related to the recent changes to how Google handles custom installed certificates and apps needing to opt in. Immich won't let me log in without checking the ignore self-signed certs, even though it is most definitely valid. I also get the same effects with background sync not working as other people described, so I think I'm getting treated the same way as a self signed certificate.

ktm-91 added bug Something isn't working needs triage Bug that needs triage from maintainer labels Dec 8, 2023

shenlong-tanwen added the 📱mobile label Dec 19, 2023

This was referenced Jan 27, 2024

Background backup throws error when proxypassed #6686

Closed

Background upload not working for me on IOS 16.5 #6682

Closed

ktm-91 changed the title ~~[BUG] Background Backup feature on Android app with a HTTPS reverse proxy gives SSL handshake failure~~ [BUG] Background Backup feature and Video Playback on Android app with a HTTPS reverse proxy give SSL handshake failure Feb 4, 2024

bo0tzz changed the title ~~[BUG] Background Backup feature and Video Playback on Android app with a HTTPS reverse proxy give SSL handshake failure~~ [BUG] Mobile background backup does not work with self signed certificate Apr 26, 2024

K3das mentioned this issue May 19, 2024

fix(mobile): respect SSL override during background sync #9587

Merged

zackpollard closed this as completed in #9587 May 24, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Mobile background backup does not work with self signed certificate #5562

[BUG] Mobile background backup does not work with self signed certificate #5562

ktm-91 commented Dec 8, 2023 •

edited

ktm-91 commented Dec 18, 2023

ktm-91 commented Dec 23, 2023

harshitandro commented Dec 31, 2023

sivel27 commented Jan 14, 2024

wociscz commented Jan 27, 2024 •

edited

jacob-horton commented Jan 28, 2024

ktm-91 commented Jan 28, 2024

xxTBxx commented Feb 8, 2024

ktm-91 commented Apr 5, 2024

Sammy1Am commented Apr 8, 2024

ktm-91 commented Apr 8, 2024

CommanderBubble commented Apr 22, 2024 •

edited

sidamos commented May 13, 2024 •

edited

jasonmhite commented May 16, 2024

[BUG] Mobile background backup does not work with self signed certificate #5562

[BUG] Mobile background backup does not work with self signed certificate #5562

Comments

ktm-91 commented Dec 8, 2023 • edited

The bug

The OS that Immich Server is running on

Version of Immich Server

Version of Immich Mobile App

Platform with the issue

Your docker-compose.yml content

Your .env content

Reproduction steps

Additional information

ktm-91 commented Dec 18, 2023

ktm-91 commented Dec 23, 2023

harshitandro commented Dec 31, 2023

sivel27 commented Jan 14, 2024

wociscz commented Jan 27, 2024 • edited

jacob-horton commented Jan 28, 2024

ktm-91 commented Jan 28, 2024

xxTBxx commented Feb 8, 2024

Background Upload Android Steps

Logs

Foreground Upload Logs

ktm-91 commented Apr 5, 2024

Sammy1Am commented Apr 8, 2024

ktm-91 commented Apr 8, 2024

CommanderBubble commented Apr 22, 2024 • edited

sidamos commented May 13, 2024 • edited

jasonmhite commented May 16, 2024

ktm-91 commented Dec 8, 2023 •

edited

wociscz commented Jan 27, 2024 •

edited

CommanderBubble commented Apr 22, 2024 •

edited

sidamos commented May 13, 2024 •

edited