Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to view object in AR mode #4582

Open
ikhizarwork opened this issue Nov 30, 2023 · 30 comments
Open

Unable to view object in AR mode #4582

ikhizarwork opened this issue Nov 30, 2023 · 30 comments
Assignees
Labels
flag: not model-viewer Problems that are related to our project but are ultimately some other project's issue

Comments

@ikhizarwork
Copy link

I am using the google modelviewer to view the 3D and then further use the AR functionality to view the object in AR form , my 3D is working perfectly fine , i have multiple 3D gltf files which i am succesfully able to view both in 3D and in AR form , but some of the files are not getting viewed in AR form , it is not showing anything when i am clicking on small cubicle form which opens up the object in AR form , there are some gltf files which are around 22MB size , which is not getting rendered in AR , and some files are around 10MB but still not getting opened up in AR while it is perfectly getting viewed in 3D form .

please help with the following issue .

@palashk92
Copy link

Just want to add that we're experiencing a similar issue with AR using scene-viewer. The GLB that previously were working no longer load. We get a 'Couldn't load object' error. The file itself loads correctly in all GLTF viewers, just not in AR

@elalish
Copy link
Collaborator

elalish commented Dec 4, 2023

@ikhizarwork Can you give us the details requested in our issue template you deleted? We can only help if we know what device/browser/AR-mode and such you're using.

@palashk92 Can you link a GLB that used to work and no longer loads in scene-viewer? We can pass that bug along.

@palashk92
Copy link

Here is the file that's failing (I am unable to attach the GLB file so here's a link to download it)

https://admin-fts.threekit.com/api/files/f9fa0d33-b75c-48dd-8b19-0c326e3ba0cd/content

@elalish
Copy link
Collaborator

elalish commented Dec 8, 2023

@eugenecys Can you take a look at this Scene Viewer regression (@palashk92's comments)?

@elalish elalish added the flag: not model-viewer Problems that are related to our project but are ultimately some other project's issue label Dec 8, 2023
@ikhizarwork
Copy link
Author

ikhizarwork commented Dec 8, 2023

@elalish we are experiencing a similar issue as @palashk92 has mentioned, earlier our gltf and glb files were running properly in modelviewer and as well as in AR mode , but currently the same old gltf and glb files are not opening in AR mode in both the devices i.e andriod and IOS .

Please note that the problem is with the AR mode only , Model is getting viewed properly in Model-viewer component ,
the issue is with AR mode , when a user wants to see the model in AR mode , then that thing is failing. (The model is not rendering in AR mode)

@palashk92
Copy link

palashk92 commented Dec 8, 2023

Here's another example of a file that does not work (Please download and extract it from the ZIP file). No errors or warnings were reported by the glTF viewer(s). When I tried to debug it using adb I found these logs
another_attempt.zip

[ 12-08 11:15:24.926  1329: 9928 W/ActivityManager ]
Scheduling restart of crashed service com.google.android.googlequicksearchbox/com.google.ar.imp.view.ipc.LoaderService in 1000ms for connection


[ 12-08 09:56:32.619 27372:27372 E/ImpView  ]
Future failed with error INTERNAL: Pipe closed


[ 12-08 09:56:32.619 27372:27372 E/ImpView  ]
Failed to Load Gltf url '<redacted>' reason: Pipe closed

@elalish
Copy link
Collaborator

elalish commented Dec 8, 2023

@milesgreen @hybridherbst Since you are generally the most up to date on testing AR across platforms. Have you seen these problems? Any insights?

@hybridherbst
Copy link
Contributor

hybridherbst commented Dec 9, 2023

Hm, one thing that I can note is that this warning:
image
does tend to trip up some viewers (some will refuse to load the file).
Seems that one of the normals (image 13) contains sRGB IEC61966-2.1 as color profile (should not have a color profile).

Additionally, the model contains 21 textures (all 1-2k, some of which redundant) – should be fine for most viewers but I've seen older devices crash with that.

To be honest I'm not usually testing against scene-viewer anymore (just when someone runs into a specific problem), almost everyone uses WebXR by now for the additional interactive capabilities.

@palashk92
Copy link

Thank you. I'll look into fixing that warning. However the other file I provided has no warnings or errors, but simply fails to load

@milesgreen
Copy link
Contributor

@palashk92

One thing I noticed is that for both models, when entering AR, Three.js is throwing a warning to the console that USDZExporter does not support negative scales:

Screenshot 2023-12-12 at 22 14 32

However, I don't think this is necessarily causing your issue, as even with those warnings, the models are still working.

I have tested both models in the following configurations:

  • iOS 17.3 beta, ModelViewer 3.3.0 using auto USDZ generation, viewed in Safari and Chrome:
    ✅ Both models loaded into MV and were able to be viewed in ARQuickLook.

  • iOS 16.6.1, ModelViewer 3.3.0 using auto USDZ generation, viewed in Safari and Chrome
    ✅ Both models loaded into MV and were able to be viewed in ARQuickLook.

  • iOS 15.8, ModelViewer 3.3.0 using auto USDZ generation, viewed in Safari and Chrome
    ✅ another_attempt.glb - loaded into MV and was able to be viewed in ARQuickLook. Took a long time to open, but was tested on an old iPhone 7 device.
    ❌ failing-ar-file.glb - loaded into MV but would not open in ARQuickLook. It looks like the device just runs out of memory while generating the USDZ and never gets to the point of attempting to open ARQL. But this was just a little old iPhone7.

@palashk92 - are you able to confirm which version of MV you're using and which version of iOS you're seeing an issue with?

@milesgreen
Copy link
Contributor

@ikhizarwork

  • do you have any example models or links you can share?

  • which version of ModelViewer are you using and with which version of iOS are you seeing these issues?

@palashk92
Copy link

palashk92 commented Dec 12, 2023

Our issue is only on Android. We're using scene-viewer 1.0. Apologies for not clarifying that in my initial messages. This is how we're generating the intent URL(the actual URLs are redacted):

We tested it on an OnePlus 6 and a Pixel 8 Pro.

intent://arvr.google.com/scene-viewer/1.0?$<redacted url>#Intent;scheme=https;package=com.google.ar.core;action=android.intent.action.VIEW;S.browser_fallback_url=<redacted>;end;

@palashk92
Copy link

Hi, I was wondering if there was any update on this?

@hybridherbst
Copy link
Contributor

@palashk92 you wrote

Thank you. I'll look into fixing that warning.

Did you? Also, did you test with less textures as described above?

@palashk92
Copy link

palashk92 commented Dec 20, 2023

I have fixed that warning. As for textures, I can't change that particular file/model due to certain constraints, but I tried with a different model that has less textures (11) and all the textures are now 512x512. It still fails. Attached is that file

attempt_2.zip

@palashk92
Copy link

Hello, Happy New Year to you guys!

I was wondering if there was any update based on my updated findings?

@palashk92
Copy link

Hi, can we have an update on this? It impacts our production environment. If there's anything else we can try to workaround/mitigate the issue please let us know

@elalish
Copy link
Collaborator

elalish commented Jan 19, 2024

@AdrianAtGoogle Can you repro this Scene Viewer bug?

@Azeem00786
Copy link

Azeem00786 commented Feb 6, 2024

@elalish
I am also encountering an issue with some GLB files. Specifically, they are not rendering in AR mode on iOS devices, although they work fine in AR mode on Android devices.
video-link-https://github.com/google/model-viewer/assets/91094418/38ab5d90-4e6a-4b1b-82bb-9253640cb944
this is my code-
<model-viewer src="fried_checken.glb" ar ar-modes="webxr scene-viewer quick-look" camera-controls tone-mapping="commerce" poster="poster.webp" shadow-intensity="1" > </model-viewer>
This is my GLB file link
1-https://drive.google.com/file/d/1CWMp-OR_V-TRHksFc-6kgd79fCC4Ab5n/view?usp=sharing
2-https://drive.google.com/file/d/1BfxTIUR2lDfS_xjF1ycxDPXNJLz3NM5B/view?usp=sharing

Device Details
1- iPhone 11
iOS version 16.6.1
2- iPhone 15
iOS version 17.3

Kindly help with this.

@elalish
Copy link
Collaborator

elalish commented Feb 12, 2024

@Azeem00786 I think your iOS issue is unrelated to the Scene Viewer (Android) issue here. That looks like a Quick Look bug (@grorg) - did it actually crash, or did you close it? I've noticed QuickLook getting slower and slower at loading models - my impression is it's not getting a lot of work these days. You can try filing a bug on WebKit - sometimes we get some traction there for QuickLook issues.

@Azeem00786
Copy link

@elalish Thanks for your reply. I closed the AR view manually. I will add this bug to WebKit because there are a lot of 3D GLB models that are not opening inside iOS.

@clochardM33
Copy link
Contributor

clochardM33 commented Apr 28, 2024

@elalish we are experiencing a similar issue as @palashk92 has mentioned, earlier our gltf and glb files were running properly in modelviewer and as well as in AR mode , but currently the same old gltf and glb files are not opening in AR mode in both the devices i.e andriod and IOS .

Please note that the problem is with the AR mode only , Model is getting viewed properly in Model-viewer component , the issue is with AR mode , when a user wants to see the model in AR mode , then that thing is failing. (The model is not rendering in AR mode)

Snap. Any glb I make throws the same Couldn't load object error when it moves over into Android AR. Your example pages work just fine.

Oh this is getting silly, now it works; for some models.
I'm checking this in Edge on Android, as their camera in search bar is nicest.
Here's one that nver works (in AR) https://blacreative.co.uk/demo/receptiondesk.php
And here's one (that's a much smaller model) that sometimes works https://blacreative.co.uk/demo/bd%20bathroom%20cabinets/

@elalish
Copy link
Collaborator

elalish commented Apr 29, 2024

@clochardM33 Is this with WebXR mode or Scene Viewer? WebXR should work the same as 3D mode for any GLB, since it's using the same rendering code.

@clochardM33
Copy link
Contributor

Scene Viewer.
Have done a little more investigating. The "failure" is browser and model specific.
I have 3 pages and looked at them in Chrome, Edge and Brave on Android
Page...Chrome...Edge...Brave
1 - OK - OK - OK
2 - OK - Not OK - OK
3 - Not OK - Not OK - Not OK

All 3 pages are configured with "ar-modes = scene-viewer webxr quick-look"
The models are reasonably different sizes in triangles, but very different in texture sizes (1 smallest, 3 largest)
And as an aside 2 and 3 crash Safari on an iPhone 8 when the page loads even before you get to AR.
Is this starting to smell of an out of resources situation?

1 - https://blacreative.co.uk/demo/bd%20bathroom%20cabinets/
2 - https://blacreative.co.uk/demo/hightable.php
3 - https://blacreative.co.uk/demo/receptiondesk.php

@elalish
Copy link
Collaborator

elalish commented Apr 29, 2024

Indeed it is. You might look at KTX2 textures to reduce your GPU RAM footprint.

@clochardM33
Copy link
Contributor

Indeed it is. You might look at KTX2 textures to reduce your GPU RAM footprint.

On it. Thanks.
I was wondering what I was going to get bamboozled by this week!

@ConfidantCommunications

I can confirm the browser being a problem. I have an iPhone 8 and the auto-generated usdz fails in Firefox but works in Safari.

@clochardM33
Copy link
Contributor

Indeed it is. You might look at KTX2 textures to reduce your GPU RAM footprint.

Of course @elalish is on the money yet again.
There is a really nice introduction read here and a link to khronos' compressor below that.
But a word of warning about the compressor. If you are a lazy modeller like me and you don't start with nice square textures you will come across the compressor's usability snags all too quickly. It's fine unless it doesn't like something. Then it's back to square one with you as it will latch or something similar and it's quicker to drag the model in and start from scratch.

Following this up did reveal another gottcha for me, but that is for a different Issue.

https://www.khronos.org/assets/uploads/developers/presentations/3D_Formats_Wayfair_KTX2_SIGGRAPH_Aug21.pdf

https://github.khronos.org/glTF-Compressor-Release/

@elalish
Copy link
Collaborator

elalish commented May 29, 2024

@ConfidantCommunications Are you saying AR works in Firefox on iOS now? I wasn't aware of that. Any errors or information regarding the failure? Have you filed an issue on Firefox?

@ConfidantCommunications

@elalish Yes Firefox iOS works if given a pre-generated USDZ. I've not pursued this further yet. I'm currently consumed with upgrading a Three.js project to use the AR capability of model-viewer. Once I actually have something that works in Safari I will look at getting Firefox working with auto-USDZ models.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flag: not model-viewer Problems that are related to our project but are ultimately some other project's issue
Projects
None yet
Development

No branches or pull requests

9 participants