Replies: 39 comments 33 replies
-
|
@Jonathhhan actually called me a while ago and we discussed cleaning up the PR by splitting it into bite size PRs. it takes me quite long to cleanly do this though, because i don't have that much experience and need to double/tripple check every little step and am frequently distracted by work, but i am very happy to be involved here. would you prefer if the PRs directly target master, or would you rather have a testing branch to collect the PRs and later merge them in master together? |
Beta Was this translation helpful? Give feedback.
-
|
Here is a pull request, only with the necessary changes: https://github.com/Jonathhhan/openFrameworks/tree/emscripten_3.1.19 (should also work with current Emscripten). Still struggeling with Emscriptens current audioWorklet implementation (which is not included in this pull request). My problem with the current audioWorklet implementation is a mutex issue, that I have only with ofxOfelia (but it did not happen with the inofficial implementation). I have some additional ideas that are not included in this pull request (they already work, but could be optimized), like webMIDI, audio/video lib improvements or some filesystem stuff (like loading local files into Emscripten). Here is the current audioWorklet discussion (maybe someone has an idea?): emscripten-core/emscripten#16449 |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @Jonathhhan @themancalledjakob - awesome you are working together on this. I think the PR ( #7056 ) could be merged once it is passing the CI tests. I'll ask some questions in the PR. Also happy to look at the audioWorklet stuff once we get the current PR merged in. |
Beta Was this translation helpful? Give feedback.
-
|
I made a list with examples that do not work or have some issues: |
Beta Was this translation helpful? Give feedback.
-
|
Awesome thanks for this list @Jonathhhan ! With the fix the first model works now - will do a PR.
Also the video grabber fixes seem to work great. 🙏👏 |
Beta Was this translation helpful? Give feedback.
-
|
No luck with the thread examples, but since Emscripten supports threads, maybe it is possible... |
Beta Was this translation helpful? Give feedback.
-
uuh, threads would be sexy of course, and it seems as if most major browsers have support for them now https://caniuse.com/sharedarraybuffer
what was your experience with having a single one? Any downsides? is there an oF etherpad somewhere? otherwise I put @Jonathhhan 's list in here, so we can update it as we go: |
Beta Was this translation helpful? Give feedback.
-
@themancalledjakob no, I did not experienced any downsides and in my logic one is enough. But I am not sure. And |
Beta Was this translation helpful? Give feedback.
-
|
In this example I implemented webMIDI and it is possible to load and save MIDI-files: https://midifilemarkovb.handmadeproductions.de/ |
Beta Was this translation helpful? Give feedback.
-
|
Here is a draft for having only one audioContext: https://github.com/Jonathhhan/openFrameworks/tree/one_audioContext |
Beta Was this translation helpful? Give feedback.
-
|
Here is my second fix for ofxEmscriptenVideoPlayer. Need to test it more, before I make a pull request. |
Beta Was this translation helpful? Give feedback.
-
|
@Jonathhhan those changes look pretty good actually. |
Beta Was this translation helpful? Give feedback.
-
|
@ofTheo Yes, I guess so. From the forum: https://forum.openframeworks.cc/t/ofxemscriptenvideoplayer-optimization/39015/5 And because of this (I guess, thats related): (It works for me on Chrome without any problems, but maybe needs some more tests...) |
Beta Was this translation helpful? Give feedback.
-
|
Here is a new (more complete) PR: #7217 (I closed it, because I was unsure...) |
Beta Was this translation helpful? Give feedback.
-
|
Here I found an interesting graphics project, that uses Emscripten, too: https://floooh.github.io/sokol-html5/index.html |
Beta Was this translation helpful? Give feedback.
-
|
Hi @Jonathhhan, |
Beta Was this translation helpful? Give feedback.
-
|
Loving all of the emscripten work! OF on the web is absolutely amazing :) |
Beta Was this translation helpful? Give feedback.
-
|
@ofTheo the thing is, GLES3 (which is specified with |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @Jonathhhan - I'll maybe ask @danoli3 @danzeeeman chime in about GLES3. I was thinking that maybe we could default to it on and then you could aways specify 2 or 1 via the window settings? |
Beta Was this translation helpful? Give feedback.
-
|
@ofTheo that sounds good. Maybe |
Beta Was this translation helpful? Give feedback.
-
|
This is a way to get a list of the connected devices (works with Chrome, Firefox has some issues): But no idea, how to select a device from that list... |
Beta Was this translation helpful? Give feedback.
-
|
enumerateDevices should give you an id that you can use with does this help? https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia relevant snippet from there: getUserMedia({
video: {
deviceId: myPreferredCameraDeviceId
}
}) |
Beta Was this translation helpful? Give feedback.
-
|
@themancalledjakob thanks, I guess that helps :) Yesterday I made a PR that prints the devices in Java Script, just not sure how to pass the result back to OF. #7226 |
Beta Was this translation helpful? Give feedback.
-
|
This PR looks good @Jonathhhan - I think just being able to list the devices is really helpful for now. Or as you mentioned you could pack it all to a single string and then unpack it on the OF end? eg: then in C++/OF I might merge the PR as is, but feel free to add the returning of the devices as another PR if you can figure it out 🙂 |
Beta Was this translation helpful? Give feedback.
-
|
@ofTheo I was wrong, sending a string is actually quite easy, its also already done in the addon:
|
Beta Was this translation helpful? Give feedback.
-
|
With some very small changes it was possible to make ofxImGui work: https://ofximgui.handmadeproductions.de/ |
Beta Was this translation helpful? Give feedback.
-
|
For the svgExample I need the libxml2 lib in the lib folder (which does not seem to be installed by default anymore). |
Beta Was this translation helpful? Give feedback.
-
|
I simplified loading local files (compared to how I did that before). https://github.com/Jonathhhan/ofEmscriptenExamples/tree/main/emscriptenImport_Em_Async_JS_FilePicker The other example uses https://import.handmadeproductions.de/ (that is the old version) Maybe someone can have a look at those methods and maybe optimize them (and do not now Java Script well, and it is possible that I just miss something)? There is still one issue, that is the GUI freezes, if the file browser is discarded. And it unfreezes if the next file is selected (I guess it is, because the methods are not resolved, if no file is selected). The old approach with embind still has some advantages, like not stopping the video while opening the browser, but is much more complicated to implemernt: https://github.com/Jonathhhan/ofEmscriptenExamples/tree/main/emscriptenImport And I had the idea to implement this: https://github.com/GoogleChromeLabs/browser-fs-access Another finding is, that apps without |
Beta Was this translation helpful? Give feedback.
-
|
Hey @ofTheo, The other prs could need some discussion / changes / improvements: I guess thats it from my side. |
Beta Was this translation helpful? Give feedback.
-
|
It seems Emscriptens audioWorklet support will be merged into their main branch soon, I think it could be quite a big improvement to implement them in OF too... emscripten-core/emscripten#16449 |
Beta Was this translation helpful? Give feedback.
-
|
Since yesterday audioWorklets are supported by Emscriptens main branch. |
Beta Was this translation helpful? Give feedback.
-
|
The reason that compute and geometry shaders do not work with Emscripten is (at the moment), that Emscripten uses webGL2 which is based on GL ES 3.0. Compute shaders were introduced in GL ES 3.1 and geometry shaders in GL ES 3.2, so it does not seem very likely that the they will work soon (maybe and hopefully I am wrong). Same for And with vboMeshDrawInstancedExample I get (after enabling |
Beta Was this translation helpful? Give feedback.
-
|
Updated the pad: https://pads.ccc.de/Lu6s668Ruk |
Beta Was this translation helpful? Give feedback.
-
|
@Jonathhhan I saw the multiTextureShaderExample listed in the pad, would you like me to update that one for Emscripten? |
Beta Was this translation helpful? Give feedback.
-
|
@NickHardeman of course, that would be great :) |
Beta Was this translation helpful? Give feedback.
-
|
@Jonathhhan submitted #7323 for the multiTextureShaderExample :) |
Beta Was this translation helpful? Give feedback.
-
|
@NickHardeman thank you. The reason that the video does not start to play is that it needs a user interaction, like audio too. The materialPBR and materialPBRAdvanced do also work, which is great. |
Beta Was this translation helpful? Give feedback.
-
|
Just printed the glInfoExample. Here is the result (maybe helpful for checking what could work). Can also depend on the GPU, of course. And maybe some of the info is not correct, because I edited the patch wrong. There is also a list of the available extensions: |
Beta Was this translation helpful? Give feedback.
-
|
A drawback from multi-threading is that the js 'AnalyserNode' for audio fft does not work anymore. But I think that it is worth it, because fft can be generated in OF too. |
Beta Was this translation helpful? Give feedback.
-
|
@Jonathhhan is there a way to determine if Emscripten is running on mobile? |
Beta Was this translation helpful? Give feedback.
-
|
@NickHardeman I only know how to check if it is a touch device (with setting isTouch=true with the first touch input)... |
Beta Was this translation helpful? Give feedback.
-
|
This seems to work for detecting between OSX safari and iOS safari, haven't tested on anything else. |
Beta Was this translation helpful? Give feedback.
-
|
NIce, looks as if it could work for other OSs too. |
Beta Was this translation helpful? Give feedback.
-
|
Since multithreading is working, it could also be possible to make the udp/osc examples work. At least with the oscReceive example I can receive something (the console print the correct value, but then it aborts with an error). |
Beta Was this translation helpful? Give feedback.
-
|
Hey @ofTheo, A couple of things I noticed:
1) error when not using emrunWhen you're not using emrun to serve the project, an error is thrown: This error is afaik not violent, but it may be confusing or undesirable. An easy fix would be to introduce an environment variable that you can set in your e.g. by default we do emrun. possible implementation: diff --git a/libs/openFrameworksCompiled/project/emscripten/config.emscripten.default.mk b/libs/openFrameworksCompiled/project/emscripten/config.emscripten.default.mk
index cfb193bc5..c914d663e 100644
--- a/libs/openFrameworksCompiled/project/emscripten/config.emscripten.default.mk
+++ b/libs/openFrameworksCompiled/project/emscripten/config.emscripten.default.mk
@@ -93,7 +93,13 @@ ifdef USE_CCACHE
endif
endif
-PLATFORM_LDFLAGS = -Wl --gc-sections --preload-file bin/data@data --emrun --bind --profiling-funcs -s USE_FREETYPE=1 -s ALLOW_MEMORY_GROWTH=1 -s MAX_WEBGL_VERSION=2 -s WEBGL2_BACKWARDS_COMPATIBILITY_EMULATION=1 -s FULL_ES2 -sFULL_ES3=1 -pthread
+
+PLATFORM_LDFLAGS = -Wl --gc-sections --preload-file bin/data@data --bind --profiling-funcs -s USE_FREETYPE=1 -s ALLOW_MEMORY_GROWTH=1 -s MAX_WEBGL_VERSION=2 -s WEBGL2_BACKWARDS_COMPATIBILITY_EMULATION=1 -s FULL_ES2 -sFULL_ES3=1 -pthread
+
+ifndef PROJECT_EMSCRIPTEN_NO_EMRUN
+ PLATFORM_LDFLAGS += --emrun
+endif
+
PLATFORM_LDFLAGS += --js-library $(OF_ADDONS_PATH)/ofxEmscripten/libs/html5video/lib/emscripten/library_html5video.js
PLATFORM_LDFLAGS += --js-library $(OF_ADDONS_PATH)/ofxEmscripten/libs/html5audio/lib/emscripten/library_html5audio.js2) missing explanation how to host (SharedArrayBuffer)We need to set some headers now, it would be helpful to get advice on how to properly host us. 3) copied data does not refresh every compilationI noticed, that I needed to compile a couple of times to see changes. Especially when having copied files from an addon, it takes two or even three compilation cycles to update. #!/bin/bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
PREVIOUS_DIR=$(pwd)
cd $DIR
project=$(basename $DIR)
rm -rf bin/$project*
rm -rf bin/data/ofxMsdfgen
rm -rf bin/data/ofxGPUFont
cp -r ../../../addons/ofxMsdfgen/data/ofxMsdfgen ./bin/data/
cp -r ../../../addons/ofxGPUFont/data/ofxGPUFont ./bin/data/
cd $PREVIOUS_DIRI run this before every compilation. This cleans out all files, that need to be packaged, so that the everything updates with a single compile. 4) gallery of examplesThis is not really related to the next release, but a while ago I played around with the gallery of examples: https://dev.pointer.click/of/ 5) webgl 2.0 & OpenGL ES 3.xI noticed, that opengl abilities are more limited than necessary in the preprocessor macros. For example textures can handle more webgl 2, than oF currently allows. I believe multisampled textures are such a case. I circumvented this, by writing the opengl calls directly, but it was also my first time to really deal with opengl, maybe I just missed something? While we're at this, OpenGL ES only checks for major versions. It's quite simple to add checks for minor versions as well. It's another story to adjust all preprocessor macros though. That may be a lot of work. Also, there are some things like glTexBuffer, which were introduced in ES 3.2 (https://registry.khronos.org/OpenGL-Refpages/es3/html/glTexBuffer.xhtml). I'm not sure, as far as I know it is quite impossible (is that true?) to check for this with a macro, as the version is determined in runtime. I hope these comments are a bit clear, some of these topics are super new to me, and I hope I don't just completely misunderstand something. :) |
Beta Was this translation helpful? Give feedback.
-
|
thanks for this @themancalledjakob!
|
Beta Was this translation helpful? Give feedback.
-
|
@themancalledjakob also thanks from me. If I am right, it is just needed to add |
Beta Was this translation helpful? Give feedback.
-
no, that's not what I mean :) that won't work for other reasons.
cool, I agree this sounds good.
yes. I haven't yet figured out what it is. I made a quick test script to describe the issue. see below
nice! is there already progress somewhere? Can I join the effort?
cool! i'm curious how to check for features or ES version. should that be done in runtime? Appendix for 3. so. To better portray the issue here is a little script to create a test file with timestamp in the data folder and check at various stages where it is. If I would have a better understanding of the Make-process, I probably wouldn't need that... but maybe it helps to show you what is going wrong. the script: #!/bin/bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
printf -v timestamp '%(%Y-%m-%d_%H:%M:%S)T' -1
cd $DIR
project=$(basename $DIR)
# change these to match with your test project
addonsDataTestFile=ofxGPUFont/test
addon=ofxGPUFont
emsdk_sh=/home/jrkb/git/tools/emsdk/emsdk_env.sh
# info
echo "testing $project with timestamp:"
echo ""
echo $timestamp
echo ""
# create / overwrite test file
echo "copyaddonsdata: $timestamp" > ../../../addons/$addon/data/$addonsDataTestFile
# get emsdk ready
source $emsdk_sh
# compile and capture output for later analysis
echo $(emmake make) >> "$0_${timestamp}_emmake_make"
# check that it is properly copied
echo "timestamp in data folder:"
echo ""
cat bin/data/$addonsDataTestFile
echo ""
# make emrun fly on its own, and capture pid
nohup emrun --port 6931 --no_browser bin/$project.html < /dev/null > /dev/null 2>&1 &
pid=$!
echo "pid $pid"
# check the test file as it is served
URL="http://localhost:6931/data/$addonsDataTestFile"
is_up="no"
while [ "$is_up" == "no" ]; do
wget_output=$(wget -q "$URL")
if [ $? -eq 0 ]; then
is_up="yes"
fi
done
wget $URL
# save for later analysis
mv test "$0_${timestamp}_test"
# show it though
echo "timestamp downloaded:"
echo ""
cat ./$0_${timestamp}_test
echo ""
# and this is the part where it goes wrong
# check, how it is packed in the data
echo "timestamp packed:"
echo ""
cat bin/$project.data | grep -a "copyaddonsdata"
echo ""
# pull the trigger on emrun.. sorry
kill -9 $pidhere output (notice the last packed timestamp): 08:35:00 ~/openFrameworks/apps/lalala/thisIsAmazing
$ ./testDataCopy.sh
testing msdf-theatre with timestamp:
2023-05-02_08:35:00
Setting up EMSDK environment (suppress these messages with EMSDK_QUIET=1)
Setting environment variables:
make: make
timestamp in data folder:
copyaddonsdata: 2023-05-02_08:35:00
pid 17440
--2023-05-02 08:35:01-- http://localhost:6931/data/ofxGPUFont/test
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:6931... failed: Connection refused.
Connecting to localhost (localhost)|127.0.0.1|:6931... connected.
HTTP request sent, awaiting response... 200 OK
Length: 36 [application/octet-stream]
Saving to: ‘test.4’
test.4 100%[=====================================================================================================>] 36 --.-KB/s in 0s
2023-05-02 08:35:01 (3.28 MB/s) - ‘test.4’ saved [36/36]
timestamp downloaded:
copyaddonsdata: 2023-05-02_08:35:00
timestamp packed:
copyaddonsdata: 2023-05-02_08:15:25
|
Beta Was this translation helpful? Give feedback.
-
|
@themancalledjakob thank you for this write up. |
Beta Was this translation helpful? Give feedback.
-
|
I tested the current OF nightly with Emscripten 3.1.37 and These are just warnings (which maybe can be avoided, but not sure): |
Beta Was this translation helpful? Give feedback.
-
|
note: the above code seems to work for me in Firefox and Chrome but not Safari.
|
Beta Was this translation helpful? Give feedback.
-
|
@ofTheo thank you. With the current nightly audio input works again. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @Jonathhhan! Maybe I need to check my emscripten version then? The buffer thing for the first screenshot is probably an issue with the buffer size being allocated larger but then the system only using half the buffer size. Def seems like a bug. The Ubuntu screenshot seems even worse 🤦♂️ @dimitre - would you be able to see if you can reproduce it? |
Beta Was this translation helpful? Give feedback.
-
|
@ofTheo yes, without those changes and the same Emscripten version (3.1.37). Even tested with the old nightly again yesterday, and was able to reproduce the issue... maybe I will find the pr that caused the issue (or it is something else, but I did not change anything in both nightlies). |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
We have a bunch of Open GL ES 3 support unmerged as of yet. Might be worth doing so if Emscripten web can benefit massively from it. Assimp has some major updates recently. I've just merged to their master branch some fixes for us to compile latest for oF hopefully that fixes some issues when step up |
Beta Was this translation helpful? Give feedback.
-
|
totally agree! |
Beta Was this translation helpful? Give feedback.
-
Wanted to create this discussion to sync up some of the different emscripten efforts.
@Jonathhhan has been working on a bunch of fixes but I think we need a bit of work to get things integrated cleanly.
@themancalledjakob has also been working on emscripten fixes
There are quite a lot of open emscripten issues:
https://github.com/openframeworks/openFrameworks/issues?q=is%3Aissue+is%3Aopen+Emscripten
I think the short term goal should be to get emscripten on parity with the desktop releases ( ie: fixing obvious bugs and missing features ).
Like:
#6758
#6789
Currently there is a massive PR from @Jonathhhan that is hard to merge.
It would be amazing if we could do bite size PRs for the individual issues with fixes cherry picked if needed.
Once the major issues are fixed, would be happy to continue to think about how we could improve the OF emscripten tools.
@dimitre @Jonathhhan @themancalledjakob is there anyone else we should tag for this discussion?
Thanks so much!
Theo
Beta Was this translation helpful? Give feedback.
All reactions