WebUI: Optimize i18next stuff #20643

sledgehammer999 · 2024-03-31T02:51:17Z

In the HTML files the i18next works this way:
This

<p>QBT_TR(Some text)QBT_TR[CONTEXT=context]</p>

becomes

<p data-i18n>Some text_context</p>

This

<input type="submit" id="loginButton" value="QBT_TR(Login)QBT_TR[CONTEXT=context]" />

becomes

<input type="submit" id="loginButton" data-i18n="[value]Login_context" />

This

<p customAtrr1="QBT_TR(value1)QBT_TR[CONTEXT=context] customAtrr2="QBT_TR(value2)QBT_TR[CONTEXT=context]>QBT_TR(text)QBT_TR[CONTEXT=context]</p>

becomes

<p data-i18n="[customAtrr1]value1_context;[customAtrr2]value2_context;text_context">this text will be ignored as source text and will be replaced by the appropriate data-i18n value</p>

Python tooling

translations_qt2i18next.py

It is meant to be run when strings are migrated from QBT_TR to i18next and AFTER the i18next-parser has created them in their respective JSON files. The translations_qt2i18next.py script will load the translated strings from the [locale].ts into the [locale].json.
Aka it will migrate already existing translations to lessen the burden of the translators.

split_json.py

It is meant to be run after translations are pulled from Transifex. It will split the strings from the .ts files into their respective .json files

tstool.py

Old behavior: It scanned all the .js/.html/.css files for QBT_TR and updated the .ts files
New behavior: In addition to the old behavior it also merges the keys/context from public/lang/en.json and private/lang/en.json into the .ts files. It accounts for the case where new strings are introduced via the i18next format.

Context vs disambiguation

According to Transifex's page on Qt TS format, Transifex will present the context value as a comment to translators. So I opted to not remap the .ts context/name fields to comment fields.

The QBT_TR method required a context present. It is unclear to me what will happen if in the i18next way a context isn't given. Aka it is unclear to me what the interaction between the python tooling will be with a missing context.

Reverted .ts files from before commit 6964132 because they contain the strings from public part before the drop of QBT_TR Then run tstool.py to merge new strings from the JSON files.

glassez · 2024-03-31T07:12:04Z

Guys, I think you're creating a pain in your ass with these "legacy" contexts. IMO, they need to be adjusted to meet current needs.
Just go back to its origins to understand that it doesn't need to continue to be maintained in the same way as before.
Initially, WebUI used the same translations as GUI, so we had to explicitly specify the appropriate contexts in the QBT_TR function. When we separated own WebUI translations, we could have switched to using implicit contexts, but we did not do this, because apparently no one wanted to do this additional work while it could continue to work in the old way.
But now is a good time to get rid of this legacy method with explicit contexts, because in any case it requires a lot of work.

sledgehammer999 · 2024-03-31T09:02:38Z

But now is a good time to get rid of this legacy method with explicit contexts, because in any case it requires a lot of work.

Sure, but we still want to allow for some kind of comment (optionally). It could be used for disambiguation.
So maybe, should I do the necessary changes to make QBT_TR context optional?

glassez · 2024-03-31T09:41:50Z

A look from a different angle...
Tracking your progress in this job out of the corner of my eye, I asked the following questions:
Use i18next does not look trivial, so what is its undeniable advantage? Wouldn't it be easier/more convenient to write a custom script that would do the same job, but at the same time use existing resources, i.e. load translations from currently maintained .ts files and replace QBT_TR statements?

sledgehammer999 · 2024-03-31T16:04:21Z

@glassez the way we currently use it, it is the simplest one and is like a glorified dictionary data structure.
However, for me, it's advantage is their scanning tool that can extract the source strings from html/js without much fuss. I don't think it is an easy task to replicate ourselves.

glassez · 2024-03-31T16:43:19Z

However, for me, it's advantage is their scanning tool that can extract the source strings from html/js without much fuss. I don't think it is an easy task to replicate ourselves.

We already have a tool that does this (extracts the source strings from html/js and synchronizes them with .ts files). All you need is a JS script that would replace QBT_TR statements with translated strings loaded from .ts files.

sledgehammer999 · 2024-03-31T17:00:38Z

However, for me, it's advantage is their scanning tool that can extract the source strings from html/js without much fuss. I don't think it is an easy task to replicate ourselves.

We already have a tool that does this (extracts the source strings from html/js and synchronizes them with .ts files). All you need is a JS script that would replace QBT_TR statements with translated strings loaded from .ts files.

Maybe. But for efficiency reasons, when trying to figure out what needs translating in an HTML document you probably need some kind of mark/tag like we do with the data-i18n attribute (like the examples in my OP).

I am neutral on this. I can go both ways. I'll wait for @Chocobo1's view too.

Chocobo1 · 2024-04-06T11:54:57Z

All you need is a JS script that would replace QBT_TR statements with translated strings loaded from .ts files.

I don't think it is that simple. With your way, it will require additional code to fetch the html & js files and replace the QBT_TR statements and then tell the browser to parse & run it. This is due to QBT_TR isn't valid HTML and also not valid JS. I imagine the handling code could become quite complex.

I would prefer ditching QBT_TR and ensure the new replacement is valid without requiring any preprocessing.

Chocobo1 · 2024-04-06T11:58:01Z

src/webui/www/public/scripts/login.js

@@ -99,13 +104,13 @@ function submitLoginForm(event) {
            if ((xhr.status === 200) && (xhr.responseText === "Ok."))
                location.replace(location);
            else
-                errorMsgElement.textContent = i18next.t('Invalid Username or Password.');
+                errorMsgElement.textContent = i18next.t('Invalid Username or Password.', { context: 'HttpServer' });


We should ditch the context when it is not necessary, like this one.
If necessary, it can just map it to some generic context such as WebUI in .ts file via the python scripts.

Chocobo1 · 2024-04-06T12:30:40Z

Context vs disambiguation

According to Transifex's page on Qt TS format, Transifex will present the context value as a comment to translators.

I think I need to signify the importance of disambiguation. Disambiguation is quite different to Context. For example, we use the same word Date in two different strings/places. One string has the meaning of a calendar date and the other string has the meaning of a meeting between two people. In some languages other than English, it will be translated into different words. Without providing additional disambiguation field. Transifex will only provide 1 string for translation which is insufficient and incorrect.

So I opted to not remap the .ts context/name fields to comment fields.

As explained in my reply above, we still need the message/comment fields. The context/name is less important and can be just some generic name as mentioned in #20643 (comment).

<p data-i18n>Some text_context</p>

I would really suggest we ditch the context field altogether and only use disambiguation as mentioned before.

Chocobo1 · 2024-04-06T12:50:47Z

becomes

<p data-i18n="[customAtrr1]value1_context;[customAtrr2]value2_context;text_context">this text will be ignored as source text and will be replaced by the appropriate data-i18n value</p>

I somehow don't like this example. It think it is important that the html can still be rendered correctly (with the default language Eng) in case i18next failed to work (or accidentally broken). That means we can't put everything only into data-i18n. I do agree we should strive to cut down duplicates but only after the first point is guaranteed to work.

Assuming we can build our own new string extractor, perhaps we can do:

<p customAtrr1="value1" customAtrr2="value2" data-i18n="customAtrr1;customAtrr2;textContent">text123</p>

The textContent is a special value that maps to element.textContent (which eventually points to text123). When data-i18n attribute is present but empty, it automatically means textContent.
Just an idea...

Excuse me If I misunderstood something, the design is quite open and broad and also needs to consider the ability of the tools, it isn't easy and straightforward.

glassez · 2024-04-06T13:04:48Z

All you need is a JS script that would replace QBT_TR statements with translated strings loaded from .ts files.

I don't think it is that simple. With your way, it will require additional code to fetch the html & js files and replace the QBT_TR statements and then tell the browser to parse & run it. This is due to QBT_TR isn't valid HTML and also not valid JS. I imagine the handling code could become quite complex.

Okay, forget it.
It was just a wild idea. It was caused by the fact that I don't like the way the current HTML/IS code examples look.

glassez · 2024-04-06T13:08:40Z

Assuming we can build our own new string extractor, perhaps we can do:
text123
The textContent is a special value that maps to element.textContent (which eventually points to text123). When data-i18n attribute is present but empty, it automatically means textContent. Just an idea...

👍
This has a separation between the source text and the service stuff, which looks much better to me than the examples from the OP.

glassez · 2024-04-06T13:23:35Z

@Chocobo1
As for the context/disambiguation. I believe the both are important if used appropriately.
I consider context as an implicit disambiguation. It looks difficult (at least inconvenient) to maintain disambiguation across several sources. So we could use contexts to improve this if they were managed implicitly, as is done in Qt, where the Linguist generates a separate context for each class (which usually means for each source file).
Explicit disambiguation is appropriate to be used within the same context (source file).

sledgehammer999 · 2024-04-06T14:06:39Z

becomes
this text will be ignored as source text and will be replaced by the appropriate data-i18n value
I somehow don't like this example. It think it is important that the html can still be rendered correctly (with the default language Eng) in case i18next failed to work (or accidentally broken). That means we can't put everything only into data-i18n. I do agree we should strive to cut down duplicates but only after the first point is guaranteed to work.

Assuming we can build our own new string extractor, perhaps we can do:
text123
The textContent is a special value that maps to element.textContent (which eventually points to text123). When data-i18n attribute is present but empty, it automatically means textContent. Just an idea...

Excuse me If I misunderstood something, the design is quite open and broad and also needs to consider the ability of the tools, it isn't easy and straightforward.

Your example looks good but has the disadvantage of requiring to write our string extractor.
However, I have a possible solution for my example that will address your concerns: On page load and until the i18n is fetched we do a pass on the DOM and "translate" it. The keys we use are the english strings so it will result in a fully functional and readable page. So even if the resource fetching fails, we will still have a readable page. In this case we won't need to write our own string extractor.
How does that sound?

Chocobo1 · 2024-04-06T16:06:13Z

So we could use contexts to improve this if they were managed implicitly, as is done in Qt, where the Linguist generates a separate context for each class (which usually means for each source file).

The webui code is quite unorganized and many of it is just free functions. It would be unclear to say what is the context of it. A simple method is use the source filename as the context. But it require a new string extractor and existing tools won't be enough.

Also I could imagine it might cause string duplication, for example the same string appeared in different context (or different source file, assuming we choose it that way). Is it really necessary to differentiate them?

Your example looks good but has the disadvantage of requiring to write our string extractor.

Perhaps it isn't too hard to write a string extractor for .html: similarly to the replaceI18nText(), find all elements with the data-i18n attribute. Then parse/extract the target attributes and find the associated string/value and put (or merge) the strings into a .json file. It would be quite easy in js code and maybe not too different in python.
As for string extractor for .js, just use i18next-parser. So we'll first invoke i18next-parser for .js and then our custom extractor for .html.
However, I still haven't considered the case of <script> within .html. Maybe i18next-parser has configuration knobs for it.

However, I have a possible solution for my example that will address your concerns: On page load and until the i18n is fetched we do a pass on the DOM and "translate" it. The keys we use are the english strings so it will result in a fully functional and readable page. So even if the resource fetching fails, we will still have a readable page. In this case we won't need to write our own string extractor.
How does that sound?

It doesn't sound bad, but I can only make the decision after seeing the result.

glassez · 2024-04-06T17:32:19Z

Also I could imagine it might cause string duplication, for example the same string appeared in different context.

Such duplication in different contexts is a lesser evil than the requirement to manually take care of disambiguation when adding each new string, IMO.
As I said above, it is not a big difficulty to do this within the same context (i.e. within the same source file in this case), besides, we hardly have any other way to take care of ambiguous strings within the same file, except explicitly specifying disambiguation. But it would be extremely inconvenient to manually take care of disambiguation within all WebUI sources.

sledgehammer999 · 2024-04-06T18:12:12Z

It doesn't sound bad, but I can only make the decision after seeing the result.

I have pushed a new commit implementing this.
However now this:

<p data-i18n>Some text_context</p>

becomes invalid. It should be done like this:

<p data-i18n="Some text_context"></p>

Reason: After the 1st pass the _context is lost. So subsequent lang changes can't find the correct translation due to missing context info.

However, I still haven't considered the case of <script> within .html. Maybe i18next-parser has configuration knobs for it.

I did a quick test. It isn't supported. I don't know if there are config options for it in i18next-parser. But, IMO, it is best that JS and HTML are separate.

sledgehammer999 · 2024-04-06T18:26:06Z

Also I could imagine it might cause string duplication, for example the same string appeared in different context.

Such duplication in different contexts is a lesser evil than the requirement to manually take care of disambiguation when adding each new string, IMO. As I said above, it is not a big difficulty to do this within the same context (i.e. within the same source file in this case), besides, we hardly have any other way to take care of ambiguous strings within the same file, except explicitly specifying disambiguation. But it would be extremely inconvenient to manually take care of disambiguation within all WebUI sources.

IMO, if we currently don't have any duplicate string in the WebUI it is a good indication that disambiguating where needed will be enough and not inconvenient.

Chocobo1 · 2024-04-07T08:29:38Z

However now this:
Some text_context
becomes invalid. It should be done like this:

Reason: After the 1st pass the _context is lost. So subsequent lang changes can't find the correct translation due to missing context info.

It is unfortunate that the more workaround we put in the more workaround is required. The result not only stray from our original visions it also (might) become brittle for maintenance in the long term. Not sure about others, but I wouldn't suggest going this way.
I somehow feel we should really write our own string extractor (replacing i18next-parser) and i18n library (replacing i18next).

A few questions beforehand:

Is the 'client-side translation' a blocking issue for v5.0 release? If not then we can take our time for writing our own i18n tools/library. If yes then we might just settle on the least intrusive changes that just work.
Do we have language requirements for the string extractor? I researched a bit and it seems JS libraries is the most up-to-date. The HTML spec and JS spec is continuously evolving and parsers written in python doesn't seem to catch up. If no blocking concerns I would have done it in nodejs.
The string extractor will scan the .html and .js files and produce a Qt .ts file. At this point I thought it would be easier for us to skip .json altogether. The i18n library will read in .ts file directly and use it for look up.

sledgehammer999 · 2024-04-07T10:35:20Z

It is unfortunate that the more workaround we put in the more workaround is required. The result not only stray from our original visions it also (might) become brittle for maintenance in the long term. Not sure about others, but I wouldn't suggest going this way.
I somehow feel we should really write our own string extractor (replacing i18next-parser) and i18n library (replacing i18next).

The way I see it, no matter what method we invent, it is impossible to use HTMLELement.textContent to hold the unstranslated string/macro AND have it survive the 1st translation pass. What if the user wants to translate again in the same session to another language?

FYI, i18next-parser is just a helpful script. There's also i18next-scanner which is more of a scanning lib. You decide if it is helpful or not.

At the moment, I don't think I have much to offer. If you decide to go the custom extractor/i18n library route, then feel free to close my PRs. They were more for discussions with concrete examples.

Is the 'client-side translation' a blocking issue for v5.0 release? If not then we can take our time for writing our own i18n tools/library. If yes then we might just settle on the least intrusive changes that just work.

I wouldn't want to ship v5.0 with only the public portion transition, especially if we are going to go towards another route. If the new route isn't ready, I suggest reverting the public portion to QBT_TR() again.

Do we have language requirements for the string extractor? I researched a bit and it seems JS libraries is the most up-to-date. The HTML spec and JS spec is continuously evolving and parsers written in python doesn't seem to catch up. If no blocking concerns I would have done it in nodejs.

I suspect that implementing this in JS/node will be vastly more productive and hassle-free. Doesn't nodejs effectively provide an up-to-date way to parse HTML and JS?

The string extractor will scan the .html and .js files and produce a Qt .ts file. At this point I thought it would be easier for us to skip .json altogether. The i18n library will read in .ts file directly and use it for look up.

I am bit skeptical about Qt .ts on the client side during runtime. For me XML is a pain to work with when compared to JSON. But I am not insisting on this. An alternative would be to have a "build step" that would produce the final webui and it would convert the Qt ts to json files. eg using a bundler like Vite. A bundler would have the additional benefit of tree-shaking the JS files (something like LTO for JS).

glassez · 2024-04-07T12:15:28Z

Is the 'client-side translation' a blocking issue for v5.0 release?

As I mentioned earlier, I strongly object to 'client-side translation' being a required feature of v5.0. The only requirement from me is that we must return the possibility of server-side translation of 3rd-party WebUIs until 'client-side translation' is implemented.

Chocobo1 · 2024-04-08T08:46:17Z

The way I see it, no matter what method we invent, it is impossible to use HTMLELement.textContent to hold the unstranslated string/macro AND have it survive the 1st translation pass. What if the user wants to translate again in the same session to another language?

You mean changing to another language on the fly? Maybe it isn't a concern. The language setting resides in Options dialog and closing the dialog will reload the whole WebUI which will in turn re-run the i18n procedure. The reload is already there for a long time.

FYI, i18next-parser is just a helpful script. There's also i18next-scanner which is more of a scanning lib. You decide if it is helpful or not.

Thanks. It indeed looks useful.

I wouldn't want to ship v5.0 with only the public portion transition, especially if we are going to go towards another route. If the new route isn't ready, I suggest reverting the public portion to QBT_TR() again.

As I mentioned earlier, I strongly object to 'client-side translation' being a required feature of v5.0. The only requirement from me is that we must return the possibility of server-side translation of 3rd-party WebUIs until 'client-side translation' is implemented.

OK. In any case if the 'client-side translation' isn't fully ready for v5.0, we'll revert to QBT_TR and enable 'server-side translation for 3rd-party'. This will be done only in v5.0 branch and not in master branch.

I suspect that implementing this in JS/node will be vastly more productive and hassle-free. Doesn't nodejs effectively provide an up-to-date way to parse HTML and JS?

All my google result points to 3rd party libraries.

I am bit skeptical about Qt .ts on the client side during runtime. For me XML is a pain to work with when compared to JSON. But I am not insisting on this.

Reason I proposed ditching JSON is that if we want to have proper support for context, disambiguation fields then the result JSON looks essentially like .ts format. And why bother converting it to .ts when we are going our own way and can use .ts directly?
The browser seems to have native support for parsing XML: https://developer.mozilla.org/en-US/docs/Web/XML/Parsing_and_serializing_XML

An alternative would be to have a "build step" that would produce the final webui and it would convert the Qt ts to json files. eg using a bundler like Vite. A bundler would have the additional benefit of tree-shaking the JS files (something like LTO for JS).

The 'build step' springs into my mind from time to time. Although it would bring many possibilities but I always called it off. I'm concerned that it might affect packaging and development. Wouldn't it become difficult to build from source? Wouldn't it become harder to debug? In the end I decided to keep it as is, it isn't worth the effort IMO.

BTW, I pondered on the context concept and it seems it is impossible to apply it to .js code. For example, file a.js has function x() that is called in b.html and c.html. The translation string in x() has context a (the filename). However when x() is invoked in b.html, it is impossible to get the original filename a.js, it can only get the current scope/file b.html.
I guess this is why in i18next, they provided context in the sense of disambiguation instead of the Qt context meaning.

glassez · 2024-04-08T12:50:46Z

if the 'client-side translation' isn't fully ready for v5.0, we'll revert to QBT_TR and enable 'server-side translation for 3rd-party'. This will be done only in v5.0 branch and not in master branch.

👍

The 'build step' springs into my mind from time to time. Although it would bring many possibilities but I always called it off. I'm concerned that it might affect packaging and development. Wouldn't it become difficult to build from source? Wouldn't it become harder to debug? In the end I decided to keep it as is, it isn't worth the effort IMO.

I agree. I would try to avoid any additional/intermediate steps if it is possible to do it in an acceptable way.

glassez · 2024-04-08T12:58:32Z

BTW, I pondered on the context concept and it seems it is impossible to apply it to .js code. For example, file a.js has function x() that is called in b.html and c.html. The translation string in x() has context a (the filename). However when x() is invoked in b.html, it is impossible to get the original filename a.js, it can only get the current scope/file b.html.
I guess this is why in i18next, they provided context in the sense of disambiguation instead of the Qt context meaning.

OK, got it.
But without mandatory context specified, we will have to take more care of disambiguation. (Note that some words/phrases require disambiguation to be added even if they are used once through all sources.)

sledgehammer999 · 2024-04-08T16:38:23Z

The language setting resides in Options dialog and closing the dialog will reload the whole WebUI which will in turn re-run the i18n procedure

OK. Please consider investigating if we could make it work with i18next in an acceptable way (to avoid reinventing the wheel and tool).
Re-fetching the html to do retranslation seems inefficient to me but oh well...

All my google result points to 3rd party libraries

I hadn't investigated. You're correct. But still, IMO, nodejs is preferable to implement these scripts. The ecosystem seems to me more active than python (at least in the web dev front). Plus, it is far more easier to document and install depedencies and run scripts(a simple npm install takes care of that). In contrast, doing the same in python with pip is awkard because you'll need to know beforehand which dependencies each script needs.

The browser seems to have native support for parsing XML

Sure. It's probably my personal distaste to the format.
PS: Does the verbosity of XML count as a pro for JSON? (in terms of bytes transferred in the network).

The 'build step' springs into my mind from time to time. Although it would bring many possibilities but I always called it off. I'm concerned that it might affect packaging and development. Wouldn't it become difficult to build from source? Wouldn't it become harder to debug? In the end I decided to keep it as is, it isn't worth the effort IMO.

I am going offtopic. This is just for FYI and not a dependency for i18n. Vite has two modes. The development mode will essentially leave the HTML/JS as-is which helps with debug. The production mode performs all kinds of optimizations like minification and tree-shaking. Ideally, the build-step would run inside our build-dist.sh script so the final WebUI code would always be included in source tarballs. And for git development it could be plugged as a command/script in package.json.

I guess this is why in i18next, they provided context in the sense of disambiguation instead of the Qt context meaning

Probably the reason why their namespaces (equivalent for .ts context) seem to require explicit action.

I am afraid I don't have any concrete proposal for the i18n stuff. I'll wait for your proposal.

Chocobo1 · 2024-04-10T09:22:02Z

I wouldn't want to ship v5.0 with only the public portion transition, especially if we are going to go towards another route.

I had a rethink about this. The public part of webui should/must have client-side translation is because I don't want the user preference of locale to be leaked publicly. Everyone able to access your webui will know what language you are speaking/prefer. The private part of webui doesn't have this problem.
Please remind me of your rationale.

Reason I rethink is that even if we have a perfect custom string extractor and i18n translation library, One will still have to migrate all QBT_TR instance by hand (assuming using the format in #20643 (comment)) and this isn't trivial work. The current server-side translation is stable and functional and what will we gain from switching it to 'client-side translation'? Not many I suppose, then why do it? If the concern is related to Transifex and the .py scripts then I would prefer to put work on that.

glassez · 2024-04-10T11:39:10Z

The public part of webui should/must have client-side translation is because I don't want the user preference of locale to be leaked publicly. Everyone able to access your webui will know what language you are speaking/prefer. The private part of webui doesn't have this problem.

Can't this problem be solved on top of the server-side translation? I believe it can. And if we are not going to switch entire WebUI to client side translation, I would strictly insist that we do exactly that.

glassez · 2024-04-10T14:09:03Z

The public part of webui should/must have client-side translation is because I don't want the user preference of locale to be leaked publicly.

So it's not about necessarily using client-side translation, but about using client-side locale (or locale requested by client-side). IMO, it's generally a good idea to do this for the entire WebUI.

Chocobo1 · 2024-04-13T06:12:32Z

Can't this problem be solved on top of the server-side translation? I believe it can. And if we are not going to switch entire WebUI to client side translation, I would strictly insist that we do exactly that.

I have some ideas but I'm not sure they are good. I'll look into it.

glassez · 2024-04-13T11:49:00Z

Can't this problem be solved on top of the server-side translation? I believe it can. And if we are not going to switch entire WebUI to client side translation, I would strictly insist that we do exactly that.

I have some ideas but I'm not sure they are good. I'll look into it.

IIRC doing some quick Internet searches, from the server side it could look like this:

if corresponding GET parameter (e.g. "lang") is set, then try to follow it,
if Accepted-Language HTTP header is set, then try to follow it,
otherwise return the untranslated text.

sledgehammer999 · 2024-04-14T10:13:38Z

I wouldn't want to ship v5.0 with only the public portion transition, especially if we are going to go towards another route.

I had a rethink about this. The public part of webui should/must have client-side translation is because I don't want the user preference of locale to be leaked publicly. Everyone able to access your webui will know what language you are speaking/prefer. The private part of webui doesn't have this problem. Please remind me of your rationale.

Reason I rethink is that even if we have a perfect custom string extractor and i18n translation library, One will still have to migrate all QBT_TR instance by hand (assuming using the format in #20643 (comment)) and this isn't trivial work. The current server-side translation is stable and functional and what will we gain from switching it to 'client-side translation'? Not many I suppose, then why do it? If the concern is related to Transifex and the .py scripts then I would prefer to put work on that.

My reply loosely addresses your comment:

I dislike having 2 entirely different translation systems in the webui. Client-side for public and server-side for private.
I understand that you don't want to expose the user's preferred locale. Although I am not sure I agree that it is a "big deal" to do so.
Ideally, I would prefer our server to be a dumb http server (+ endpoint server). Just serve files as-is and do client-side translation.

One will still have to migrate all QBT_TR instance by hand (assuming using the format in #20643 (comment)) and this isn't trivial work

In case you have missed it, I have done some work in #20610. Each commit in that PR documents the specific regex used. The regex'es can be reused to implement some other scheme if needed.

glassez · 2024-05-03T12:35:22Z

Can't this problem be solved on top of the server-side translation? I believe it can. And if we are not going to switch entire WebUI to client side translation, I would strictly insist that we do exactly that.

I have some ideas but I'm not sure they are good. I'll look into it.

@Chocobo1
Do you have any progress on this?

Chocobo1 · 2024-05-03T12:42:25Z

Do you have any progress on this?

I've done some client-side experiments but the result isn't good, I have more ideas but will still need time.

Also stumbled upon this while researching: https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation#the_accept-language_header

sledgehammer999 added 2 commits March 30, 2024 21:26

WebUI: Optimize i18n string marking

bb8e3aa

WebUI: Reintroduce the old i18n context

c34dc36

sledgehammer999 added WebUI WebUI-related issues/changes Translations Related to i18n or Transifex labels Mar 31, 2024

sledgehammer999 added this to the 5.0 milestone Mar 31, 2024

sledgehammer999 mentioned this pull request Mar 31, 2024

[WIP] Convert private WebUI strings to i18next #20610

Draft

sledgehammer999 added 2 commits March 31, 2024 05:57

WebUI: Refactor the python i18n tooling

dc49976

WebUI: Update translations

a34b740

Reverted .ts files from before commit 6964132 because they contain the strings from public part before the drop of QBT_TR Then run tstool.py to merge new strings from the JSON files.

sledgehammer999 force-pushed the webui_optimize_public branch from df84d9b to a34b740 Compare March 31, 2024 02:57

Chocobo1 reviewed Apr 6, 2024

View reviewed changes

Pre-render HTML with default language

f4880ae

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WebUI: Optimize i18next stuff #20643

WebUI: Optimize i18next stuff #20643

sledgehammer999 commented Mar 31, 2024 •

edited

glassez commented Mar 31, 2024

sledgehammer999 commented Mar 31, 2024 •

edited

glassez commented Mar 31, 2024

sledgehammer999 commented Mar 31, 2024 •

edited

glassez commented Mar 31, 2024

sledgehammer999 commented Mar 31, 2024 •

edited

Chocobo1 commented Apr 6, 2024 •

edited

Chocobo1 Apr 6, 2024

Chocobo1 commented Apr 6, 2024

Context vs disambiguation

Chocobo1 commented Apr 6, 2024 •

edited

glassez commented Apr 6, 2024

glassez commented Apr 6, 2024

glassez commented Apr 6, 2024

sledgehammer999 commented Apr 6, 2024

Chocobo1 commented Apr 6, 2024 •

edited

glassez commented Apr 6, 2024

sledgehammer999 commented Apr 6, 2024

sledgehammer999 commented Apr 6, 2024

Chocobo1 commented Apr 7, 2024

sledgehammer999 commented Apr 7, 2024

glassez commented Apr 7, 2024

Chocobo1 commented Apr 8, 2024

glassez commented Apr 8, 2024

glassez commented Apr 8, 2024

sledgehammer999 commented Apr 8, 2024

Chocobo1 commented Apr 10, 2024

glassez commented Apr 10, 2024

glassez commented Apr 10, 2024

Chocobo1 commented Apr 13, 2024

glassez commented Apr 13, 2024

sledgehammer999 commented Apr 14, 2024

glassez commented May 3, 2024

Chocobo1 commented May 3, 2024

WebUI: Optimize i18next stuff #20643

Are you sure you want to change the base?

WebUI: Optimize i18next stuff #20643

Conversation

sledgehammer999 commented Mar 31, 2024 • edited

Python tooling

translations_qt2i18next.py

split_json.py

tstool.py

Context vs disambiguation

glassez commented Mar 31, 2024

sledgehammer999 commented Mar 31, 2024 • edited

glassez commented Mar 31, 2024

sledgehammer999 commented Mar 31, 2024 • edited

glassez commented Mar 31, 2024

sledgehammer999 commented Mar 31, 2024 • edited

Chocobo1 commented Apr 6, 2024 • edited

Chocobo1 Apr 6, 2024

Choose a reason for hiding this comment

Chocobo1 commented Apr 6, 2024

Context vs disambiguation

Chocobo1 commented Apr 6, 2024 • edited

glassez commented Apr 6, 2024

glassez commented Apr 6, 2024

glassez commented Apr 6, 2024

sledgehammer999 commented Apr 6, 2024

Chocobo1 commented Apr 6, 2024 • edited

glassez commented Apr 6, 2024

sledgehammer999 commented Apr 6, 2024

sledgehammer999 commented Apr 6, 2024

Chocobo1 commented Apr 7, 2024

sledgehammer999 commented Apr 7, 2024

glassez commented Apr 7, 2024

Chocobo1 commented Apr 8, 2024

glassez commented Apr 8, 2024

glassez commented Apr 8, 2024

sledgehammer999 commented Apr 8, 2024

Chocobo1 commented Apr 10, 2024

glassez commented Apr 10, 2024

glassez commented Apr 10, 2024

Chocobo1 commented Apr 13, 2024

glassez commented Apr 13, 2024

sledgehammer999 commented Apr 14, 2024

glassez commented May 3, 2024

Chocobo1 commented May 3, 2024

sledgehammer999 commented Mar 31, 2024 •

edited

sledgehammer999 commented Mar 31, 2024 •

edited

sledgehammer999 commented Mar 31, 2024 •

edited

sledgehammer999 commented Mar 31, 2024 •

edited

Chocobo1 commented Apr 6, 2024 •

edited

Chocobo1 commented Apr 6, 2024 •

edited

Chocobo1 commented Apr 6, 2024 •

edited