Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatic Category Assignment #5779

Open
deskicio opened this issue Sep 27, 2016 · 29 comments · May be fixed by #20502
Open

Automatic Category Assignment #5779

deskicio opened this issue Sep 27, 2016 · 29 comments · May be fixed by #20502

Comments

@deskicio
Copy link

deskicio commented Sep 27, 2016

I've been searching for quite a bit and i couldnt find anything.

I would love a system that automatically picks the category according to key words associated with it.

Good'ol emule used to have this function:
Automatic Category Assignment
When starting new downloads, they can be assigned automatically to this category, if the filename matches a given pattern. | is used to separate different keywords and * can be used as a wildcard for an arbitrary string.
Example: .avi|a.*
This would assign new files with the extension 'avi', and filenames starting with an 'a' to this category.
(http://www.emule-project.net/home/perl/help.cgi?l=1&rm=show_topic&topic_id=143)

I believe it would be a great addtion to qbit's current features.

Thanks for a great product.

@Misiek304
Copy link

Users ask for this before, also with setting different file path to each of the categories but it's not like a priority. I don't even think they are considering implementing this in a near future.

@tgregerson
Copy link
Contributor

I think this request is also covered in #5201.

@CypherConjured
Copy link

What is the point of automatic torrent management if you have to manually assign the category? All I want is to automatically separate my .mp4 files into my video directory.

@leo-carmo
Copy link

What is the point of automatic torrent management if you have to manually assign the category? All I want is to automatically separate my .mp4 files into my video directory.

I second that. A way to auto-assign the categories based on torrentname by regex or by tracker would be handy!

@sig-kill
Copy link

For an example of this, you could take a look at the LabelPlus plugin for deluge, which has all of this functionality and makes it painless to auto-categorize torrents based on torrent/tracker name. Something like this would be immensely helpful!

@FranciscoPombal FranciscoPombal changed the title [Feature Request] Automatic Category Assignment Automatic Category Assignment May 28, 2020
@anewuser
Copy link

Here's a GUI example from another torrent client (Tixati):

tixati

All new torrents will automatically be added to the temp category, unless they contain trackers with either the fedora or ubuntu strings.

@guiltlab

This comment has been minimized.

@raspopov
Copy link

raspopov commented Mar 8, 2022

Wow this issue opened for 6 years already... 🤔

For example the old Shareaza has a "Download groups" with a similar functionality:

Download groups

The "Filters" field matches either any part of download file name or also matches any available metadata including for example tracker URL substring, by default it contains a file extensions according current schema ("File type").

"BitTorrent downloads" option effectively matches any file downloaded using torrent since Shareaza is a multi-protocol client, in qB case it can be for example "DHT-only" checkbox.

@raspopov
Copy link

raspopov commented Mar 8, 2022

Another rich example is a JDownloader 2 "Rule" dialog:

JDownloader 2

The right column of buttons toggles regular expression matches.
The bottom input box is for testing purposes.

@Serene-Arc
Copy link

It's a little amazing that there's a widely used torrent client without this feature. I'd live to switch away from Deluge since it's so incredibly buggy but this really is a deal breaker. It's so much more difficult to categorise hundreds or thousands of torrents manually.

@glassez
Copy link
Member

glassez commented Sep 7, 2022

Good'ol emule used to have this function:
Automatic Category Assignment
When starting new downloads, they can be assigned automatically to this category, if the filename matches a given pattern

Doesn't emule download single file per task?
Note that torrents are often multifile so what file exactly is to apply the pattern?

@Serene-Arc
Copy link

@glassez Deluge's labelplus applies the regex to all files, and if any of them match then the label applies.

@glassez
Copy link
Member

glassez commented Sep 7, 2022

@glassez Deluge's labelplus applies the regex to all files, and if any of them match then the label applies.

Then it is possible that different files will match the different categories so only first match will win. (Although it is also possible that single-file torrent could match several categories.)

@Serene-Arc
Copy link

Yes, Deluge orders the categories from top to bottom, with the first match dictating the label. Not saying it's the best solution, but it is a solution. I had to add a negative regex match to make it easier but it does work if you only use file matches as the last level of possible resort.

@KingPsychopath
Copy link

Hilarious that this is still open, does qBittorent have a plugins interface?

@luzpaz
Copy link
Contributor

luzpaz commented Feb 14, 2024

Any volunteers to draw up a proposal ?

@leo-carmo
Copy link

God, this is 8 years old! Almost all other torrent clients and download managers have a similar feature, but the good qBittorrent doesn't?

@tgregerson
Copy link
Contributor

I spent some timing scoping out the backend portion of the implementation.

When making rules based off the torrent descriptor or when using a torrent file with built-in metadata, it is reasonably straightforward. Most of the complexity comes when torrent metadata needs to be downloaded, since this happens asynchronously. Seems doable though.

@tgregerson
Copy link
Contributor

I have a basic prototype working.

  • A configurable list of Rules are applied when adding a new torrent
  • A Rule consists of a Condition and a Modifier. If the Condition is met based on torrent metadata, the Modifier is applied
  • Conditions and Modifiers can each be compounded
  • Rules are applied in order. If multiple Rules match, the Modifiers are applied in order.
  • I've only implemented Conditions for file paths and tracker URLs and have only implemented Modifiers for setting the category and adding tags, but the code is designed to be modular so others could be added later.
  • Currently there is no GUI for adding / editing rules. You have to edit a JSON file that is loaded at startup.

Example Rules JSON:

{
    "rules": [
        {
            "condition": {
                "regex_pattern": ".*",
                "type": "AnyFilePathRegex"
            },
            "modifier": {
                "category": "DefaultCategory",
                "type": "SetCategory"
            }
        },
        {
            "condition": {
                "regex_pattern": ".*(jpg|webm)",
                "type": "AnyFilePathRegex"
            },
            "modifier": {
                "category": "Media",
                "type": "SetCategory"
            }
        },
        {
            "condition": {
                "regex_pattern": ".*",
                "type": "AnyTrackerUrlRegex"
            },
            "modifier": {
                "tag": "AnyTracker",
                "type": "AddTag"
            }
        },
        {
            "condition": {
                "regex_pattern": ".*jpg",
                "type": "AnyFilePathRegex"
            },
            "modifier": {
                "tag": "JPEG",
                "type": "AddTag"
            }
        },
        {
            "condition": {
                "regex_pattern": ".*webm",
                "type": "AnyFilePathRegex"
            },
            "modifier": {
                "tag": "WEBM",
                "type": "AddTag"
            }
        },
        {
            "condition": {
                "conditions": [
                    {
                        "regex_pattern": ".*mp3",
                        "type": "AnyFilePathRegex"
                    },
                    {
                        "regex_pattern": ".*webm",
                        "type": "AnyFilePathRegex"
                    },
                    {
                        "regex_pattern": ".*jpg",
                        "type": "AnyFilePathRegex"
                    }
                ],
                "type": "AnyOfConditions"
            },
            "modifier": {
                "tag": "Media",
                "type": "AddTag"
            }
        },
        {
            "condition": {
                "conditions": [
                    {
                        "regex_pattern": ".*webm",
                        "type": "AnyFilePathRegex"
                    },
                    {
                        "regex_pattern": ".*poster.*(jpg|png)",
                        "type": "AnyFilePathRegex"
                    }
                ],
                "type": "AllOfConditions"
            },
            "modifier": {
                "modifiers": [
                    {
                        "tag": "VideoWithPoster",
                        "type": "AddTag"
                    },
                    {
                        "tag": "MixedMedia",
                        "type": "AddTag"
                    }
                 ],
                "type": "Compound"
            }
        }
    ]
}

Example applied to a test torrent from https://webtorrent.io/torrents/tears-of-steel.torrent

Screenshot-from-2024-02-25-10-05-41

@glassez
Copy link
Member

glassez commented Feb 26, 2024

  • Currently there is no GUI for adding / editing rules. You have to edit a JSON file that is loaded at startup.

I suspect that providing an UI for editing such rules will require an order of magnitude (or even several orders of magnitude) more effort.

@glassez
Copy link
Member

glassez commented Feb 26, 2024

@tgregerson
There are several different cases of processing added torrents that should be taken into account.

@tgregerson
Copy link
Contributor

tgregerson commented Feb 26, 2024

I suspect that providing an UI for editing such rules will require an order of magnitude (or even several orders of magnitude) more effort.

I was thinking the same thing. My plan for tackling this was to try to submit the non-GUI version first, assuming that is OK with the maintainers.

This way the GUI could be tackled as a separate PR, and not block progress on the core functionality.

There are several different cases of processing added torrents that should be taken into account.

So far the two important factors I've run into are whether full metadata is initially available (e.g. torrent file vs magnet link) and whether the user has enabled the add torrent GUI from my earlier screenshot.

  • If all metadata is initially available, it's the simplest case. We can apply the rules before adding the torrent and before showing the GUI (if enabled).
  • If the GUI is enabled and metadata is not available, the torrent is added in hidden mode and asynchronously downloads metadata while the GUI is being displayed. If the metadata download finishes before the user has clicked OK, then we need to apply the rules and update the GUI to reflect them.
  • If the GUI is disabled and metadata is not available (or GUI is enabled and the user clicked OK before metadata download finished), then we add the torrent, wait for the metadata downloaded alert from libtorrent, then apply the rules, then update the GUI for the torrent list, etc.

@glassez
Copy link
Member

glassez commented Feb 29, 2024

So far the two important factors I've run into are whether full metadata is initially available (e.g. torrent file vs magnet link) and whether the user has enabled the add torrent GUI from my earlier screenshot.

  • If all metadata is initially available, it's the simplest case. We can apply the rules before adding the torrent and before showing the GUI (if enabled).
  • If the GUI is enabled and metadata is not available, the torrent is added in hidden mode and asynchronously downloads metadata while the GUI is being displayed. If the metadata download finishes before the user has clicked OK, then we need to apply the rules and update the GUI to reflect them.
  • If the GUI is disabled and metadata is not available (or GUI is enabled and the user clicked OK before metadata download finished), then we add the torrent, wait for the metadata downloaded alert from libtorrent, then apply the rules, then update the GUI for the torrent list, etc.

In general, I agree. Although there may be difficulties in correctly implementing it (from the perspective of the application architecture). But we will be able to talk about this in detail only when you provide the PR.

tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules consist of one
or more conditions and one or more modifiers. If a torrent satisfies
the rule's conditions, the modifiers are applied. Rules are specified
via a JSON file loaded on startup.

Included conditions:
* Tracker URL regex matching
* File path regex matching
* Combining multiple conditions as logical AND or OR

Included modifiers:
* Setting torrent category
* Adding a tag
* Combining multiple modifiers

The system is designed to be modular to allow further conditions and
modifiers can be added in the future.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules consist of one
or more conditions and one or more modifiers. If a torrent satisfies
the rule's conditions, the modifiers are applied. Rules are specified
via a JSON file loaded on startup.

Included conditions:
* Tracker URL regex matching
* File path regex matching
* Combining multiple conditions as logical AND or OR

Included modifiers:
* Setting torrent category
* Adding a tag
* Combining multiple modifiers

The system is designed to be modular to allow further conditions and
modifiers can be added in the future.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules consist of one
or more conditions and one or more modifiers. If a torrent satisfies
the rule's conditions, the modifiers are applied. Rules are specified
via a JSON file loaded on startup.

Included conditions:
* Tracker URL regex matching
* File path regex matching
* Combining multiple conditions as logical AND or OR

Included modifiers:
* Setting torrent category
* Adding a tag
* Combining multiple modifiers

The system is designed to be modular to allow further conditions and
modifiers can be added in the future.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules consist of one
or more conditions and one or more modifiers. If a torrent satisfies
the rule's conditions, the modifiers are applied. Rules are specified
via a JSON file loaded on startup.

Included conditions:
* Tracker URL regex matching
* File path regex matching
* Combining multiple conditions as logical AND or OR

Included modifiers:
* Setting torrent category
* Adding a tag
* Combining multiple modifiers

The system is designed to be modular to allow further conditions and
modifiers can be added in the future.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 2, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 3, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 3, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 3, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 3, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 3, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 3, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
@tgregerson tgregerson linked a pull request Mar 4, 2024 that will close this issue
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 4, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 4, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 4, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 5, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
tgregerson added a commit to tgregerson/qBittorrent that referenced this issue Mar 5, 2024
Adds a system for configuring a set of rules that modify torrent
parameters based on the torrent's metadata. Rules are specified
via a JSON file loaded on startup.

Closes qbittorrent#5779.
@j-peeters
Copy link

@tgregerson thanks for your work.

I've read through the thread but I haven't yet understood fully how this would work.
My use case would be to put torrents in one watch folder and let the rule json determine the categorie that is attached.
I have an Unraid server with QB in a docker. For now I've setup five or so watch folders with different download paths to get downloads in the right folder, but I was used from TransmissionBT that I could just drop everything in one folder and let the rules sort it out.
My questions are:

  • How to make this json file process the torrent, where do I place it and how does QB know what to do with it?
  • I'm also wondering (since you have the screnshot under "Example applied to a test torrent" if this json can be used without user interaction.

I would love to test this and see if I can get QB to automatically attach categories.
Thanks again.

@tgregerson
Copy link
Contributor

The PR has not been approved yet, so these answers are subject to change.

How to make this json file process the torrent, where do I place it and how does QB know what to do with it?

The file will be stored in the preferences directory.

https://github.com/qbittorrent/qBittorrent/wiki/Frequently-Asked-Questions#where-does-qbittorrent-save-its-settings

qBt will automatically check for the file on startup. If the file exists, then any rules you have defined will be applied automatically every time you add a new torrent.

I'm also wondering (since you have the screnshot under "Example applied to a test torrent" if this json can be used without user interaction.

No interaction required.

@j-peeters
Copy link

@tgregerson terrific, I hope it gets approved soon. Thanks again!

@leo-carmo
Copy link

I'd love to see automatic category assignment not only by keywords, but by other possible criteria, such as total torrent size or tracker.

@tgregerson
Copy link
Contributor

I'd love to see automatic category assignment not only by keywords, but by other possible criteria, such as total torrent size or tracker.

The initial PR supports using trackers as a condition.

I'm open to adding other options such as torrent size in subsequent PRs, provided the initial one is accepted.

@KingPsychopath
Copy link

I'd love to see automatic category assignment not only by keywords, but by other possible criteria, such as total torrent size or tracker.

Bit outside of the scope of this current issue, but we've come a long way.

Can't wait for the PR pull.

I have a basic prototype working.

  • A configurable list of Rules are applied when adding a new torrent
  • A Rule consists of a Condition and a Modifier. If the Condition is met based on torrent metadata, the Modifier is applied
  • Conditions and Modifiers can each be compounded
  • Rules are applied in order. If multiple Rules match, the Modifiers are applied in order.
  • I've only implemented Conditions for file paths and tracker URLs and have only implemented Modifiers for setting the category and adding tags, but the code is designed to be modular so others could be added later.
  • Currently there is no GUI for adding / editing rules. You have to edit a JSON file that is loaded at startup.

Example Rules JSON:

{
    "rules": [
        {
            "condition": {
                "regex_pattern": ".*",
                "type": "AnyFilePathRegex"
            },
            "modifier": {
                "category": "DefaultCategory",
                "type": "SetCategory"
            }
        },
        {
            "condition": {
                "regex_pattern": ".*(jpg|webm)",
                "type": "AnyFilePathRegex"
            },
            "modifier": {
                "category": "Media",
                "type": "SetCategory"
            }
        },
        {
            "condition": {
                "regex_pattern": ".*",
                "type": "AnyTrackerUrlRegex"
            },
            "modifier": {
                "tag": "AnyTracker",
                "type": "AddTag"
            }
        },
        {
            "condition": {
                "regex_pattern": ".*jpg",
                "type": "AnyFilePathRegex"
            },
            "modifier": {
                "tag": "JPEG",
                "type": "AddTag"
            }
        },
        {
            "condition": {
                "regex_pattern": ".*webm",
                "type": "AnyFilePathRegex"
            },
            "modifier": {
                "tag": "WEBM",
                "type": "AddTag"
            }
        },
        {
            "condition": {
                "conditions": [
                    {
                        "regex_pattern": ".*mp3",
                        "type": "AnyFilePathRegex"
                    },
                    {
                        "regex_pattern": ".*webm",
                        "type": "AnyFilePathRegex"
                    },
                    {
                        "regex_pattern": ".*jpg",
                        "type": "AnyFilePathRegex"
                    }
                ],
                "type": "AnyOfConditions"
            },
            "modifier": {
                "tag": "Media",
                "type": "AddTag"
            }
        },
        {
            "condition": {
                "conditions": [
                    {
                        "regex_pattern": ".*webm",
                        "type": "AnyFilePathRegex"
                    },
                    {
                        "regex_pattern": ".*poster.*(jpg|png)",
                        "type": "AnyFilePathRegex"
                    }
                ],
                "type": "AllOfConditions"
            },
            "modifier": {
                "modifiers": [
                    {
                        "tag": "VideoWithPoster",
                        "type": "AddTag"
                    },
                    {
                        "tag": "MixedMedia",
                        "type": "AddTag"
                    }
                 ],
                "type": "Compound"
            }
        }
    ]
}

Example applied to a test torrent from https://webtorrent.io/torrents/tears-of-steel.torrent

Screenshot-from-2024-02-25-10-05-41

I had something similar except it was just a daemon script running agnostic of qBit since I'm not well versed in C++
Thank you for your work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.