Provider views rewrite (.files, .folders => .partialTree) #5050

lakesare · 2024-03-29T00:24:37Z

Description

enables indeterminate checkmark states
[fixes Nested folder selection does not work correctly #4609]
enables folder caching
fixes the issue where Unsplash was only loading one page
[fixes Unsplash doesn't load further pages #5000]
reworks restrictions system
[fixes Fix logic with adding folders toast message #4414]

In particular aggregate restrictions are now shown in the footer:
absDirPath and relDirPath are injected in a single place
[fixes pull/4537#discussion_r1260683896]
nOfSelectedFiles is as smart as it gets now
fixes the UI issue where shift-clicking files gets them highlighted:
fixes the way shift-clicking works in grid providers such as Instagram/Unpslash
[fixes Shift-clicking works chaotically with Instagram/Unsplash #5063]
makes the GoogleDrive's VIRTUAL_SHARED_DIR checkable
[see this discussion transloadit.slack.com/archives/C0FMW9PSB/p1714529071856209]

TODOs

For each provider, check if 1. signing in 2. checking deeply nested folders 3. downloading them 4. signing out works.
[To reviewers: I'll do it, no need to check all providers yourselves]

TODOs for next PRs

This PR removes @mifi's change that's reporting loading progress (bring back "loaded X files..." #5097).
I'll work on "mifi's PR + but custom handling for smaller n of files" in further PRs.

Notes to reviewers

I made deliberate effort not to touch the folder structure at all (for ease of reviewing & because we didn't set our minds on which one we'd prefer yet).
Don't worry about 1k of added lines - this PR actually reduces the number of lines by a few hundred lines - the increase is due to test files/comments.
We discussed a shared .env in Slack - it's underway, but not yet fully ready.
If you need access to a particular provider - message me, and I'll send you the credentials.

This PR spans 2.5k lines across 46 files.

I separated some PRs from this PR, however further separation will make it harder to review rather than easier, this is largely a holistic change.

I think a sane way to review this PR (apart from looking through "Files changed" on github) would be to look through the code in /uppy/packages/@uppy/provider-views folder locally in your editor - just read the files and see if they make sense/if something worries you. Most of these files were updated/rewritten.

GoogleDrive - travelling down into folders works - checking a file works - breadcrumbs DONT work

…ed()` Clarification: `.toReversed()` is no supported by all browsers

See #5232

lakesare · 2024-06-07T12:59:59Z

packages/@uppy/google-drive/src/DriveProviderViews.ts

EXPLANATION

We don't need this code, because we hide checkmarks for team drives in <ListItem/> now (see #5232).
This version was leading to a weird behaviour where the checkbox was there, but it wasn't working.

lakesare · 2024-06-07T13:05:10Z

packages/@uppy/provider-views/src/Item/components/ItemIcon.tsx

EXPLANATION

These changes are here just to remove the /* eslint-disable react/require-default-props */ line.

lakesare · 2024-06-07T13:08:54Z

packages/@uppy/provider-views/src/ProviderView/AuthView.tsx

EXPLANATION

Again - these changes are here just to remove the /* eslint-disable react/require-default-props */ line.

lakesare · 2024-06-07T13:10:37Z

packages/@uppy/provider-views/src/ProviderView/User.tsx

-      <span className="uppy-ProviderBrowser-user" key="username">
-        {username}
-      </span>
+      {username && (


EXPLANATION

This username && thing fixes the issue in some providers where we would see a small dot even when we don't have the username (doesn't look good).

lakesare · 2024-06-07T13:12:04Z

packages/@uppy/provider-views/src/SearchInput.tsx

EXPLANATION

Hugely simplifying <SearchInput/> here by making it a controlled component.
That's also why we don't need nanoid anymore.

lakesare · 2024-06-07T13:15:23Z

packages/@uppy/provider-views/src/View.ts

EXPLANATION

Removed View.ts, all code from here moves to ProviderView.ts and SearchProviderView.ts.

Pretty much all of the code that was in the View.ts superclass needed to be moved into /utils files (code is complex enough to deserve a predictable util file that's easy to single out to work on & to unit test), so View.ts was becoming thinner and thinner as I went on.
When ProviderView.ts and SearchProviderView.ts have the same behaviour, they just use the same function from /utils now.

As a bonus, removing View.ts superclass makes the types much more straightforward, and removes "method from childClass calls a method from superClass which depends on a method from childClass" situations.

Murderlon

Overall very happy with the functionality! Congrats.

Unfortunately the PR is overwhelmingly big, I'm having a hard time reviewing it.

A lot of time went into this already and there are code comments, tests, and from a glance most things seem very well structured. So I don't think it makes sense to force you to split this up into many PRs.

But I would like to point out that I do see it as an expectation to know how to make small PRs, and this could've probably been 5 to 10 PRs. It's more a note for us as a team (I'm also still improving at forcing myself to make small PRs).

Murderlon · 2024-06-10T11:59:11Z

packages/@uppy/provider-views/src/utils/getTagFile.ts

+// TODO: document what is a "tagFile" or get rid of this concept
+const getTagFile = <M extends Meta, B extends Body>(


Did it became more clear to you why this is needed?

[To be clear to onlookers - it's not my comment, I carried this comment over from another file]
I didn't dive into this, but can be my next PR, I'm deep enough in this context.
I'll record this in my #todos.

It's not too important, if you happened to know we could've add a comment :)

lakesare · 2024-06-12T16:33:36Z

Unfortunately the PR is overwhelmingly big, I'm having a hard time reviewing it.
A lot of time went into this already and there are code comments, tests, and from a glance most things seem very well structured. So I don't think it makes sense to force you to split this up into many PRs.

Realistically the only way this could have been a set of smaller PRs is if I have been splitting this PR post-factum.

There hasn't been a way forward with a number of smaller PRs. I changed the underlying data structure multiple times as I was going through with this PR, and every such overhaul was touching most files. It's only now clear what smaller PRs could have been of use in the final version.

I can still do the post-factum split, but I don't believe it will significantly ease the review for you - the main workload of this PR is in changing the data structure used throughout provider views, which by its nature touches 40 files.

Sometimes (say, when the data structure used by all files changes completely) a big PR is unavoidable.

I understand it's challenging to review - like mentioned in the PR description, I suggest going through the folders in your editor, and seeing if it make sense. In particular would be great if the reviewers could go through code in /utils and see if it compiles in their head.

lakesare added 30 commits February 14, 2024 10:10

Merge branch 'main' of https://github.com/transloadit/uppy into main

c4ba080

Merge branch 'main' of https://github.com/transloadit/uppy

631abc6

Merge branch 'main' of https://github.com/transloadit/uppy into main

75eae0a

Merge branch 'main' of https://github.com/transloadit/uppy

462e61f

Merge branch 'main' of https://github.com/transloadit/uppy into main

47df4a6

Merge branch 'main' of https://github.com/transloadit/uppy

df682a1

Merge branch 'main' of https://github.com/transloadit/uppy into main

ac3badf

Merge branch 'main' of https://github.com/transloadit/uppy into main

76615a1

ProviderView.tsx - fix onedrive breadcrumbs

b234045

Merge branch 'lakesare/fix-breadcrumbs' into lakesare/provider-folders

092a390

providers - correct ch-unch-indeterminate states

10cbbf7

providers - made .breadcrumbs derived from .partialTree

8796c35

everywhere - { files, folders, isChecked } => .partialTree

e2d0d04

GoogleDrive - travelling down into folders works - checking a file works - breadcrumbs DONT work

GoogleDrive - made breadcrumbs work

ab42ad6

.getFolder() - remove the name argument

1ba90c7

<Breadcrumbs/> - refactors "/"

ec532a2

Instagram - made files get fetched onScroll

51fc807

clearSelection() - recover the functionality

ba0b097

GoogleDrive - recover custom .toggleCheckbox() functionality

dc589c8

providers - recover .isDisabled functionality

55ef784

<SearchProviderView/> - made Unsplash use .partialTree

5f356a0

Facebook - change .files, .folders => .partialTree

74b01c5

everywhere - we don't need to ! partialTreeFile.data anymore

101032c

<ProviderView/> - implement folder caching

e12b26f

<View/> - enable shift-clicking

4bc143f

everywhere - get rid of unnecessary .getNextFolder()

f96f99f

everywhere - fixing types

296cae3

<ProviderView/> - rename requestPath to folderId

7123684

all providers - get rid of .onFirstRender()

a5dc9e5

provider views - get rid of .onFirstRender()

60a9c1e

lakesare added 17 commits June 4, 2024 12:34

merge

e72c709

fixing 8 (yarn run build:ts)

ad8f966

fixing 9 (run corepack yarn)

1163198

prettier - undo indentation harm done by prettier

e447cb7

getBreadcrumbs() - add tests, and rewrite to avoid using `.toRevers…

0b6571b

…ed()` Clarification: `.toReversed()` is no supported by all browsers

merge

947bba3

<SearchInput/> - make it work for eslint

07ed69b

everywhere - remove eslint-disable react/require-default-props

bf0ed8d

<GridItem/>, <ListItem/> - refactor to avoid prop drilling

d358c9a

<ListItem/> - disable checkboxes for GoogleDrive team drives

294a301

See #5232

merge (fixing up some lines from the previous merge)

a65dae3

merge (fixing up some lines from the previous merge)

258c379

everywhere - remove TEMP development values

b54d12e

this.validateSingleFile() - switch to .restrictionError

464f614

afterToggleCheckbox.ts - refactor, add comments

c94b893

afterToggleCheckbox.ts - refactor to use ids instead of whole objects

ab6a13c

afterToggleCheckbox.ts - try to satisfy prettier

55af6d8

lakesare commented Jun 7, 2024

View reviewed changes

lakesare marked this pull request as ready for review June 7, 2024 13:19

lakesare requested review from Murderlon and mifi June 7, 2024 13:19

lakesare added 2 commits June 12, 2024 18:16

fixing 10 (try to satisfy npx webpack)

657ef98

fixing 11 (try to satisfy npx webpack)

4081f06

Murderlon reviewed Jun 12, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Provider views rewrite (.files, .folders => .partialTree) #5050

Provider views rewrite (.files, .folders => .partialTree) #5050

lakesare commented Mar 29, 2024 •

edited

lakesare Jun 7, 2024 •

edited

lakesare Jun 7, 2024

lakesare Jun 7, 2024

lakesare Jun 7, 2024 •

edited

lakesare Jun 7, 2024

lakesare Jun 7, 2024 •

edited

Murderlon left a comment •

edited

Murderlon Jun 10, 2024

lakesare Jun 12, 2024

Murderlon Jun 12, 2024

lakesare commented Jun 12, 2024 •

edited

		// TODO: document what is a "tagFile" or get rid of this concept
		const getTagFile = <M extends Meta, B extends Body>(

Provider views rewrite (.files, .folders => .partialTree) #5050

Are you sure you want to change the base?

Provider views rewrite (.files, .folders => .partialTree) #5050

Conversation

lakesare commented Mar 29, 2024 • edited

Description

TODOs

TODOs for next PRs

Notes to reviewers

lakesare Jun 7, 2024 • edited

Choose a reason for hiding this comment

lakesare Jun 7, 2024

Choose a reason for hiding this comment

lakesare Jun 7, 2024

Choose a reason for hiding this comment

lakesare Jun 7, 2024 • edited

Choose a reason for hiding this comment

lakesare Jun 7, 2024

Choose a reason for hiding this comment

lakesare Jun 7, 2024 • edited

Choose a reason for hiding this comment

Murderlon left a comment • edited

Choose a reason for hiding this comment

Murderlon Jun 10, 2024

Choose a reason for hiding this comment

lakesare Jun 12, 2024

Choose a reason for hiding this comment

Murderlon Jun 12, 2024

Choose a reason for hiding this comment

lakesare commented Jun 12, 2024 • edited

lakesare commented Mar 29, 2024 •

edited

lakesare Jun 7, 2024 •

edited

lakesare Jun 7, 2024 •

edited

lakesare Jun 7, 2024 •

edited

Murderlon left a comment •

edited

lakesare commented Jun 12, 2024 •

edited