-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
aws-s3-multipart + golden retriever #5180
Comments
I am experiencing the exact same problem with implementing uppy with the s3 multipart upload + golden retriever in an attempt to get it to resume the file upload where it left off after cutting the connection via a browser page refresh. Main difference between your implementation and mine is that i did not use companion or Tus because i really do not want to setup a server just to transfer the file to s3 in the end. I think it is more efficient and not overly complicated to achieve resumable uploads without the need of another server for the file to flow through before it goes to s3. Thinking that this may not be resolved and may have to build a solution myself that mimics this library. |
Probably better for the ecosystem and to save yourself some effort if you are willing to contribute rather than building it again. Would you be interested in taking a look? |
Wanted to post an update about this issue here. My co-workers have been able to take a crack at the implementation of Uppy with the golden retriever plugin without the flag for the webWorker and have been successful. The main issue they were able to solve was an error regarding an invalid signature (s3 signing) when attempting to resume the upload from where it was left off. After this was resolved, it seems that the solution works the way we need it (resumable uploads without the need of another server and without the need of Companion or Tus) My suspicion is that the invalid signature error was caused by a malformed listParts command on the server which is used when attempting to resume the upload. Note: The Error posted at the beginning of this thread I believe is still valid. The good thing is that it seems seems that the webWorker may not be needed in order to implement a client -> s3 upload solution with support for resumable uploading. For reference, I had used the example for the nodejs-aws server endpoints that are in this file: https://github.com/transloadit/uppy/blob/main/examples/aws-nodejs/index.js The problem code is in this part:
The solution that seems to work well is changing how the listParts is called, this is what I have come up with that seems to work (need to change the endpoint handler to be asynchronous) :
I have submitted a PR to fix that issue in the aws-nodejs example to hopefully help others avoid the same error. I hope this helps others who are trying to setup a similar solution using Uppy *EDIT: I have realized that the solution I have explained here does not need to change the code so much, instead it is just a matter of conditionally passing in the PartNumberMarker into the config of the ListPartsCommand that is sent to s3. I have also updated the PR to reflect this. here is what that endpoint handler looks like now with the update.
|
Initial checklist
Link to runnable example
No response
Steps to reproduce
Similar to #4985, where the problem was "listParts". However, my "listParts" function is never run. Not even if I declare it in js in "use(AwsS3", so I guess it crashes before then.
Use multipart + golden retriever
I have implemented the companion endpoints for multipart uploads in laravel as shown below:
Everything works perfectly, except restoring the upload.
Expected behavior
restore all chunks.
Actual behavior
Crashes on reading size when restoring files.
Thanks for the help.
The text was updated successfully, but these errors were encountered: