-
Notifications
You must be signed in to change notification settings - Fork 6.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HDFS is unsupported #64126
HDFS is unsupported #64126
Conversation
This is an automated comment for commit 39b84d0 with description of existing statuses. It's updated for the latest CI running ✅ Click here to open a full report in a separate page Successful checks
|
docs/en/operations/storing-data.md
Outdated
@@ -464,9 +466,11 @@ In this sample configuration: | |||
</clickhouse> | |||
``` | |||
|
|||
Keep in mind that HDFS does not work at all. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May be this message is too much? "At all" is not really true... If if did not work at all, there would have been no sense having hdfs in documentation at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kssenii latest HDFS have a hard limit 1million files (objects) per folder.
Clickhouse stores all objects in a single HDFS folder (for HDFS as external storage, unlike S3).
The average table with 500 columns reaches this limit of 1million objects easily, it's less than 1000 active+inactive wide parts, after that a table is unable to insert or to merge.
Co-authored-by: Kseniia Sumarokova <54203879+kssenii@users.noreply.github.com>
Changelog category (leave one):
Modify your CI run
NOTE: If your merge the PR with modified CI you MUST KNOW what you are doing
NOTE: Checked options will be applied if set before CI RunConfig/PrepareRunConfig step
Include tests (required builds will be added automatically):
Exclude tests:
Extra options:
Only specified batches in multi-batch jobs: