We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Currently, parquet native write doesn't support customized option as block_size, page_size etc. when write data.
block_size
page_size
There is a onging PR in velox side https://github.com/facebookincubator/velox/pull/8864/files#diff-5a2dd3766d9a74bbef58d62d96f0abfb111e8e507ce9bcecd35f69d2c8669ed7 to let support pass flushPolicy which support customize block_size, but not page_size.
flushPolicy
Let use this issue to track for supporting customize all parquet options when write data.
The text was updated successfully, but these errors were encountered:
@JkSelf let use this issue to track the customized block_size, page_size not support problem.
Sorry, something went wrong.
@gaoyangxiaozhu can you list all the parquet write parameters Spark supports and velox/arrow supports? Let's pass all supported params to Velox
No branches or pull requests
Description
Currently, parquet native write doesn't support customized option as
block_size
,page_size
etc. when write data.There is a onging PR in velox side https://github.com/facebookincubator/velox/pull/8864/files#diff-5a2dd3766d9a74bbef58d62d96f0abfb111e8e507ce9bcecd35f69d2c8669ed7 to let support pass
flushPolicy
which support customizeblock_size
, but notpage_size
.Let use this issue to track for supporting customize all parquet options when write data.
The text was updated successfully, but these errors were encountered: