代码拉取完成,页面将自动刷新
1252
[157] Spiller serialization optimizations
已合并
What type of PR is this?
/kind feature
What does this PR do / why do we need it:
- Single pass direct serialize / deserialize, instead of 2 pass (Page -> SerializedPage -> file/stream)
- Added Snappy Compressed writter
- Added configuration for direct serializer enable
- Added Spill/Unspill Timing stats capture and relayed to CLI
Performance test observation: [over TPCDS Q74]
Sl No | Base(No Spill) | Base(With Spill) | with Direct | Improvement |
---|---|---|---|---|
Spill Write (s) | 0 | 999.2 | 684 | 31.5% |
Spill Read (s) | 0 | 256 | 151 | 41.0% |
Overall Spill (m) | 0 | 4.21 | 2.77 | 33.5% |
QueryTime (m) | 5.89 | 25.58 | 20.19 | 19.6% |
QueryTime Degrade(Ratio) | 0 | 4.3x | 3.3x | 1x |
Which issue(s) this PR fixes:
Fixes #157 , #I4JUF0:Performance Improvement in Spilling to Disk
Special notes for your reviewers:
This pr was synchronized from the Github Repository