1 Star 0 Fork 0

luohan / pika

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
BSD-3-Clause

Pika

简介 English

Pika是一个可持久化的大容量redis存储服务,兼容string、hash、list、zset、set的绝大部分接口(兼容详情),解决redis由于存储数据量巨大而导致内存不够用的容量瓶颈,并且可以像redis一样,通过slaveof命令进行主从备份,支持全同步和部分同步,pika还可以用在twemproxy或者codis中来实现静态数据分片(pika已经可以支持codis的动态迁移slot功能,目前在合并到master分支,欢迎使用,感谢作者left2rightfancy-rabbit提交的pr)

Pika用户

Qihoo 360game Weibo Garena
Apus Ffan Meituan XES
HX XL GWD DYD
YM XM XL YM
MM VIP LK KS

更多

特点

  • 容量大,支持百G数据量的存储
  • 兼容redis,不用修改代码即可平滑从redis迁移到pika
  • 支持主从(slaveof)
  • 完善的运维命令

使用

二进制包使用

用户可以直接从releases下载最新的二进制版本包直接使用.

编译使用

1.在编译机上安装snappy,glog,CentOS系统可以用yum安装,Ubuntu可以用apt-get安装。如是CentOS系统,执行如下命令:

    yum install snappy-devel glog-devel

2.安装g++(若没有安装), 在CentOS上执行如下命令:

    yum install gcc-c++

3.把gcc版本临时切换到4.8(若已是,则忽略), 在CentOS上执行如下命令:

	a. sudo wget http://people.centos.org/tru/devtools-2/devtools-2.repo -O /etc/yum.repos.d/devtools-2.repo
	b. sudo yum install -y devtoolset-2-gcc devtoolset-2-binutils devtoolset-2-gcc-c++
	c. scl enable devtoolset-2 bash

4.获取源代码

	git clone https://github.com/Qihoo360/pika.git && cd pika

5.切换到最新release版本

	a. 执行 git tag 查看最新的release tag,(如 v2.2.5)
	b. 执行 git checkout TAG切换到最新版本,(如 git checkout v2.2.5)

6.编译

	make

若编译过程中,提示有依赖的库没有安装,则有提示安装后再重新编译

注:我们推荐使用TCMalloc来进行内存管理

使用

	./output/bin/pika -c ./conf/pika.conf

清空编译

  如果需要清空编译内容,视不同情况使用一下两种方法其一:

  1. 执行make clean来清空pika的编译内容
  2. 执行make distclean来清空pika及所有依赖的编译内容(一般用于彻底重新编译)

性能 (感谢deep011提供性能测试结果)

注!!!

本测试结果是在特定环境特定场景下得出的,不能够代表所有环境及场景下的表现,仅供参考

推荐大家在使用pika前在自己的环境根据自己的使用场景详细测试以评估pika是否满足要求

测试环境

CPU型号:Intel(R) Xeon(R) CPU E5-2690 v4 @ 2.60GHz

CPU线程数:56

MEMORY:256G

DISK:3T flash

NETWORK:10GBase-T/Full * 2

OS:centos 6.6

Pika版本:2.2.4

压测工具

vire-benchmark

测试一

测试目的

测试在pika不同worker线程数量下,其QPS上限。

测试条件

pika数据容量:800G

value:128字节

CPU未绑定

测试结果

说明:横轴Pika线程数,纵轴QPS,value为128字节。set3/get7代表30%的set和70%的get。

1

结论

从以上测试图可以看出,pika的worker线程数设置为20-24比较划算。

测试二

测试目的

测试在最佳worker线程数(20线程)下,pika的rtt表现。

测试条件

pika数据容量:800G

value:128字节

测试结果

====== GET ======
  10000000 requests completed in 23.10 seconds
  200 parallel clients
  3 bytes payload
  keep alive: 1
99.89% <= 1 milliseconds
100.00% <= 2 milliseconds
100.00% <= 3 milliseconds
100.00% <= 5 milliseconds
100.00% <= 6 milliseconds
100.00% <= 7 milliseconds
100.00% <= 7 milliseconds
432862.97 requests per second
====== SET ======
  10000000 requests completed in 36.15 seconds
  200 parallel clients
  3 bytes payload
  keep alive: 1
91.97% <= 1 milliseconds
99.98% <= 2 milliseconds
99.98% <= 3 milliseconds
99.98% <= 4 milliseconds
99.98% <= 5 milliseconds
99.98% <= 6 milliseconds
99.98% <= 7 milliseconds
99.98% <= 9 milliseconds
99.98% <= 10 milliseconds
99.98% <= 11 milliseconds
99.98% <= 12 milliseconds
99.98% <= 13 milliseconds
99.98% <= 16 milliseconds
99.98% <= 18 milliseconds
99.99% <= 19 milliseconds
99.99% <= 23 milliseconds
99.99% <= 24 milliseconds
99.99% <= 25 milliseconds
99.99% <= 27 milliseconds
99.99% <= 28 milliseconds
99.99% <= 34 milliseconds
99.99% <= 37 milliseconds
99.99% <= 39 milliseconds
99.99% <= 40 milliseconds
99.99% <= 46 milliseconds
99.99% <= 48 milliseconds
99.99% <= 49 milliseconds
99.99% <= 50 milliseconds
99.99% <= 51 milliseconds
99.99% <= 52 milliseconds
99.99% <= 61 milliseconds
99.99% <= 63 milliseconds
99.99% <= 72 milliseconds
99.99% <= 73 milliseconds
99.99% <= 74 milliseconds
99.99% <= 76 milliseconds
99.99% <= 83 milliseconds
99.99% <= 84 milliseconds
99.99% <= 88 milliseconds
99.99% <= 89 milliseconds
99.99% <= 133 milliseconds
99.99% <= 134 milliseconds
99.99% <= 146 milliseconds
99.99% <= 147 milliseconds
100.00% <= 203 milliseconds
100.00% <= 204 milliseconds
100.00% <= 208 milliseconds
100.00% <= 217 milliseconds
100.00% <= 218 milliseconds
100.00% <= 219 milliseconds
100.00% <= 220 milliseconds
100.00% <= 229 milliseconds
100.00% <= 229 milliseconds
276617.50 requests per second

结论

get/set 响应时间 99.9%都在2ms以内。

测试三

测试目的

在pika最佳的worker线程数下,查看各命令的极限QPS。

测试条件

pika的worker线程数:20

key数量:10000

field数量:100(list除外)

value:128字节

命令执行次数:1000万(lrange除外)

测试结果

PING_INLINE: 548606.50 requests per second
PING_BULK: 544573.31 requests per second
SET: 231830.31 requests per second
GET: 512163.91 requests per second
INCR: 230861.56 requests per second
MSET (10 keys): 94991.12 requests per second
LPUSH: 196093.81 requests per second
RPUSH: 195186.69 requests per second
LPOP: 131156.14 requests per second
RPOP: 152292.77 requests per second
LPUSH (needed to benchmark LRANGE): 196734.20 requests per second
LRANGE_10 (first 10 elements): 334448.16 requests per second
LRANGE_100 (first 100 elements): 50705.12 requests per second
LRANGE_300 (first 300 elements): 16745.16 requests per second
LRANGE_450 (first 450 elements): 6787.94 requests per second
LRANGE_600 (first 600 elements): 3170.38 requests per second
SADD: 160885.52 requests per second
SPOP: 128920.80 requests per second
HSET: 180209.41 requests per second
HINCRBY: 153364.81 requests per second
HINCRBYFLOAT: 141095.47 requests per second
HGET: 506791.00 requests per second
HMSET (10 fields): 27777.31 requests per second
HMGET (10 fields): 38998.52 requests per second
HGETALL: 109059.58 requests per second
ZADD: 120583.62 requests per second
ZREM: 161689.33 requests per second
PFADD: 6153.47 requests per second
PFCOUNT: 28312.57 requests per second
PFADD (needed to benchmark PFMERGE): 6166.37 requests per second
PFMERGE: 6007.09 requests per second

结论

整体表现很不错,个别命令表现较弱(LRANGE,PFADD,PFMERGE)。

测试四

测试目的

Pika与Redis的极限QPS对比。

测试条件

pika的worker线程数:20

key数量:10000

field数量:100(list除外)

value:128字节

命令执行次数:1000万(lrange除外)

Redis版本:3.2.0

测试结果

1

文档

  1. [Wiki] (https://github.com/Qihoo360/pika/wiki)

联系方式

邮箱:g-infra@360.cn

QQ群:294254078

Copyright (c) 2015-2020, Qihoo360 All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

简介

暂无描述 展开 收起
C++ 等 5 种语言
BSD-3-Clause
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
1
https://gitee.com/luohanzju/pika.git
git@gitee.com:luohanzju/pika.git
luohanzju
pika
pika
master

搜索帮助