22 Star 81 Fork 22

weiboad / fiery

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

Fiery

logo Build Status License

Fiery 是一款为PHP提供服务的性能跟踪监控系统,可以方便的查看线上调用关系,响应性能,回放每次请求的具体执行过程、参数、异常。并且对系统异常及依赖数据性能做了去重统计,具有部署简单方便,开箱即用的优点。

部署在开发环境可以方便调试,部署在线上服务器可以快速定位线上故障及接口性能分析,有助于改善完善项目稳定性。

此项目还有C++11企业版本,但是由于依赖过多暂时并未对外开源。

使用Java开源一个针对PHP的服务只是为了方便用户部署,降低中小流量用户入门门槛。

集成此服务并不会影响原有接口性能

最低配置要求

  • PHP 5.3 or later with bcmath 扩展
  • 目前仅支持64位 UTF8编码PHP项目
  • Linux, MacOS、Windows
  • 内存: 4G+ (实际使用2G左右,预留2G应对突发峰值)
  • Java 8 Runtime

概念指引

模块及功能

dataflow

Ragnar Fiery主要由三个部分组成:

  • FierySDK:PHP埋点库(https://github.com/weiboad/fierysdk)
  • Log Pusher:日志收集及推送 在本项目内
  • Fiery Server:日志存储索引统计及管理界面 在本项目内

FierySDK

FierySDK需要在Nginx引入一个环境变量文件(fierysdk/nginx/fiery_fastcgi_params),通过这个文件可以获取一些环境变量,如FierySDK生成日志路径。当前服务器IP等信息。

FierySDK目前已打包成Composer,使用时使用Composer引入项目。按照集成指引在:

  • 框架入口
  • CURL基类
  • Mysql基类

以上类内埋点后即可工作,验证SDK是否工作可以查看Nginx变量内指定的路径内是否有日志产生,如果没有,需要检查目录是否有写入权限。

如仍旧不工作可以在FierySDK的src内RagnarSDK.php的代码打断点 进行原因排查。

后期若有精力可以对PHP项目的分级日志,以及一些性能关键函数内进行埋点,可以获得更多信息。

LogPusher

LogPusher部署在每一台运行集成FierySDK项目的服务器内,主要职能是收集指定目录内日志内容,通过HTTP或Kafka推送所有日志到Fiery服务,通过这个服务我们才可以将日志传输到存储和分析服务器内。本服务支持多子目录内日志监控,能够定期清理过期日志(outtime参数)。

Fiery Server

主要日志存储索引统计服务,所有日志都会存储索引在这个服务,这个服务目前服务于一个2000w QPS的项目(为了保证质量,接口请求返回都做了记录,每天40G左右的日志)限制服务使用4G内存,实际使用2G左右。Fiery Server会从Kafka或http推送获取Logpusher取来的FierySDK日志。通过对日志的统计、分析、存储、索引,完成分布式跟踪过程。

服务端安装步骤

  1. 下载并安装 Java 8 Runtime
  2. 下载Fiery最新的 Fiery Release page jar包
  3. 在jar所在目录创建文件夹 mkdir logs index db
  4. 通过以下命令启动主服务:
nohup java -XX:-MaxFDLimit -Xms3750m -Xmx3750m -XX:ReservedCodeCacheSize=240m -XX:+UseCompressedOops -jar ragnarserver-版本号.jar  --server.port=9090 &
  1. 服务启动后 浏览器访问地址: http://127.0.0.1:9090/ragnar/ 即可

PHP项目埋点库埋点介绍

LogPusher 日志收集及推送服务

日志推送服务,可以监控一个目录下所有日志是否有更新,并将内容推送到主服务

nohup java -XX:-MaxFDLimit -Xms128m -Xmx450m -XX:ReservedCodeCacheSize=240m -XX:+UseCompressedOops -jar logpusher-版本号.jar -c ./conf/logpusher.properties &

参数及配置文件说明

Logpusher(使用配置文件,由命令行参数-c指定配置文件路径)

    配置选项            选项       说明   
path             要监控的日志路径 此选项用于logpusher 指向ragnarsdk产生日志目录
outTime         本地日志保存时间 超时日志会被自动清理
pushType   推送类型 http(默认) kafka(用于大流量用户) 日志传输模式
host 127.0.0.1:9090 日志收集到后会推送到fiery server,pushtype=http时用的参数
threadCount 推送日志线程数默认8 越大越快,但是CPU使用会增加
kafkaServer   kafka服务器列表如 10.10.1.1:9192,10.10.1.2:9192 kafka broker服务器列表
kafkaTopic   kafka内的topic如fiery_test 向那个topic推送日志,支持多topic订阅,如fiery_test1,fiery_test2

Server(使用命令行参数)

    命令行参数            选项       说明   
--server.port fiery 服务监听端口如9090
--fiery.kafkaenable true false 是否开启kafka的消费端
--fiery.kafkaserver 10.1.1.1:9191,10.1.1.2:9191 kafka的broker服务器ip:port列表
--fiery.kafkatopic kafka topic kafka topic名称可以订阅多个,使用英文逗号隔开即可
--fiery.kafkagroupid kafka groupid 消费端groupid
--fiery.keepdataday 日志及数据保留时间(天)默认五天 统计,日志,排行信息等保存天数

Kafka版本 默认集成的是0.9的kafka协议,如果需要更新的协议支持,直接修改pom.xml的kafka依赖包版本即可


功能界面介绍

调用回放

showtrace

  • 展示多级API的调用关系,接口性能,响应状态,调用层级,服务器IP,用于查看此次服务质量,响应结果,以及相关参数及日志
  • 通过这个功能可以对线上所有请求进行回放调用过程方便快速找到接口故障原因(性能问题,逻辑问题等)
  • 线下线上运行或测试 时出现故障后可直接提供Traceid给研发直接查看原因
  • 线上故障可在错误提示附带Traceid方便快速查找问题
最近请求

recentrequest

  • 查看最近的请求列表,一般会有5秒延迟,通过列表可以直接查看当前最新请求的信息且可以跳转到调用回放界面进行查看
性能排行

apitop

  • 查看每天埋点的服务性能排行,记录平均响应时间、响应时间比例、调用次数、httpcode比例,可方便的发现线上接口性能情况
  • 可查看按响应时间排序的请求列表
依赖服务排行
  • 可查看同性能排行一样的数据指标,用于统计第三方API无埋点库的性能
  • 可查看不同时段的性能统计信息
SQL性能统计
  • 可查看所有SQL的性能情况,SQL自动去重合并统计
  • 可查看不同时段的SQL性能情况,SQL自动去重合并统计
线上故障去重
  • 线上所有通过埋点库产生的警报日志、异常日志、通知日志都会在这里汇总,去重
  • 可查看第一次产生故障调用回放以及最后一次调用回放
依赖开源声明
  • Alibaba Druid
  • Spring Boot
  • Facebook RocksDB
  • Lucene
  • Google Gson

联系我们

  • WeiboAD ADINF Team
  • QQ Group: 318051466
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

Fiery 是一款为PHP性能跟踪监控系统,可以方便的查看线上调用关系,性能,回放请求过程,参数,系统异常统计等 展开 收起
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Java
1
https://gitee.com/weiboad/fiery.git
git@gitee.com:weiboad/fiery.git
weiboad
fiery
fiery
master

搜索帮助