1 Star 0 Fork 1

Ross / net_inspect

forked from Elinpf / net_inspect 
加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

net_inspect 介绍

net_inspect是网络设备数据结构化分析框架

安装方法

PyPI

pip install net_inspect

Poetry

git clone https://github.com/Elinpf/net_inspect
cd net_inspect
poetry install

插件介绍

net_inspect采用了插件模块的框架,分别是Input, Parse, Analysis, Output 模块。

通过编写模块插件,实现快速定制的能力。

InputPlugin

Input插件的功能是将已经获得的设备检查命令的日志转化为命令与命令输出的对应关系。如果是通过直接对设备进行操作获取的日志,可以使用console这个插件进行处理。如果是用的第三方平台进行的自动化收集,那么就需要自行编写Input插件实现命令与输出的对应。

ParsePlugin

Parse插件的功能是将每条命令的输出进行解析并且结构化。提取出关键信息并以List[dict]的方式进行存储。

现有的Parse解析模块使用的是ntc-templates-elinpf这个库,是ntc-templates库的分支,由于主仓更新频率很慢且不会增加国内常用的设备厂家,所以我Fork后进行了修改。

AnalysisPlugin

Analysis插件的功能是将解析的信息进行分析,对分析的内容进行告警分级。例如电源是否异常。这个工作是在分析模块中进行的。

OutputPlugin

输出模块可能是最需要自定义编写的地方。将解析和分析的结果按照自己想要的格式展现出来。

使用方法

net_inspect有三种使用方式

  1. 作为三方库提供API
  2. CLI命令行操作
  3. 本地Web界面操作 (TODO)

使用库

示例:

from net_inspect import NetInspect, OutputPluginAbstract, PluginError
from rich.table import Table
from rich.console import Console


class Output(OutputPluginAbstract):
    def main(self):
        if not self.args.output_params.get('company'):
            raise PluginError('args of `company` is missing')

        console = Console()

        table = Table(title=self.args.output_params.get(
            'company'), show_lines=False)
        table.add_column('name', justify='center')
        table.add_column('ip', justify='center')
        table.add_column('model', justify='center')
        table.add_column('version', justify='center')
        table.add_column('power', justify='center')
        table.row_styles = ['green']

        for device in self.args.devices:  # 一种方式是按照每台设备的对应情况进行单独设置
            if device.vendor.PLATFORM == 'huawei_vrp':
                data = [device.info.hostname, device.info.ip]
                ps = device.parse_result('display version')
                data.append(ps[0].get('model'))
                data.append(ps[0].get('vrp_version'))
                power_analysis = device.analysis_result.get('Power Status')
                power_desc = []
                for alarm in power_analysis:
                    if alarm.above_focus:
                        power_desc.append(alarm.message)
                data.append('\n'.join(power_desc) if power_desc else 'Normal')

                table.add_row(*data)

            else: # 还有一种方式是将每台设备的基础信息直接调取出来使用
                table.add_row(
                    device.info.hostname,
                    device.info.ip,
                    device.info.model,
                    device.info.version,
                    'Abnormal' if device.info.analysis.power else 'Normal'
                )

        console.print(table)


net = NetInspect()
# net.set_log_level('DEBUG')
net.set_plugins(input_plugin='console', output_plugin=Output) # 设置输入输出插件
cluster = net.run('warning_test', output_plugin_params={
                  'company': 'Company Name'})

API 解释

初始化

from net_inspect import NetInspect

net = NetInspect()

设置日志级别

net.verbose(1)

开启verbose表示为打开了DEBUG级别的日志, 默认为INFOverbose级别可以通过verbose()方法设置,总共0~3

  • 0: 日志关闭
  • 1: 提供Output模块的日志和Parse模块的日志
  • 2: 追加提供Analysis模块的日志
  • 3: 追加提供Parse模块不支持命令的信息日志和命令为无效的信息

设置插件

一般来说,只需要设置input_pluginoutput_plugin即可。

net.set_plugins(input_plugin='smartone', output_plugin='device_warning_logging')

可以使用字符串的简写,也可以自己继承插件类后重写main()方法,然后将类传递进来。

执行

提供输入的文件路径即可, 可以是文件或者目录。

net.run('log_files')

如果output中需要提供参数,可以使用output_plugin_params参数,例如:

net.run('log_files', output_plugin_params={'company': 'Company Name'})

获取设备的基本信息

已经提供了获取设备的基本信息的方法,可以通过NetInspect.get_base_info()方法获取。

例如:

net.run('log_files')
all_info = net.get_base_info()

for info in all_info:
    print(info.hostname)
    print(info.sn)

新增设备基本信息类

from net_inspect import NetInspect, EachVendorDeviceInfo, BaseInfo, Device
from rich.table import Table
from rich import print


class AppendClock(BaseInfo):
    clock: str = ''  # 巡检时间


class EachVendorWithClock(EachVendorDeviceInfo):

    base_info_class = AppendClock # 基本信息类

    def do_huawei_vrp_baseinfo_2(self, device: Device, info: AppendClock): 
        # 添加do_<vendor_platform>_baseinfo_<something>方法,可以自动运行
        with device.search_cmd('display clock') as cmd:
            if cmd.parse_result:
                row = cmd.parse_result[0]
                info.clock = f'{row["year"]}-{row["month"]}-{row["day"]} {row["time"]}'


net = NetInspect()
net.set_base_info_handler(EachVendorWithClock) # 设置获取设备基本信息的处理类
net.set_input_plugin('smartone') 
net.run_input('地市巡检')

net.set_input_plugin('console') # 可以多个输入插件同时使用
net.run_input('cisco')

net.run_parse()
net.run_analysis()

table = Table(title='设备信息')
table.add_column('设备名称', style='bold green')
table.add_column('设备型号', style='bold green')
table.add_column('CPU 利用率', style='bold green')
table.add_column('巡检时间', style='bold green')
for device in net.cluster.devices:
    info = device.info  # type: AppendClock
    row = [
        info.hostname,
        info.model,
        info.cpu_usage,
        info.clock # 这里的clock就是新增的
    ]
    table.add_row(*row)

print(table)

通过set_base_info_handler()方法设置获取设备基本信息的处理类,可以自己增加通用的基础信息,方便后面的调用。

调用外部的模板文件

net_inspect支持调用外部模板文件,可以通过NetInspect.set_external_templates(template_dir)api设置。

需要注意的是,调用的templates文件夹中必须包含index文件,并且文件开头必须为以下内容:

Template, Hostname, Platform, Command

设置示例:

net.set_external_templates('external_templates_dir')

CLI 命令行操作

net_inspect -i log_files

关于贡献

分析插件还在持续开发中,develop_script.py脚本就是为高效开发提供的一个工具。

开发一个分析插件的流程,以开发检查风扇状态的fan_status插件为例:

  1. 创建一个新的插件文件, 对应的文件初始状态会一并准备好
python ./develop_script.py -p fan_status -g
  1. 在对应的文件中实现插件对每个厂商分析的函数
class AnalysisPluginWithFanStatus(AnalysisPluginAbc):
    """
    要求设备所有在位风扇模块运行在正常状态。
    """
    @analysis.vendor(vendor.H3C)
    @analysis.template_key('hp_comware_display_fan.textfsm', ['slot', 'id', 'status'])
    def hp_comware(template: TemplateInfo, result: AnalysisResult):
        """模块状态不为Normal的时候告警"""
        for row in template['display fan']:
            if row['status'].lower() != 'normal':
                result.add_warning(
                    f'Slot {row["slot"]} Fan {row["id"]} 状态异常' if row['slot'] else f'Fan {row["id"]} 状态异常')

其中@analysis是用来记录插件的分析类型的,vendor记录插件的厂商类型,template_key记录分析模块所需要的textfsm文件以及里面的哪些值。 这些值会在参数template: TemplateInfo中给出。

result: AnalysisResult用来记录分析结果。可以添加告警信息。

分析方法为类方法,不需要self,不需要给出返回值。

插件中的类注释和方法注释都会被记录下来,方便后续调用。

  1. 创建对应的测试文件

当编写了对应的分析方法后,再次执行创建命令,工具会自动根据分析方法中需要的命令,生成对应的测试文件。

测试文件路径为tests/check_analysis_plugins/<plugin_name>/<funcation_name>.raw

python ./development_script.py -p fan_status -f hp_comware -g
  1. 在测试文件中添加测试用例
  2. 执行测试
python ./development_script.py -p fan_status -f hp_comware -t
  1. 完成测试,确认测试结果为正常后,生成yml文件作为参考文件。
python ./development_script.py -p fan_status -f hp_comware -y
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [Elinpf] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

网络设备数据结构化分析框架 展开 收起
Python
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
马建仓 AI 助手
尝试更多
代码解读
代码找茬
代码优化
Python
1
https://gitee.com/byross/net_inspect.git
git@gitee.com:byross/net_inspect.git
byross
net_inspect
net_inspect
master

搜索帮助

344bd9b3 5694891 D2dac590 5694891