1 Star 4 Fork 0

LoopStack / AMC-自动化Mock组件

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

AMC-自动化Mock组件

介绍

AMC(Automated Mock Component)自动化Mock组件:对HTTP请求进行Mock、拦截超时、指定返回结果,方便研发、QA进行测试和回归

什么是自动化Mock组件:AMC(Automated Mock Component)

  • 在我们的项目开发的过程中,对接外部系统,经常会请求其接口或者服务。在很多情况下,我们在开发完毕的时候需要对业务代码进行自测、联调
  • 但是会出现一种情况,需要造数据,来测试各种场景的业务逻辑是否可以正常执行。但是真实的情况下,很难快速的模拟各种正常、异常数据
  • 来验证系统的稳定性,甚至在QA同学在进行测试的时候,都会使用一些网络工具或者Mock工具进行Mock,经常会在Mock上耗费非常多的时间。
  • 现在我来举例经常出现的一些问题,比如某个接口返回100种错误码,但是这些错误码非常难Mock,因此可能需要下很多种场景
  • 甚至拦截或者是在代码中进行处理,因此会有风险,也有隐患。因此如果能够实现无侵入的方式,那么则是友好的。但是无侵入的方式很难,因为对接的外部接口的方式不是同一的标准,因此,做不同的适配是不合适的
  • 服务的接入方式:提供Mock-Cloud的方式进行数据交互

什么是Mock-Cloud

  • Mock-Cloud,是AMC的一种服务提供方式。在上文中,我们提到,依赖方可能不想有对Jar的依赖,因此只需要提供对外暴露的HTTP接口即可,通过对AMC的封装,可以实现任意场景的Mock
  • 并且,组件会对Mock的进行实现,记录"案发现场",实际上,针对这些重复的调用,每个会生成唯一的记录,这样即可进行数据的溯源,也可以复用执行请求

功能设计

  • 能够支持客户端和HTTP服务端通信的方式
  • 能够根据项目进行划分,区分不同项目的执行数据
  • 能够对历史的执行请求进行记录,请求和返回,以及业务执行日志
  • 可以Mock指定的数据,系统架构中设计指定的请求执行返回结果
  • 定义执行契约,对数据进行处理
  • 页面UI设计

系统架构设计

  • 因为Mock是一个测试系统,所以我们这里使用简洁的架构方案,使用DDK进行通知,来完成更新的功能

amc

设计要点

  • 表结构设计,分为配置数据和日志数据;目前配置数据都写在MongoDB
  • 分布式雪花ID,基于Hutool的工具实现
  • 此AMC和Mock Cloud的设计,基于AP模型。只要有一个服务端可用,那么整体就是可用的
  • 在实际的业务开发中,可能这种通知不仅仅在项目中实现,也会在业务方定制的SDK中使用,那么这种情况,就需要考虑同一个项目使用多个SDK的问题,是否需要限制多个SDK的启动?但是实际上项目在启动的时候,所有配置都是基于SpringBean实现的
  • 对于通知配置,Admin提供了通知接口。在请求达到的时候,会进行发送处理,其实就是发布,发布之后会通过DDK进行通知,然后调用更新的方法,进行更新数据,因此AMC-Cloud依赖于DDK
  • 对Mock-Cloud进行开发,提供Facade接口的方式进行提供服务,并且记录埋点
  • 总体上来说,系统分为3个实现,第一个是Admin配置站点,第二个是Facade服务站点;还有一个测试项目,作为测试使用
  • 但是实际上,无需设计的这么复杂,对于Mock的场景来说,对于数据的更新,我们可以通过Core提供接口来进行刷新数据,但是存在一种问题,
  • 也就是在集群的情况下,接口的请求并不是平均分配,因此此种方式不能保证集群的每一台机器都刷新本地的缓存数据。
  • 当然了,这种方式我们可以通过分布式注册中心,进行通知,但是本质上这种服务我并不想设计的太过复杂,也就是说不需要依赖太多的组件,简单易用就可以,
  • 因此一种简单的设计方式是,写更新数据到通知表,然后各个Facade服务,各自轮询这个数据,进行数据的更新,但是这种方式有一种问题,那就是如果判断数据是最新的数据,
  • 并轮询也会消耗CPU的部分性能,尽管当代计算机这种消耗已经可以忽略不计,因此,我们还是和之前一样,通过手动实现注册中心的方式进行消息通知,
  • 后期会将注册中心进行单独设计,将其抽象出来
  • 但是之前做的DDK,已经可以完成通知的功能,因此,此次将使用DDK去完成通知功能,而不用在手写注册中心那哪些套路了

技术与架构选型

  • 存储技术选型:MongoDB
  • 开发框架:SpringBoot 2.2.2.RELEASE、Vue2.0、Netty4.x、Vue-Template-Admin
  • 数据请求:Hutool HttpClient;超时与重试,基于线程池和FutureTask实现

AMC&Mock-Cloud要素

事业群

  • 事业群是最高平级维度,一个公司可能有多个事业部

项目组

  • 一个事业群下有多个项目组

项目

  • 一个项目组有多个项目:项目的标识是唯一的

项目MockTest配置

  • 对于Mock来说,会写一些执行脚本,因此,我们需要将一些常用的或者复杂的配置数据先行保存,方便后面在进行测试的时候进行处理,因此有这个前置数据模块

项目Mock配置

  • 在一个项目中有多个Mock场景配置,我们可以将其进行分类,然后编写对应的配置,然后定义契约即可进行访问

项目Mock回调配置

  • 在某些情况下,我们需要进行mock回调,也就是需要处理接入系统被调用的情况,因此会有这个Mock回调功能,但是第一个版本暂时不做处理

Mock报告

  • 对于研发和QA来说,我们实现的功能是能够记录到案发现场,因此我们需要做一些记录和埋点,记录请求参数和返回参数,然后通过文件导出,
  • 并尝试借助这种方式来补充我们的提测报告,Mock报告的导出方式分别为markdown、pdf、excel文档的形式,设计成markdown是为了能够对文档进行修改,导出为pdf则是直接使用,但是第一版本的实现,只实现了excel的导出

数据表设计

  • 数据表设计,根据动态字典键&注册中心要素可以发现聚合了5张表,有些数据是固化的,每5张表数据都有的,如下表,因此这些字段不再单独列出,具体的可在存储的数据结构中看到
字段名称 类型 备注
id Object MongoDB数据库自增id
uuid String 雪花算法随机UUID
desc String 功能描述
createTime Date 创建时间
updateTime Date 更新时间
isDelete int 状态 1删除,0未删除
env String 环境

事业群

  • 表名字:amc-group
字段名称 类型 备注
groupId String 事业群组id,唯一
groupName String 组名字

项目组

  • 表名字:amc-team
字段名称 类型 备注
teamId String 项目组id,唯一
teamName String 项目组名字
groupUuid String 事业群关联uuid

项目组

  • 表名字:amc-app
字段名称 类型 备注
appId String 项目id,全局唯一
appName String 项目名字
teamUuid String 项目组关联uuid

项目MockTest配置

  • 表名字:amc-app-mock-call-test
字段名称 类型 备注
mockTestName String 配置Test名称
testJson String 执行的测试内容
appUuid String 项目关联uuid

项目Mock配置

  • 表名字:amc-app-mock-call
字段名称 类型 备注
mockName String 配置名称
scriptType String 脚本执行类型
script String 执行的脚本内容
appUuid String 项目关联uuid

项目Mock回调配置

  • 表名字:amc-app-mock-callback
  • 一期版本没有

Mock报告

  • 记录触发条件、执行结果的现场数据,为Mock执行的结果进行数据聚合,规整成为标准文档
  • 表名字:amc-app-mock-call-report
字段名称 类型 备注
success boolean mock过程是否成功
mockName String mockName
mockUuid String mockUuid
mockRequest String mock请求
runtime long mock调用耗时
mockErrorMessage String mock调用异常
mockResponse String mock调用返回结果

AMC-Cloud消息更新

  • 消息更新使用DDK进行触发配置,AMC-Cloud会将自己注册到DDK,然后通过发布将数据请求到DDK,由DDK接管数据的更新功能
  • 具体的yml配置如下
# 接入方需要上报自己的信息给注册中心
ddk:
  env: test
  load: true
  client-port: 12000
  server-ips: 127.0.0.1
  server-port: 9998
  app-id: lsi.bs.java.amc
  • 具体的通知配置资源如下,AmcMockCallRepository 内部逻辑会判断是否是第一次加载,如果是则全量加载,然后加载配置通知值;否则就只加载配置
@Service @DdkResource("#mock.cloud.notice") public class DDKResource {

    private final Logger logger = LoggerFactory.getLogger(DDKResource.class);

    @Resource private AmcMockCallRepository amcMockCallRepository;

    @DdkBeforeCall private synchronized void ddkBeforeCall(String mockCallUuid) {
        // No Op
    }

    @DdkCall private synchronized void ddkCall(String mockCallUuid) {
        logger.info("[DDKResource][ddkCall] mockCallUuid:{}, now is Start", mockCallUuid);
        amcMockCallRepository.refresh(mockCallUuid);
        logger.info("[DDKResource][ddkCall] mockCallUuid:{}, now is End", mockCallUuid);
    }

    @DdkAfterCall private synchronized void ddkAfterCall(String mockCallUuid) {
        // No Op
    }
}

页面展示

事业群

amc amc

项目组

amc amc

项目

amc amc

Mock配置

amc amc

MockCall配置

amc amc

MockCall报告

amc amc

Mock-Cloud在DDK的注册

amc

接入方式

  • 启动DDK,动态字典键和配置中心
  • 启动AMC-Admin,进行相关的配置
  • 启动AMC-Cloud,加载数据到缓存
  • 启动接入项目,接入项目只需要引入依赖模型包即可,如下
<!-- lsi -->
<lsi.version>0.0.0.1-SNAPSHOT</lsi.version>
<!-- amc -->
<amc.version>1.0-SNAPSHOT</amc.version>

<dependency>
    <groupId>cn.icanci.loopstack</groupId>
    <artifactId>lsi-api</artifactId>
    <version>${lsi.version}</version>
</dependency>
<dependency>
    <groupId>cn.icanci.loopstack.amc</groupId>
    <artifactId>cloud-common</artifactId>
    <version>${amc.version}</version>
</dependency>
  • 数据请求测试如下,在cloud-common包中提供WrapperUtils ,可将Wrapper对象解析为各种类型,具体的转换参见WrapperUtils即可
@Service
public class MockCallServiceImpl implements MockCallService {
    private static final Logger logger = LoggerFactory.getLogger(MockCallServiceImpl.class);

    @Resource
    private Client              httpClient;

    @Value("${amc.url}")
    private String              amcUrl;

    @Override
    public void testMockCall(String mockUuid) {
        MockCallRequest request = new MockCallRequest();
        request.setTraceId(UUID.randomUUID().toString());
        request.setMockCallUuid(mockUuid);
        HashMap<Object, Object> obj = new HashMap<>();
        obj.put("uuid", UUID.randomUUID().toString());
        request.setMockRequest(JSONUtil.toJsonStr(obj));
        Client.RpcRequest rpcRequest = new Client.RpcRequest(amcUrl, request, Maps.newHashMap(), Method.POST, 3, TimeUnit.SECONDS, 0);
        logger.info("[MockCallService][testMockCall] req:{}", JSONUtil.toJsonStr(request));
        MockCallResponse call = httpClient.call(rpcRequest, MockCallResponse.class);
        //        MockCallWrapper wrapper = call.getWrapper();
        //        Object mockResponse = wrapper.getMockResponse();
        //
        //        System.out.println(WrapperUtils.toStringValue(mockResponse));
        //        Quota x = WrapperUtils.toObjectBean(mockResponse, Quota.class);
        //        System.out.println(x);

        logger.info("[MockCallService][testMockCall] resp:{}", JSONUtil.toJsonStr(call.getWrapper()));
        System.out.println();
    }
}
  • 有的童鞋可能对为什么要使用DDK进行通知,原因是因为AMC-Cloud可能是集群部署的,因此,在集群部署情况下,需要进行分布式的通知,因此需要使用DDK进行消息通知

迭代版本

  • 第一个版本完成了配置的更新、和DDK的整合、Mock报告的现场还原,完成了对数据的管理等

项目地址

备注

  • 前端开发如果提示有报错,请禁用:ESLint
  • npm 版本过高会有打包问题
  • mvn clean deploy
  • mvn versions:set -DnewVersion=0.0.0.1
  • mvn versions:revert
  • mvn versions:commit

ISSUE

  • mongodb数据库导出:mongodump -d amc -o /Users/icanci/Desktop
  • 测试mongo的数据文档参见 doc/amc
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

AMC(Automated Mock Component)自动化Mock组件:对HTTP请求进行Mock、拦截超时、指定返回结果,方便研发、QA进行测试和回归 展开 收起
Apache-2.0
取消

贡献者

全部

近期动态

加载更多
不能加载更多了
Java
1
https://gitee.com/loopstack/amc.git
git@gitee.com:loopstack/amc.git
loopstack
amc
AMC-自动化Mock组件
master

搜索帮助