1 Star 0 Fork 540

sunjunjie / ModelLink

forked from Ascend / ModelLink 
加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
MIT

GitHub Documentation

简体中文 | English

ModelLink旨在为华为 昇腾芯片 上的大语言模型提供端到端的解决方案, 包含模型,算法,以及下游任务。

ModelLink解决方案概览


支持功能

当前ModelLink支撑大模型使用功能:

强化学习等特性持续研发中....

支持模型

当前ModelLink支持下列模型的预训练以及微调:

模型 参数 预训练 推理 LoRA SFT 对话 评估 贡献方
Aquila 7B pretrain generate -- -- -- eval 【昇腾贡献】
Aquila2 7B pretrain generate -- -- -- eval 【社区贡献】
34B pretrain generate -- -- -- eval 【社区贡献】
Baichuan 7B pretrain generate -- -- -- eval 【昇腾贡献】
13B pretrain generate -- -- -- eval 【昇腾贡献】
Baichuan2 7B pretrain generate -- -- -- eval 【昇腾贡献】
13B pretrain generate -- -- -- eval 【昇腾贡献】
Bloom 7B1 pretrain generate -- -- -- eval 【昇腾贡献】
176B pretrain generate -- -- -- eval 【昇腾贡献】
CodeLlama 34B pretrain generate -- -- -- eval 【社区贡献】
InternLM 7B pretrain generate -- -- -- eval 【昇腾贡献】
65B pretrain -- -- -- -- -- 【昇腾贡献】
LLaMA 7B pretrain generate lora -- -- eval 【昇腾贡献】
13B pretrain generate lora -- -- eval 【昇腾贡献】
33B pretrain generate lora -- -- eval 【昇腾贡献】
65B pretrain generate lora -- -- eval 【昇腾贡献】
LLaMA2 7B pretrain generate lora -- -- eval 【昇腾贡献】
13B pretrain generate lora -- -- eval 【昇腾贡献】
34B pretrain generate lora -- -- eval 【昇腾贡献】
70B pretrain generate lora -- -- eval 【昇腾贡献】
LLaMA3 8B pretrain generate -- -- chat eval 【昇腾贡献】
70B pretrain generate -- -- -- eval 【昇腾贡献】
Qwen 7B pretrain generate -- -- -- eval 【昇腾贡献】
14B pretrain generate -- -- -- eval 【昇腾贡献】
72B pretrain generate -- -- -- eval 【昇腾贡献】
Qwen1.5 7B pretrain generate -- -- -- eval 【社区贡献】
14B pretrain generate -- -- -- eval 【社区贡献】
Yi 34B pretrain generate -- -- -- eval 【社区贡献】
Mixtral 8x7B pretrain generate -- -- -- eval 【昇腾贡献】
Mistral 7B pretrain generate -- -- -- eval 【昇腾贡献】
Gemma 2B pretrain generate -- -- -- eval 【昇腾贡献】
7B pretrain generate lora -- -- eval 【昇腾贡献】
GPT3 175B pretrain -- -- -- -- -- 【社区贡献】

脚本命名规则

脚本 规则
pretrain_xxx.sh 预训练脚本
tune_xxx.sh LoRA脚本
generate_xxx.sh 推理脚本
xxxx_chat_xx.sh 对话脚本
evaluation_xxx.sh 评估脚本

模型版本与性能说明

上述列表中支持的模型,我们在examples文件夹中提供了各模型的训练脚本和readme说明,里面有详细的模型训练、推理、评估流程。

【需要注意模型使用时的配套环境版本,参考如下】

软件 版本
Python 3.8
driver Ascend HDK 23.0.0
firmware Ascend HDK 23.0.0
CANN CANN 7.0.0
torch 2.1.0、2.2.0
torch_npu release v5.0.0

【基于现版本我们实测的性能情况统计如下】

模型 参数 集群规模 精度模式 性能 参考性能
Aquila 7B 1x8 BF16 2849 2874
Aquila2 7B 1x8 FP16 3323 2673
34B 2x8 BF16 854 732
Baichuan 7B 1x8 FP16 2685 2036
13B 1x8 FP16 1213 862
Baichuan2 7B 1x8 BF16 2664 3969
13B 1x8 BF16 1668 2062
Bloom 7B1 1x8 FP16 2034 2525
176B 12x8 BF16 100 107
CodeLlama 34B 2x8 BF16 837 762
InternLM 7B 1x8 BF16 2776 2854
65B 4x8 BF16 341 414
LLaMA 7B 1x8 FP16 3600 3804
13B 1x8 FP16 1895 2012
33B 4x8 FP16 621 776
65B 4x8
BF16 348 426
LLaMA2 7B 1x8 BF16 4200 3850
13B 1x8 BF16 1990 1920
34B 2x8 BF16 690 796
70B 8x8 BF16 350 339
LLaMA3 8B 1x8 BF16 2483 2674
70B 8x8 BF16 283 355
Qwen 7B 1x8 BF16 2499 2867
14B 1x8 BF16 1560 1578
72B 16x8 BF16 285 345
Qwen1.5 7B 1x8 BF16 2862 2621
14B 1x8 BF16 1717 1702
Yi 34B 2x8 BF16 809 730
Mixtral 8x7B 2x8 BF16 487 610
Mistral 7B 1x8 BF16 2806 2734
Gemma 2B 1x8 BF16 6821 7602
7B 1x8 BF16 2938 2607
GPT3 175B 16x8 FP16 153 --

加速算法与融合算子

ModelLink支持张量并行、流水线并行、序列并行、重计算、分布式优化器等多种加速算法和融合算子,下表为各种加速特性对应的使能开关:

加速特性 使能参数
张量并行 --tensor-model-parallel-size
流水线并行 --pipeline-model-parallel-size
流水并行动态划分 --num-layer-list
序列并行 --sequence-parallel
重计算 --recompute-granularity
分布式优化器 --use-distributed-optimizer
DDP allreduce 掩盖 --overlap-grad-reduce
DDP allgather 掩盖 --overlap-param-gather
Flash attention --use-flash-attn
Fused rmsnorm --use-fused-rmsnorm
Fused swiglu --use-fused-swiglu
mc2 --use-mc2
Fused rotary
position embedding
--use-fused-rotary-pos-emb
Sliding window attention --sliding-window
torchrun $DISTRIBUTED_ARGS pretrain_gpt.py \
    --tensor-model-parallel-size ${TP} \
    --pipeline-model-parallel-size ${PP} \
    --num-layer-list 1,2,2,2,1 \
    --sequence-parallel \
    --recompute-granularity full \
    --recompute-method block \
    --recompute-num-layers 72 \
    --use-distributed-optimizer \
    --use-flash-attn \
    --use-fused-rmsnorm \
    --use-fused-swiglu \
    --overlap-grad-reduce \
    --overlap-param-gather \
    --use-fused-rotary-pos-emb \
    --use-mc2 \
    --sliding-window 4096 \
    ... \
    ...

基于昇腾芯片采集Profiling数据

Modellink支持基于昇腾芯片采集profiling数据,以提供对模型运行情况的分析,主要API如下:

--profile                        # 打开profiling采集数据开关
--profile-step-start  5          # 指定开启采集数据的步骤
--profile-step-end 6             # 指定结束采集数据的步骤,实际采集步数为 end-start,不包含end
--profile-ranks 0 1 2 3 4        # 指定采集数据的卡号,默认为0,可以设置为 0 1 2 3 4 5 6 7 8 9 列表指定全局卡号
--profile-level level2           # 数据采集水平,level0, 1, 2, 级别越高采集信息越多,默认为level0
--profile-with-cpu               # 是否采集CPU数据,加入参数采集
--profile-with-stack             # 采集指令运行堆栈,加入参数采集
--profile-with-memory            # 是否采集内存,加入参数采集
--profile-record-shapes          # 是否采集计算shape,加入参数采集
--profile-save-path ./profile_dir    # profiling数据采集保存路径

基于昇腾芯片的确定性计算功能

昇腾芯片默认采用了不确定计算加速模型训练,有时为了重复实验与对比实验需要确定性的计算结果,ModelLink使能确定性计算的开关如下:

  • 启动命令中加入开关
--use-deter-comp
  • 环境变量中加入开关
export HCCL_DETERMINISITIC=True

致谢


ModelLink由华为公司的下列部门联合贡献 :

  • 昇腾计算产品部
  • 计算算法部
  • 计算研究部
  • 开源计算工具部
  • 公共开发部
  • 全球技术服务部

感谢来自社区的每一个PR,欢迎贡献 ModelLink


安全声明

ModelLink安全声明

The following applies to all files unless otherwise noted: # Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved. # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in the # documentation and/or other materials provided with the distribution. # * Neither the name of NVIDIA CORPORATION nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY # EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR # PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR # CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, # EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, # PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR # PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY # OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -- This repository also contains code from Hugging Face Inc., Google Research, Facebook (from their Fairseq and Dino projects), Microsoft(from their Swin-Transformer project)and Philip Popien. Files from these organizations have notices at the top of each file. Below are licenses used in those files, as indicated. ------------- LICENSE FOR Facebook, huggingface and Google Research code -------------- Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work. To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright [yyyy] [name of copyright owner] Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ------------- LICENSE FOR Facebook Fairseq code -------------- MIT License Copyright (c) Facebook, Inc. and its affiliates. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

简介

昇腾大模型仓库 展开 收起
Python
MIT
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Python
1
https://gitee.com/sunjunjie1587/ModelLink.git
git@gitee.com:sunjunjie1587/ModelLink.git
sunjunjie1587
ModelLink
ModelLink
master

搜索帮助