Describe the current behavior / 问题描述 (Mandatory / 必填)

[ST][MS][MF][glm3-6b-32k]网络KBK推理性能劣化，低于竞品的0.5倍
模型仓地址：https://gitee.com/mindspore/mindformers/blob/dev/docs/model_cards/glm3.md

Environment / 环境信息 (Mandatory / 必填)

• Hardware Environment(Ascend/GPU/CPU) / 硬件环境:

Please delete the backend not involved / 请删除不涉及的后端:
/device ascend

• Software Environment / 软件环境 (Mandatory / 必填):
  -- MindSpore version (e.g., 1.7.0.Bxxx) :
  -- Python version (e.g., Python 3.7.5) :
  -- OS platform and distribution (e.g., Linux Ubuntu 16.04):
  -- GCC/Compiler version (if compiled from source):

【CANN版本】：MILAN-Florence-ASL/ABL Milan-ASL V100R001C18B512 Alpha
【MindSpore版本】：MindSpore_master_1286b531(MindSpore2.3.0B522)
【MindFormers版本】：MindFormers_dev_babf7ed1(MindFormers2.3.0B522)

• Excute Mode / 执行模式 (Mandatory / 必填)(PyNative/Graph):

Please delete the mode not involved / 请删除不涉及的模式:
/mode pynative
/mode graph

Related testcase / 关联用例 (Mandatory / 必填)

用例仓地址：/MindFormers_Test/cases/llama2/7b/train/
用例：
test_mf_chatglm3_6b_32k_kbk_infer_batch_incremental_1p_0001
test_mf_chatglm3_6b_32k_kbk_infer_incremental_1p_0001

Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)

1. get code from mindformers

2. cd mindformers/scripts

3. python glm3_6b_32k_generate.py --tokenizer_path /home/workspace/large_model_ckpt//glm3_32k/tokenizer.model --checkpoint_path /home/workspace/large_model_ckpt//glm3_32k/glm32k.ckpt --yaml_file /data/jenkins_workspace/TDT_deployment/MindFormers_Test/cases/glm3/32k/infer/test_mf_chatglm3_6b_32k_kbk_infer_incremental_1p_0001/research/glm32k/predict_glm.yaml --seq_length 256 --batch_size 1 --block_size 16 --num_blocks 32 > /data/jenkins_workspace/TDT_deployment/MindFormers_Test/cases/glm3/32k/infer/test_mf_chatglm3_6b_32k_kbk_infer_incremental_1p_0001/glm3_6b_incre_infer.log 2>&1 &

4. 验证网络是否推理成功

Describe the expected behavior / 预期结果 (Mandatory / 必填)

网络推理成功

Related log / screenshot / 日志 / 截图 (Mandatory / 必填)

bs=1，seq-length=256 ： 46.3 < 56.04554375
bs=1，seq-length=512 ： 43.82 < 55.3338401
bs=1，seq-length=1024 ： 44.23 < 53.00737235
bs=1，seq-length=2048 ： 43.97 < 49.80754538
bs=8，seq-length=256 ： 351.92 < 398.4384892

Special notes for this issue/备注 (Optional / 选填)

走给吴致远