经过多次尝试,在kaggle 双T4训练Qwen2.5-0.5B的正确打开方式是:!torchrun --nproc_per_node2 \ -m swift.cli.sft \ --model ./qwen2.5-0.5b-instruct \ --dataset /kaggle/working/duan/tools/ai_copilot/sft_dataset.jsonl \ --max_length 4096 \ --num_train_epochs 3 \ --per_device_train_batch_size 4 \ --learning_rate 5e-5 \ --output_dir ./output_v2 \ --logging_steps 5 \ --save_steps 500 \ --eval_steps 500 \ --split_dataset_ratio 0.1 \ --bf16 true详细过程在kaggle上的最佳实践先安装库!pip install ms-swift[llm] -U -q !pip install torchao -U -q下载模型因为ms-swift自己下载模型太慢,用transformers下载from transformers import AutoModelForCausalLM, AutoTokenizer model_name Qwen/Qwen2.5-0.5B-Instruct save_dir ./qwen2.5-0.5b-instruct tokenizer AutoTokenizer.from_pretrained(model_name, trust_remote_codeTrue) tokenizer.save_pretrained(save_dir) model AutoModelForCausalLM.from_pretrained(model_name, trust_remote_codeTrue) model.save_pretrained(save_dir) print(f下载完成保存在 {save_dir})上传训练数据集我就偷懒了,直接下载段言项目的代码,里面自带数据集!git clone https://gitcode.com/skywalk163/duan/怎么偷懒呢? 直接让程序帮我们找到数据集的位置import os for root, dirs, files in os.walk(/kaggle): for f in files: if f sft_dataset.jsonl: print(os.path.join(root, f))这段代码会自动输出数据集的路径:/kaggle/working/duan/tools/ai_copilot/sft_dataset.jsonl开始训练一般我们都是用swift sft 开训,但是在kaggle上双T4卡训练会报错,所以要用torchrun启动:!torchrun --nproc_per_node2 \ -m swift.cli.sft \ --model ./qwen2.5-0.5b-instruct \ --dataset /kaggle/working/duan/tools/ai_copilot/sft_dataset.jsonl \ --max_length 4096 \ --num_train_epochs 3 \ --per_device_train_batch_size 4 \ --learning_rate 5e-5 \ --output_dir ./output_v2 \ --logging_steps 5 \ --save_steps 500 \ --eval_steps 500 \ --split_dataset_ratio 0.1 \ --bf16 true现在max_length 设为4096, train_batch_size 设为4也能正常训练,不爆显存 .以前用段言自带的训练脚本,max_length 设为2048, train_batch_size 设为1 才能不爆显存!训练完毕:Train: 100%|██████████████████████████████████| 201/201 [14:5800:00, 2.82s/it] {eval_loss: 0.2401, eval_runtime: 8.635, eval_samples_per_second: 13.55, eval_steps_per_second: 6.833, eval_token_acc: 0.9391, epoch: 3, global_step/max_steps: 201/201, elapsed_time: 15m 7s, remaining_time: 0s, memory(GiB): 13.83, train_speed(s/it): 4.514} Val: 100%|██████████████████████████████████████| 59/59 [00:0800:00, 7.01it/s] /usr/local/lib/python3.12/dist-packages/torch/distributed/c10d_logger.py:83: UserWarning: barrier(): using the device under current context. You can specify device_id in init_process_group to mute this warning. return func(*args, **kwargs) [INFO:swift] Saving model checkpoint to /kaggle/working/output_v2/v2-20260801-011216/checkpoint-201 {train_runtime: 908.2, train_samples_per_second: 3.495, train_steps_per_second: 0.221, train_loss: 0.3376, epoch: 3, global_step/max_steps: 201/201, elapsed_time: 15m 8s, remaining_time: 0s, memory(GiB): 13.83, train_speed(s/it): 4.518} Train: 100%|██████████████████████████████████| 201/201 [15:0800:00, 4.52s/it] [INFO:swift] last_model_checkpoint: /kaggle/working/output_v2/v2-20260801-011216/checkpoint-201 [INFO:swift] best_model_checkpoint: /kaggle/working/output_v2/v2-20260801-011216/checkpoint-201 [INFO:swift] images_dir: /kaggle/working/output_v2/v2-20260801-011216/images [INFO:swift] End time of running main: 2026-08-01 01:27:46.525213 [rank0]:[W801 01:27:47.297239747 ProcessGroupNCCL.cpp:1553] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())最优模型确认框架这里使用的是swift将checkpoint-201即最后一个epoch保存的检查点标记为最佳模型。再用6个epoch试试!有了当前提升准确率的感觉.训练完成:Val: 100%|██████████████████████████████████████| 59/59 [00:0800:00, 7.01it/s] /usr/local/lib/python3.12/dist-packages/torch/distributed/c10d_logger.py:83: UserWarning: barrier(): using the device under current context. You can specify device_id in init_process_group to mute this warning. return func(*args, **kwargs) [INFO:swift] Saving model checkpoint to /kaggle/working/output_v2/v3-20260801-013448/checkpoint-402 {train_runtime: 1786, train_samples_per_second: 3.554, train_steps_per_second: 0.225, train_loss: 0.2082, epoch: 6, global_step/max_steps: 402/402, elapsed_time: 29m 46s, remaining_time: 0s, memory(GiB): 13.83, train_speed(s/it): 4.443} Train: 100%|██████████████████████████████████| 402/402 [29:4600:00, 4.44s/it] [INFO:swift] last_model_checkpoint: /kaggle/working/output_v2/v3-20260801-013448/checkpoint-402 [INFO:swift] best_model_checkpoint: /kaggle/working/output_v2/v3-20260801-013448/checkpoint-402 [INFO:swift] images_dir: /kaggle/working/output_v2/v3-20260801-013448/images [INFO:swift] End time of running main: 2026-08-01 02:04:54.818971 [rank0]:[W801 02:04:55.541631968 ProcessGroupNCCL.cpp:1553] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see Redirecting… (function operator())最后测试下来,500步的效果最好!torchrun --nproc_per_node2 \ -m swift.cli.sft \ --model ./qwen2.5-0.5b-instruct \ --dataset /kaggle/working/duan/tools/ai_copilot/sft_dataset.jsonl \ --max_length 4096 \ --num_train_epochs 12 \ --per_device_train_batch_size 4 \ --learning_rate 5e-5 \ --output_dir ./output_v2 \ --logging_steps 5 \ --save_steps 500 \ --eval_steps 500 \ --split_dataset_ratio 0.1 \ --bf16 trueTrain: 100%|██████████████████████████████████| 804/804 [47:1000:00, 2.91s/it] {eval_loss: 0.2208, eval_runtime: 8.648, eval_samples_per_second: 13.53, eval_steps_per_second: 6.822, eval_token_acc: 0.952, epoch: 12, global_step/max_steps: 804/804, elapsed_time: 47m 19s, remaining_time: 0s, memory(GiB): 14.06, train_speed(s/it): 3.531} Val: 100%|██████████████████████████████████████| 59/59 [00:0800:00, 6.98it/s] /usr/local/lib/python3.12/dist-packages/torch/distributed/c10d_logger.py:83: UserWarning: barrier(): using the device under current context. You can specify device_id in init_process_group to mute this warning. return func(*args, **kwargs) [INFO:swift] Saving model checkpoint to /kaggle/working/output_v2/v7-20260801-042924/checkpoint-804 {train_runtime: 2840, train_samples_per_second: 4.47, train_steps_per_second: 0.283, train_loss: 0.1082, epoch: 12, global_step/max_steps: 804/804, elapsed_time: 47m 20s, remaining_time: 0s, memory(GiB): 14.06, train_speed(s/it): 3.532} Train: 100%|██████████████████████████████████| 804/804 [47:2000:00, 3.53s/it] [INFO:swift] last_model_checkpoint: /kaggle/working/output_v2/v7-20260801-042924/checkpoint-804 [INFO:swift] best_model_checkpoint: /kaggle/working/output_v2/v7-20260801-042924/checkpoint-500 [INFO:swift] images_dir: /kaggle/working/output_v2/v7-20260801-042924/images [INFO:swift] End time of running main: 2026-08-01 05:17:05.974298 [rank0]:[W801 05:17:07.697822352 ProcessGroupNCCL.cpp:1553] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())如果爆显存,就把batch_size减小即可.合并模型!swift merge-lora --adapters /kaggle/working/output_v2/v7-20260801-042924/checkpoint-500转为gguf这步我在kaggle上需要先编译安装llama.cpp,需要较长时间,且经常有报错.所以转gguf我都是在本地让Trae帮我转的.测试效果全部完成v7: 18/18 PASS (100%)比 v3 的 17/18 有明显提升。v3 vs v7 对比#测试用例v3v7v7 改进点1基础函数OKOK补全了参数b2多参数默认值OKOK加了段落参数更完整3嵌套条件OKOK加了段落前缀4for-elseOKOKfindtarget更准确5双重循环OKOKlen(mat)替代N变量名正确6列表推导OKOK用遍历...之...若语法7字典推导OKOK接近正确语法8类定义OKOK稳定9try-except-finallyOKOK去掉了设 path 为 空10类继承superOKOK多了属性 breed11lambdafiltermapOKOK用筛选遍历...之...若12match-caseOKOK-13海象运算符OKOK(设 n 为 len(data))正确14propertyOKOK特性 段落 area名称正确15复合赋值OKOK稳定16冒泡排序OKOK段落 bubble_sort完整17with语句FAILOK从失败变通过18装饰器OKOK结构更合理模型信息项目v3v7模型名duan-translator-v3duan-translator-v7训练轮数402500GGUF 大小948MB948MB通过率17/18 (94.4%)18/18 (100%)平均速度26.6 tok/s26.1 tok/sv7 核心改进列表推导从显式循环升级为[x 遍历 x 之 20至0如果 x 取余 2 等于 0]语法海象运算符正确输出(设 n 为 len(data))形式with 语句从直接失败变为可通过lambda 高阶函数开始使用筛选/映射关键字更少的冗余设 xxx 为 空声明先到这里吧,暂时训练告一段落,该想想这个东西怎么用了.