minicpm5-1b-math-lora

Model Details

This repository contains a math-focused PEFT LoRA adapter based on [openbmb/MiniCPM5-1B](https://huggingface.co/ openbmb/MiniCPM5-1B).

• Base model: openbmb/MiniCPM5-1B
• Training mode: lora
• Published artifact: adapter
• Primary task: grade-school math reasoning
• Primary eval: GSM8K
• Guard eval: PIQA
• LoRA rank: 32
• LoRA alpha: 64
• NEFTune alpha: 5.0
• Max steps: 150

Intended Use

This model is intended for research and experimentation with math reasoning fine-tuning. It is not intended for high-stakes educational, financial, legal, medical, or safety-critical decisions.

Training Data

The adapter was trained on math reasoning data, including MetaMathQA and Hendrycks MATH-style algebra/geometry examples, depending on the notebook configuration used for this run.

Training Procedure

The run used lora fine-tuning with PEFT/LoRA-style adaptation where applicable. Training configuration and adapter metadata are included in adapter_enhancement_metadata.json.

Evaluation

Before/after evaluation was run with the same base model and PEFT disable_adapter() for the baseline.

Task	Role	N	Before	After	Delta
GSM8K	primary	30	0.133	0.200	+0.067
PIQA	guard	150	0.620	0.647	+0.027

Gate status: PASS.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "openbmb/MiniCPM5-1B"
adapter_id = "MSGEncrypted/minicpm5-1b-math-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model, trust_remote_code=True)
base = AutoModelForCausalLM.from_pretrained(
  base_model,
  torch_dtype="auto",
  device_map="auto",
  trust_remote_code=True,
)
model = PeftModel.from_pretrained(base, adapter_id)

prompt = "Solve: If there are 12 apples and 5 are eaten, how many remain?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

The model may produce incorrect reasoning, hallucinated steps, or invalid arithmetic. Evaluation results from small notebook samples should be treated as smoke tests, not full benchmark claims.

Artifacts

• eval_results.json: captured GSM8K/PIQA notebook results
• adapter_enhancement_metadata.json: training and publish metadata
• Adapter/model files produced by the notebook save step

Framework Versions

• PyTorch: 2.10.0+cu128