kev-0.6b-merged: unofficial merged derivative of Kev

This is an unofficial derivative of jaredpalmer/kev-0.6b by Jared Palmer, prepared by Avartha. It is not endorsed by the Kev author. Kev and its weights are Apache-2.0, as are the Qwen3 (attention-only) base weights; see LICENSE (Kev) and LICENSE-QWEN (base).

Modifications: the Kev LoRA adapter is merged into the base weights; the pointer head is also provided as safetensors.

Sources (exact revisions)

	repo	revision
Kev release	jaredpalmer/kev-0.6b	dece6dba8d43f0f7ded45e9f5b9df12474d90843
Base	Qwen/Qwen3-0.6B-Base	da87bfb608c14b7cf20ba1ce41287e8de496c0cd
Kev code (encoder, head, merge rule)	github.com/jaredpalmer/kev@fe64b1274ea7f80d4095866df90666abb03e9cf6	Apache-2.0

What this repository is

• A Qwen3 (attention-only) backbone in bf16 with the Kev fine-tune already applied. Kev never uses the LM head.
• head.safetensors: the Kev pointer head (fp32). head.pt is the unchanged original, and kev_head.json describes the contract:

tensor	shape	dtype
q.weight	[256, 1024]	float32
q.bias	[256]	float32
k.weight	[256, 1024]	float32
k.bias	[256]	float32
temperature	[]	float32

• Readout: logit_j = ((W_k h_opt_j + b_k) . (W_q h_decide + b_q)) / sqrt(256) / T, then a softmax over the question's options. h is the backbone's final-norm hidden state.
• Temperature T: 1.0: head.pt carries no temperature, and kev-src's loader then applies 1.0.
• Token layout: <|fim_prefix|> state, <|fim_middle|> question, <|box_start|>/<|box_end|> option, <|fim_suffix|> decide. There is no chat template; each question is a causal row continuing the state.
• The tokenizer, config.json and preprocessor files come from the base at the pinned revision. That is what Kev's loader uses: it always loads the tokenizer from the base.

Procedure

Merged with upstream/kev_merge.py (sha256 260f41d81bb9901b6991b36869dabe0930611c360ab3c0c1a4d7eb7de9331eed), streaming one base shard at a time on CPU:

• For each LoRA-adapted Linear: W_bf16 = bf16( fp32(W) + (B @ A) * 2.0 ). Here 2.0 = lora_alpha / r = 32 / 16 comes from the adapter's adapter_config.json (plain LoRA: no DoRA, no rsLoRA, no rank/alpha patterns). This is exactly kev-src kev/checkpoint.py:296-312: an fp32 base, PEFT merge_and_unload, then a single cast to bf16.
• Adapter keys base_model.model.<module>.lora_{A,B}.weight map to base keys model.<module>.weight. The prefix was chosen as the only candidate under which every adapted module exists in the base index.
• 196 of 196 adapted modules were applied. The script fails if any adapted module is missing.
• Every other tensor is copied bit for bit, including tensors the base stores in fp32.
• Nothing dropped: the base has no MTP or vision tensors.
• Shard file names follow the base. Output: 310 tensors, 1,192,099,840 bytes.

Verification (CPU, before upload)

Run with upstream/kev_verify.py through kev-src's own encode(), rows_of(), DecisionModel.probs() and PointerHead. It used 5 short System One requests built with kev.api.to_record (token counts [48, 50, 43, 35, 64]). No GPU was used.

• Inventory vs base: 310 tensors. Names, shapes and dtypes equal the base's minus the 0 dropped tensors (ok=True).
• Bitwise vs Kev's own bf16 merge: all 310 backbone tensors of kev-src's PEFT fp32 merge, cast to bf16, equal this artifact bit for bit (ok=True). This covers all 196 adapted modules. 0 tensors are stored in fp32 and compared in fp32.
• Head conversion: head.safetensors tensors equal head.pt's (ok=True). The temperature equals the loader's, rounded to fp32 (ok=True).

arm vs Kev reference (fp32, adapter unmerged, head.pt)	hidden max abs	hidden max rel L2	probs max abs	argmax agree
fp32 merged (PEFT merge_and_unload) vs fp32 unmerged	0.00021	1.42e-05	2.92e-06	7/7
this artifact, bf16 weights, fp32 compute	0.663	0.0313	0.00504	7/7
this artifact, bf16 weights, bf16 compute (served form)	2.38	0.0783	0.00625	7/7

The bf16 drift is inherent to bf16 serving. Kev's own cards report served bf16 within 0.017 of fp32 for the 4B. Not measured here: accuracy on Kev's evaluation suites, and GPU end-to-end serving.

Files

file	bytes	sha256
LICENSE	11,343
LICENSE-QWEN	11,343
config.json	727
generation_config.json	138
head.pt	2,102,399	ce6cd9ffc54db41c179b65a33d60973dc8280c29e886218eb3ec07dc28d6f28b
head.safetensors	2,099,676	fafed82bab4cd26cf863be15c8ce8bf92451062e0e51c33cd0a5de65849ceaa8
kev_head.json	2,778
merge_manifest.json	9,590
merges.txt	1,671,853
model.safetensors	1,192,135,096	283ce6ee55ed651cb3c6b9e64c107076de263a7294785b10e66a037ab76a945e
tokenizer.json	7,031,645
tokenizer_config.json	9,678
upstream/adapter_config.json	1,185
upstream/kev_head.py	4,878
upstream/kev_merge.py	7,700
upstream/kev_model_card.md	6,439
upstream/kev_verify.py	10,381
upstream/provenance.json	2,848
upstream/training_config.json	1,047
upstream/training_metrics.json	323
verification.json	2,094
vocab.json	2,776,833
README.md	(this file)