Ripple-65B-Preview-NVFP4

Model Overview

• Model Architecture: RicursiveForCausalLM
  • Input: Text
  • Output: Text
• Model Optimizations:
  • Weight quantization: Mixed (FP8 / FP4)
  • Activation quantization: Mixed (FP8 / FP4)
• Release Date: 2026-11-06
• Version: 1.0
• Model Developers: RedHatAI

This model is a quantized version of ricursiveAI/Ripple-65B-Preview. It was evaluated on Terminal-Bench 2.1 to assess its quality in comparison to the original NVFP4 checkpoint, ricursiveAI/Ripple-65B-Preview-NVFP4.

Model Optimizations

This model was obtained by applying mixed-precision quantization to ricursiveAI/Ripple-65B-Preview, ready for inference with vLLM. Attention projections are quantized to FP8 (W8A8), while MLP projections are quantized to NVFP4 (W4A4).

NVFP4 stores weights and activations in 4-bit floating point with per-16-element block scaling and FP8 scale factors, using GPTQ with activation-importance weighting to preserve accuracy. The precision-sensitive attention projections, including the sigmoid gate, are kept at FP8 rather than NVFP4.

Compared with the original ricursiveAI/Ripple-65B-Preview-NVFP4, which was produced with NVIDIA ModelOpt and keeps all self-attention projections in BF16, this checkpoint also quantizes the attention projections to FP8. This makes the checkpoint smaller (~44 GB vs ~57 GB) while keeping the precision-sensitive attention stack above 4-bit.

This reduces the number of bits per parameter from 16 to 8 for attention projections and to 4 for MLP projections, lowering disk size and GPU memory requirements by approximately 66% relative to the BF16 model.

Only the weights and activations of the linear operators within transformer blocks are quantized using LLM Compressor.

Deployment

vLLM Serving

vllm serve RedHatAI/Ripple-65B-Preview-NVFP4 \
  --tensor-parallel-size 1 \
  --reasoning-parser ripple \
  --enable-auto-tool-choice \
  --tool-call-parser ripple

Inference Performance

In addition to reducing checkpoint size, this model improves serving throughput relative to the original NVFP4 checkpoint. On a single NVIDIA B200 with vLLM (TP=1), using 200 random prompts of 1,024 input and 256 output tokens at a maximum concurrency of 32 (request rate inf, ignore-eos, 4 warmups, seed 0), median output throughput across three runs was 1,527 tok/s for this checkpoint versus ~1,361 tok/s for ricursiveAI/Ripple-65B-Preview-NVFP4, an increase of ~12%. Median end-to-end latency was ~5.01 s versus ~5.62 s (11% lower). The comparison is between different quantization recipes, not a controlled ablation.

Creation

This model was created by applying LLM Compressor with calibration samples from the perfectblend dataset, as presented in the code snippet below. The model is exported in compressed-tensors format and can be loaded directly by vLLM.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

from llmcompressor import oneshot
from llmcompressor.modifiers.gptq import GPTQModifier
from llmcompressor.modifiers.quantization import QuantizationModifier

MODEL_ID = "ricursiveAI/Ripple-65B-Preview"
SAVE_DIR = "./Ripple-65B-Preview-NVFP4"

NUM_CALIBRATION_SAMPLES = 512
MAX_SEQUENCE_LENGTH = 2048

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    device_map=None,
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)

recipe = [
    QuantizationModifier(
        targets=[r"re:.*self_attn\..*"],
        scheme="FP8_DYNAMIC",
        ignore=["lm_head"],
    ),
    GPTQModifier(
        targets=[r"re:.*mlp\..*"],
        scheme="NVFP4",
        ignore=["lm_head"],
        weight_observer="nvfp4_expanded_imatrix",
        actorder="static",
        dampening_frac=0.01,
        block_size=128,
    ),
]

oneshot(
    model=model,
    tokenizer=tokenizer,
    dataset="perfectblend",
    splits="train[:512]",
    recipe=recipe,
    max_seq_length=MAX_SEQUENCE_LENGTH,
    num_calibration_samples=NUM_CALIBRATION_SAMPLES,
    trust_remote_code_model=True,
)

model.save_pretrained(SAVE_DIR, save_compressed=True)
tokenizer.save_pretrained(SAVE_DIR)

Evaluation

This model was evaluated on Terminal-Bench 2.1 using Inspect with the inspect_harbor/terminal_bench_2_1 task, served with vLLM (OpenAI-compatible API). Evaluations were run on 1x NVIDIA B200 per model.

Terminal-Bench 2.1 was run with Inspect's generic ReAct agent rather than the official Terminus 2 harness used on the public leaderboard, with a 7,200 s per-sample time limit and a 131,072-token context. Scores count all 89 tasks, with sample errors and time-limit hits counted as failures. These numbers are not directly comparable to Terminus 2 leaderboard entries or to the base model card's self-reported Terminal-Bench score.

Accuracy

Category	Benchmark	ricursiveAI/Ripple-65B-Preview-NVFP4	RedHatAI/Ripple-65B-Preview-NVFP4	Recovery
Agentic Coding	Terminal-Bench 2.1 (89 tasks, pass@1)	68.54	67.42	98.36%

Standard error: ±4.92 (original NVFP4), ±4.97 (this NVFP4+FP8 checkpoint). The 1.1-point difference is within the noise of a single 89-task pass@1 run.

Reproduction

The results were obtained using the following commands:

Terminal-Bench 2.1 (Inspect Harbor, 1 epoch, pass@1)

Start the vLLM server:

vllm serve RedHatAI/Ripple-65B-Preview-NVFP4 \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --reasoning-parser ripple \
  --enable-auto-tool-choice --tool-call-parser ripple \
  --override-generation-config '{"temperature": 1.0, "top_p": 0.95, "top_k": 64}'

Run the evaluation:

export VLLM_BASE_URL=http://localhost:8000/v1
export VLLM_API_KEY=local

inspect eval inspect_harbor/terminal_bench_2_1 \
  -T ref=sha256:7d7bdc1cbedad549fc1140404bd4dc45e5fd0ea7c4186773687d177ad3a0699a \
  -T override_cpus=0 \
  --model vllm/RedHatAI/Ripple-65B-Preview-NVFP4 \
  --epochs 1 --epochs-reducer mean \
  --temperature 1.0 --top-p 0.95 --top-k 64 \
  --max-tokens 32768 \
  --max-connections 5 \
  --max-sandboxes 5 \
  --time-limit 7200 \
  --fail-on-error 0.1 \
  -M client_timeout=7200 \
  --log-dir logs

Sampling follows the model's generation_config.json (temperature 1.0, top_p 0.95, top_k 64). Inspect's vLLM provider does not send top_k, so it is set server-side with --override-generation-config. The original NVFP4 baseline was evaluated with the same protocol using ricursiveAI/Ripple-65B-Preview-NVFP4.