How to use from the
Use from the
Transformers library
# Gated model: Login with a HF token with gated access permission
hf auth login
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("feature-extraction", model="TuKoResearch/AuriStream100M_1Pred_BigAudioDataset_500k", trust_remote_code=True)
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("TuKoResearch/AuriStream100M_1Pred_BigAudioDataset_500k", trust_remote_code=True, device_map="auto")
Quick Links

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

AuriStream100M_1Pred_BigAudioDataset_500k

AuriStream is a speech language model by Greta Tuckute and Klemen Kotar.

This model predicts cochlear tokens from a tokenizer such as WavCochCausalV8192.

Trained 100M / one-prediction model, seed 11101994, at 500,000 completed AdamW steps. Global batch 256, float32 XLA execution without autocast or activation checkpointing, 16 data workers per TPU host, peak LR 3e-4 with 2,000 warmup steps and linear decay to zero at step 498,000. Run: https://wandb.ai/naudiogpt/auristream/runs/f532454c. This is a separate checkpoint from the original seed-1110 model. Weights retain float32 precision. Training used causal WavCoch codes. Optimizer state remains in the native training checkpoint; this inference export contains model weights only.

Model Details

Parameter Value
Parameters ~0.10B
Layers 12
Hidden Size 768
Attention Heads 12
Vocab Size 8192
Prediction Steps 1

Usage

from transformers import AutoModel, AutoConfig

# Load with trust_remote_code for custom model
model = AutoModel.from_pretrained(
    "TuKoResearch/AuriStream100M_1Pred_BigAudioDataset_500k",
    trust_remote_code=True,
)

# Or load config first
config = AutoConfig.from_pretrained("TuKoResearch/AuriStream100M_1Pred_BigAudioDataset_500k", trust_remote_code=True)

Base Model Code

This checkpoint uses shared model code from TuKoResearch/AuriStream-base.

Tokenizer

This model uses cochlear tokens from WavCochCausalV8192.

Seed and counterpart

Training seed: 11101994. Completed 500,000-step model. Matched random initialization: TuKoResearch/AuriStream100M_1Pred_BigAudioDataset_500k-randinit.

Downloads last month
25
Safetensors
Model size
97.5M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support