Instructions to use leafspark/Llama-3.1-8B-MultiReflection-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use leafspark/Llama-3.1-8B-MultiReflection-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="leafspark/Llama-3.1-8B-MultiReflection-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("leafspark/Llama-3.1-8B-MultiReflection-Instruct")
model = AutoModelForCausalLM.from_pretrained("leafspark/Llama-3.1-8B-MultiReflection-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use leafspark/Llama-3.1-8B-MultiReflection-Instruct with PEFT:
```
Task type is invalid.
```
Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use leafspark/Llama-3.1-8B-MultiReflection-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "leafspark/Llama-3.1-8B-MultiReflection-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "leafspark/Llama-3.1-8B-MultiReflection-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/leafspark/Llama-3.1-8B-MultiReflection-Instruct

SGLang

How to use leafspark/Llama-3.1-8B-MultiReflection-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "leafspark/Llama-3.1-8B-MultiReflection-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "leafspark/Llama-3.1-8B-MultiReflection-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "leafspark/Llama-3.1-8B-MultiReflection-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "leafspark/Llama-3.1-8B-MultiReflection-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio new

How to use leafspark/Llama-3.1-8B-MultiReflection-Instruct with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for leafspark/Llama-3.1-8B-MultiReflection-Instruct to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for leafspark/Llama-3.1-8B-MultiReflection-Instruct to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for leafspark/Llama-3.1-8B-MultiReflection-Instruct to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="leafspark/Llama-3.1-8B-MultiReflection-Instruct",
    max_seq_length=2048,
)

Docker Model Runner
How to use leafspark/Llama-3.1-8B-MultiReflection-Instruct with Docker Model Runner:
```
docker model run hf.co/leafspark/Llama-3.1-8B-MultiReflection-Instruct
```

Llama-3.1-8B-MultiReflection-Instruct / README.md

leafspark

Adding Evaluation Results (#1)

a355780 verified over 1 year ago

preview code

raw

history blame contribute delete

14.3 kB

metadata

language:
  - en
  - de
  - fr
  - it
  - pt
  - hi
  - es
  - th
license: llama3.1
library_name: transformers
tags:
  - reflection
  - unsloth
  - peft
  - llama
base_model:
  - meta-llama/Meta-Llama-3.1-8B-Instruct
datasets:
  - leafspark/DetailedReflection-Claude-v3_5-Sonnet
metrics:
  - accuracy
pipeline_tag: text-generation
model-index:
  - name: Llama-3.1-8B-MultiReflection-Instruct
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: IFEval (0-Shot)
          type: HuggingFaceH4/ifeval
          args:
            num_few_shot: 0
        metrics:
          - type: inst_level_strict_acc and prompt_level_strict_acc
            value: 71.25
            name: strict accuracy
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=leafspark/Llama-3.1-8B-MultiReflection-Instruct
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: BBH (3-Shot)
          type: BBH
          args:
            num_few_shot: 3
        metrics:
          - type: acc_norm
            value: 28.45
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=leafspark/Llama-3.1-8B-MultiReflection-Instruct
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MATH Lvl 5 (4-Shot)
          type: hendrycks/competition_math
          args:
            num_few_shot: 4
        metrics:
          - type: exact_match
            value: 12.54
            name: exact match
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=leafspark/Llama-3.1-8B-MultiReflection-Instruct
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: GPQA (0-shot)
          type: Idavidrein/gpqa
          args:
            num_few_shot: 0
        metrics:
          - type: acc_norm
            value: 5.7
            name: acc_norm
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=leafspark/Llama-3.1-8B-MultiReflection-Instruct
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MuSR (0-shot)
          type: TAUR-Lab/MuSR
          args:
            num_few_shot: 0
        metrics:
          - type: acc_norm
            value: 8.52
            name: acc_norm
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=leafspark/Llama-3.1-8B-MultiReflection-Instruct
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MMLU-PRO (5-shot)
          type: TIGER-Lab/MMLU-Pro
          config: main
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 30.27
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=leafspark/Llama-3.1-8B-MultiReflection-Instruct
          name: Open LLM Leaderboard

Llama-3.1-8B-MultiReflection-Instruct

This model is inspired by OpenAI's o1 reasoning model. The dataset was synthetically generated using Claude 3.5 Sonnet.

Training

The training was done on Google Colab's free T4, using unsloth (duration: 52.32 minutes). The configuration is as follows:

LoRA Rank: 128
Packing: enabled
Batch size: 2
Gradient accumulation steps: 4
Epoches: 3
Steps: 30
Max sequence length: 4096

The training data comprised of 81 examples, each approximatly 3000 tokens.

Notes

It tends to produce very verbose and long reasoning responses
The coherency seems fairly good if you used the recommended sampling parameters
You should use at least 16k context, as each response is usually 2000-3000 tokens long

Recommended Sampling Parameters:

Temperature: 0.15
Min-P: 0.2
Top-K: 50
Top-P: 1
Max Tokens: uncapped
Frequency Penalty: 0.5
Presence Penalty: 0.1

Recommended System Prompt:

You are an advanced agentic assistant designed to complete tasks and provide general insight. Your responses will be formatted in XML. Start your responses with a <thinking> XML tag. You should write general process of how to complete the task provided by the user in this area. You may add nested tags, such as multiple <thinking> tags. <reflection> tags may also be used, if you've detected any inconsistencies or errors in your thinking. After completing a detailed thinking process, generate a draft response in <draft> tags. Review this response in <reflection> tags again, and refine your response as needed. Remember that you may (and are encouraged to) include multiple thinking, reflection and drafts. Provide your final response in <output> tags.

I initially thought the model was going to be medicore, but here's a sample output (quantized Q6_K, the fp16 is probably even better):

Can you teach me a cool math proof?

<thinking>
To create an engaging math proof, I should:
1. Choose a well-known and interesting proof
2. Break it down into simple steps
3. Ask the user questions to keep them engaged
4. Use LaTeX for math symbols
5. Provide explanations and context for each step
</thinking>

<draft>
Let's explore the fascinating world of math proofs together. I'd like to introduce you to the "Proof of the Pythagorean Theorem" - a classic and beautiful result in geometry.

Here's the theorem we'll be proving:

 $a^{2} + b^{2} = c^{2}$ 

where $a$ and $b$ are the lengths of the legs of a right triangle, and $c$ is the length of the hypotenuse.

To make this proof more engaging, I'll ask you questions along the way. Are you ready to get started?

1. Can you draw a simple right triangle with legs of length $a$ and $b$, and a hypotenuse of length $c$?
</draft>

<reflection>
The draft is a good start, but I should:
1. Provide more context about the proof
2. Explain why this theorem is important
3. Ask more questions to keep the user engaged
4. Use LaTeX for math symbols
</reflection>

<thinking>
Let me revise the draft to address these points:
1. I'll explain why the Pythagorean Theorem is important
2. I'll ask more questions to keep the user engaged
3. I'll use LaTeX for math symbols
4. I'll provide more context about the proof
</thinking>

<draft>
Let's embark on an exciting journey through the world of math proofs. Today, we'll explore the "Proof of the Pythagorean Theorem" - a fundamental result in geometry that has far-reaching implications in various fields, including physics, engineering, and architecture.

Here's the theorem we'll be proving:

 $a^{2} + b^{2} = c^{2}$ 

where $a$ and $b$ are the lengths of the legs of a right triangle, and $c$ is the length of the hypotenuse.

This theorem is crucial because it allows us to calculate the length of the hypotenuse (the longest side) of a right triangle, given the lengths of the other two sides. It's a powerful tool in many real-world applications, such as building design, bridge construction, and even computer graphics.

To make this proof more engaging, I'll ask you questions along the way. Are you ready to get started?

1. Can you draw a simple right triangle with legs of length $a$ and $b$, and a hypotenuse of length $c$?
2. What do you think might be a good way to approach this proof? Do you have any ideas?
</draft>

<reflection>
The revised draft is much better, but I should:
1. Provide more details about the proof
2. Explain the concept of similar triangles
3. Use more LaTeX for math symbols
4. Ask more questions to keep the user engaged
</reflection>

<thinking>
Let me revise the draft again to address these points:
1. I'll provide more details about the proof
2. I'll explain the concept of similar triangles
3. I'll use more LaTeX for math symbols
4. I'll ask more questions to keep the user engaged
</thinking>

<draft>
Let's dive into the fascinating world of math proofs. Today, we'll explore the "Proof of the Pythagorean Theorem" - a fundamental result in geometry that has far-reaching implications in various fields, including physics, engineering, and architecture.

Here's the theorem we'll be proving:

 $a^{2} + b^{2} = c^{2}$ 

where $a$ and $b$ are the lengths of the legs of a right triangle, and $c$ is the length of the hypotenuse.

This theorem is crucial because it allows us to calculate the length of the hypotenuse (the longest side) of a right triangle, given the lengths of the other two sides. It's a powerful tool in many real-world applications, such as building design, bridge construction, and even computer graphics.

To make this proof more engaging, I'll ask you questions along the way. Are you ready to get started?

1. Can you draw a simple right triangle with legs of length $a$ and $b$, and a hypotenuse of length $c$?
2. What do you think might be a good way to approach this proof? Do you have any ideas?

Let's start by considering two similar triangles. Similar triangles are triangles that have the same shape, but not necessarily the same size. In this case, we'll use the concept of similar triangles to relate the lengths of the sides of the right triangle.

Here's a diagram of the two similar triangles:

 $\begin{array}{ccc} \text{Triangle } ABC & \text{Triangle } ADE \\ a & \frac{a}{2} \\ b & \frac{b}{2} \\ c & \frac{c}{2} \\ \end{array}$ 

Notice that the two triangles are similar because they have the same angles (all right angles in this case). This means that their corresponding sides are proportional.

Now, let's use this similarity to relate the lengths of the sides of the two triangles. We can set up a proportion based on the corresponding sides:

 $\frac{a}{\frac{a}{2}} = \frac{b}{\frac{b}{2}} = \frac{c}{\frac{c}{2}}$ 

Simplifying this proportion, we get:

 $2a = \frac{a}{2}$ 

 $2b = \frac{b}{2}$ 

 $2c = \frac{c}{2}$ 

Now, let's square both sides of each equation:

 $4a^2 = \frac{a^2}{4}$ 

 $4b^2 = \frac{b^2}{4}$ 

 $4c^2 = \frac{c^2}{4}$ 

Multiplying both sides of each equation by 4, we get:

 $16 a^{2} = a^{2}$ 

 $16 b^{2} = b^{2}$ 

 $16 c^{2} = c^{2}$ 

Now, let's add the three equations together:

 $16 a^{2} + 16 b^{2} + 16 c^{2} = a^{2} + b^{2} + c^{2}$ 

Dividing both sides by 16, we get:

 $a^2 + b^2 + c^2 = \frac{1}{16}(a^2 + b^2 + c^2)$ 

This equation is true for any right triangle, so we can conclude that:

 $a^{2} + b^{2} = c^{2}$ 

This is the Pythagorean Theorem!

Now, it's your turn again. Can you think of any real-world applications of this theorem? Do you have any questions about the proof?
</draft>

<reflection>
The revised draft is excellent. It provides a clear and engaging explanation of the Pythagorean Theorem, along with a step-by-step proof. The use of LaTeX for math symbols is also well done. I'm confident that this will be a helpful and enjoyable experience for the user.
</reflection>

<output>
Let's embark on an exciting journey through the world of math proofs. Today, we'll explore the "Proof of the Pythagorean Theorem" - a fundamental result in geometry that has far-reaching implications in various fields, including physics, engineering, and architecture.

Here's the theorem we'll be proving:

 $a^{2} + b^{2} = c^{2}$ 

where $a$ and $b$ are the lengths of the legs of a right triangle, and $c$ is the length of the hypotenuse.

This theorem is crucial because it allows us to calculate the length of the hypotenuse (the longest side) of a right triangle, given the lengths of the other two sides. It's a powerful tool in many real-world applications, such as building design, bridge construction, and even computer graphics.

To make this proof more engaging, I'll ask you questions along the way. Are you ready to get started?

1. Can you draw a simple right triangle with legs of length $a$ and $b$, and a hypotenuse of length $c$?
2. What do you think might be a good way to approach this proof? Do you have any ideas?

Let's start by considering two similar triangles. Similar triangles are triangles that have the same shape, but not necessarily the same size. In this case, we'll use the concept of similar triangles to relate the lengths of the sides of the right triangle.

Here's a diagram of the two similar triangles:

 $\begin{array}{ccc} \text{Triangle } ABC & \text{Triangle } ADE \\ a & \frac{a}{2} \\ b & \frac{b}{2} \\ c & \frac{c}{2} \\ \end{array}$ 

Notice that the two triangles are similar because they have the same angles (all right angles in this case). This means that their corresponding sides are proportional.

Now, let's use this similarity to relate the lengths of the sides of the two triangles. We can set up a proportion based on the corresponding sides:

 $\frac{a}{\frac{a}{2}} = \frac{b}{\frac{b}{2}} = \frac{c}{\frac{c}{2}}$ 

Simplifying this proportion, we get:

 $2a = \frac{a}{2}$ 

 $2b = \frac{b}{2}$ 

 $2c = \frac{c}{2}$ 

Now, let's square both sides of each equation:

 $4a^2 = \frac{a^2}{4}$ 

 $4b^2 = \frac{b^2}{4}$ 

 $4c^2 = \frac{c^2}{4}$ 

Multiplying both sides of each equation by 4, we get:

 $16 a^{2} = a^{2}$ 

 $16 b^{2} = b^{2}$ 

 $16 c^{2} = c^{2}$ 

Now, let's add the three equations together:

 $16 a^{2} + 16 b^{2} + 16 c^{2} = a^{2} + b^{2} + c^{2}$ 

Dividing both sides by 16, we get:

 $a^2 + b^2 + c^2 = \frac{1}{16}(a^2 + b^2 + c^2)$ 

This equation is true for any right triangle, so we can conclude that:

 $a^{2} + b^{2} = c^{2}$ 

This is the Pythagorean Theorem!

Now, it's your turn again. Can you think of any real-world applications of this theorem? Do you have any questions about the proof?

Remember, this is a proof, not a mathematical exercise. Feel free to ask questions or share your thoughts about the theorem and its implications.
</output>

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	26.12
IFEval (0-Shot)	71.25
BBH (3-Shot)	28.45
MATH Lvl 5 (4-Shot)	12.54
GPQA (0-shot)	5.70
MuSR (0-shot)	8.52
MMLU-PRO (5-shot)	30.27