Hub documentation

Data Designer

Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Data Designer

Data Designer๋Š” LLM์„ ํ™œ์šฉํ•˜์—ฌ ๊ณ ํ’ˆ์งˆ ํ•ฉ์„ฑ ๋ฐ์ดํ„ฐ์…‹์„ ์ƒ์„ฑํ•˜๋Š” NVIDIA NeMo์˜ ํ”„๋ ˆ์ž„์›Œํฌ์ž…๋‹ˆ๋‹ค. ํ†ต๊ณ„ ์ƒ˜ํ”Œ๋Ÿฌ, LLM, ๋˜๋Š” ๊ธฐ์กด ์‹œ๋“œ ๋ฐ์ดํ„ฐ์…‹์„ ์‚ฌ์šฉํ•˜์—ฌ ๋‹ค์–‘ํ•œ ๋ฐ์ดํ„ฐ๋ฅผ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

ํ•„์ˆ˜ ์กฐ๊ฑด

pip install data-designer

Hub์—์„œ ์‹œ๋“œ๋กœ ์‚ฌ์šฉํ•  ๋ฐ์ดํ„ฐ์…‹ ๋‹ค์šด๋กœ๋“œ

HuggingFaceSeedSource๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ Hub์—์„œ ๋ฐ์ดํ„ฐ๋ฅผ ์ง์ ‘ ๋กœ๋“œํ•˜์—ฌ ์ƒ์„ฑ์šฉ ์‹œ๋“œ ๋ฐ์ดํ„ฐ๋กœ ์‚ฌ์šฉํ•˜์„ธ์š”.

import data_designer.config as dd
from data_designer.interface import DataDesigner

data_designer = DataDesigner()
config_builder = dd.DataDesignerConfigBuilder()

# Load seed data from HuggingFace
seed_source = dd.HuggingFaceSeedSource(
    path="datasets/gretelai/symptom_to_diagnosis/data/train.parquet",
    token="hf_...",  # Optional, for private datasets
)
config_builder.with_seed_dataset(seed_source)

# Reference seed columns in prompts
config_builder.add_column(
    dd.LLMTextColumnConfig(
        name="physician_notes",
        model_alias="openai-gpt-5",
        prompt="Write notes for a patient with {{ diagnosis }}. Symptoms: {{ patient_summary }}",
    )
)

preview = data_designer.preview(config_builder, num_records=5)

์ƒ์„ฑ๋œ ๋ฐ์ดํ„ฐ์…‹์„ Hub์— ํ‘ธ์‹œ

๋‚ด์žฅ๋œ push_to_hub ๋ฉ”์„œ๋“œ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ƒ์„ฑ๋œ ๋ฐ์ดํ„ฐ์…‹์„ Hub์— ์—…๋กœ๋“œํ•ฉ๋‹ˆ๋‹ค.

# Generate dataset
results = data_designer.create(config_builder, num_records=1000, dataset_name="my-dataset")

# Push to Hub
url = results.push_to_hub(
    repo_id="username/my-synthetic-dataset",
    description="Synthetic dataset generated with Data Designer.",
    tags=["medical", "notes"],
    private=False,
)

๋ฆฌ์†Œ์Šค

Update on GitHub