Choose the Pruna framework or the hosted service
Pruna AI's open-source pruna package is a model-optimization framework. You load a model in Python, choose compatible optimizations in SmashConfig, and apply them with smash. Start from the official PrunaAI/pruna repository.
Pruna's documentation homepage separates that framework from its hosted performance-model endpoints. Installing the package does not provide a hosted inference account or API credits. If your task is trying image generation or editing in a browser, the existing Pruna AI Playground guide on Yix covers that workflow. This page concerns optimizing a model you run yourself.
We checked the documentation and source revision c622a7771e0abb51f7ce6e059424573078f3e576 on October 27, 2026. The package metadata and installation documentation identify Pruna 0.3.4. We did not install it, download model weights, run inference or benchmarks, or send paid requests. The code below is a documentation-based example, not a recorded result.
Check Python, operating system and algorithm requirements
The inspected package configuration requires Python 3.10 or later, but below 3.14. Its dependency floors include PyTorch 2.7.0 and torchvision 0.22.0. The README's broader “Python 3.9 or higher” text does not override those package constraints. Use the metadata for the version you are installing.
The installation guide identifies Linux as the officially supported platform. macOS and Windows availability depends on the selected algorithms and their dependencies. Installing the core package does not establish that every compiler, quantizer or GPU kernel works on your machine.
Before choosing a recipe, record the model family, Python version, PyTorch build, operating system and intended device. Check the algorithm's hardware and compatibility requirements. A diffusion-specific cacher is not a general optimization for every language or vision model.
Install into a separate environment
For readers who already use Conda, the official guide shows a Python 3.11 environment. A version-specific installation is:
conda create -n pruna python=3.11
conda activate pruna
python -m pip install "pruna==0.3.4"
Choose an unused environment name if pruna already exists. Keep the environment separate from a working application until you have checked its dependencies. The installation page also supplies a CPU-wheel-index variant and GPU guidance; choose the route for your actual hardware rather than copying a CUDA setup from an unrelated example.
Check the version from that same environment:
python -c "import pruna; print(pruna.__version__)"
The inspected package exports __version__. A successful import verifies that stage only. It does not load your chosen model, compile it or validate model output.
Install extra dependencies only for the algorithm you selected. For example, the documentation lists a stable-fast extra for StableFast and llamacpp for Llama.cpp conversion and inference. Those extras are different requirements; installing every extra is not the first response to an error.
A minimal CPU optimization workflow
The repository's CPU tutorial uses torchvision's ViT-B/16 model and the torch_compile algorithm with the inductor backend. The example below adapts that workflow by selecting CPU explicitly and using a synthetic tensor to show the call interface:
import torch
from torchvision.models import ViT_B_16_Weights, vit_b_16
from pruna import SmashConfig, smash
base_model = vit_b_16(weights=ViT_B_16_Weights.DEFAULT).eval()
config = SmashConfig(
{"torch_compile": {"backend": "inductor"}},
device="cpu",
)
optimized_model = smash(model=base_model, smash_config=config)
example_input = torch.zeros(1, 3, 224, 224)
with torch.inference_mode():
logits = optimized_model(example_input)
print(logits.shape)
On first use, loading those pretrained weights can require a download. Have sufficient memory and storage, and review the model's terms before using it. The zero-filled tensor is an interface example, not a meaningful image-classification evaluation. For real images, use the preprocessing required by the weights and a representative sample of your task.
The compiler implementation declares CPU and CUDA support, with inductor and openvino as its CPU-compatible backends. That does not make all backends interchangeable. This example deliberately selects one compiler rather than combining quantization, caching and compilation at once.
The first-model guide describes the broader sequence: load the model, define the configuration, apply smash, evaluate, then use the optimized model. A returned PrunaModel or a printed tensor shape is not evidence of improved quality, lower memory use or faster inference.
Preserve the baseline and understand saving
Keep the original checkpoint and configuration. Pruna's FAQ notes that some compilation algorithms modify a model in place, so the same object cannot automatically serve as an untouched comparison baseline. It also notes that the first inference may be slower because compilation or other initialization happens then.
For a real evaluation, choose acceptance criteria for your task before changing algorithms. Check output quality as well as resource use, with the same relevant inputs and settings. There is no universal speedup or compression percentage established by this guide.
The save-and-load guide documents optimized_model.save_pretrained(...) and PrunaModel.from_pretrained(...) for supported combinations. Use a new output directory rather than overwriting the original checkpoint. Serialization varies by algorithm, and some transformations must be reapplied during loading. A saved directory is not a promise that a compiled model will move unchanged between operating systems, devices or dependency versions.
Troubleshoot the failed stage
| Where it stops | What to check first |
|---|---|
| Package installation | Python is within the package's supported range; you are installing into the intended environment |
| Import | The interpreter matches the environment where pip installed Pruna; retain the first dependency error |
| Algorithm selection | Model type, device, required extras and compatible algorithm combinations |
| First inference | Distinguish initial compilation from later calls; retain the actual backend error if it fails |
| No improvement or changed output | Preserve the original baseline and compare the intended workload rather than a different model or input |
| Reloading | Saved configuration, required dependencies and whether the algorithm is reapplied on load |
Do not fix a CPU incompatibility by blindly adding CUDA packages, or resolve an installation conflict by changing a working global environment. Stop at the first clear blocker and consult the matching algorithm documentation. If you open an upstream issue, include the Pruna version, Python and PyTorch versions, platform, model identifier, configuration and a small non-private reproducer. Leave credentials and private weights out of the report.