Sponsor of the Day:
Jerkmate
https://docs.vllm.ai/en/latest/api/vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w8a8_mxfp8/
compressed_tensors_moe_w8a8_mxfp8 - vLLM
compressed tensorsmoemxfp8vllm
https://docs.vllm.ai/en/latest/api/vllm/model_executor/layers/quantization/compressed_tensors/compressed_tensors_moe/compressed_tensors_moe_w4a8_fp8/
compressed_tensors_moe_w4a8_fp8 - vLLM
compressed tensorsmoefp8vllm
https://docs.vllm.ai/en/latest/api/vllm/model_executor/layers/quantization/compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4/
compressed_tensors_w4a4_nvfp4 - vLLM
compressed tensorsnvfp4vllm