Quantization - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Nov 10

llm-compressor for vLLM: The FP8/AWQ Deployment Tradeoff

#quantization #vllm #fp8 #awq

9 min read

Nov 8

2비트라는 이름의 GGUF가 3비트 이름의 파일보다 컸습니다: 공개 Darwin-180B 양자화 파일 6개를 헤더까지 열어 보니

#ai #machinelearning #llm #quantization

4 min read

GINIGEN AI

Nov 8

One Omni Model, Two GGUF Files: What LFM3-Omni-2B Really Costs in RAM, from the Tensor Headers Up

#ai #machinelearning #edgeai #quantization

12 min read

Aleksei Romanov for g factor

Nov 5

Quantization, Explained: From FP32 to INT4, and What Each Precision Costs You

#quantization #machinelearning #llm #ai

11 min read

mycute

Nov 9

We tried to build a 0.1-bit model. It worked on one task and failed on another.

#llm #machinelearning #quantization #opensource

13 min read

GINIGEN AI

Nov 2

A 763B MoE at "2 Bits": Where the Bytes Really Go in the GSQ-RCO GGUFs, and Why 100% Recovery Sits Inside the Noise

#ai #machinelearning #edgeai #quantization

15 min read

Huilin Ye

Oct 30

FP8 Cut Our TPOT by 25%. At High Concurrency, It Also Changed the Answers.

#llm #vllm #quantization #benchmarking

3 min read

amrit

Oct 26

Why T Quant is the Secret Weapon for Mobile AI in 2026

#diffusionllm #quantization #mobileai #ai

6 min read

GINIGEN AI

Oct 19

LFM2.5-24B-A2B on Your Own Hardware: The RAM, KV Cache and Bandwidth Budget

#ai #machinelearning #edgeai #quantization

11 min read

amrit

Oct 12

How One Startup Cut Diffusion‑LLM Inference Costs by 45 % – No Calibration Needed

#ai #quantization #diffusionllm

6 min read

Oct 12

Model Distillation & Quantization: Running 8B Parameter Models at a Fraction of Cloud Cost

#modeldistillation #quantization #vllm #awq

10 min read

xbill for Google Developer Experts

Oct 30

Four GPUs, One Quantization Ladder: Gemma 4 E2B From a 4 GB Laptop to an NVIDIA L4

#gemma #llamacpp #quantization #nvidia

11 min read

Oct 8

From 1.3 TB to 66 GB: On-Disk Vector Search in OpenSearch

#vectorsearch #airetrieval #opensearch #quantization

8 min read

Oct 6

Native Quantization: Let OpenSearch Service Compress Your Vectors

#vectorsearch #airetrieval #opensearch #quantization

6 min read

GINIGEN AI

Oct 6

4-bit GGUF Quality for MoE Models: Why Only 3B of 180B Params Fire, and How to Prove Parity

#ai #llm #quantization #edgeai

7 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.