Skip to main content

compress_with_sinks

sqz_engine::kv_cache_optimizer

Function compress_with_sinks 

Source 
pub fn compress_with_sinks(content: &str, model_family: &ModelFamily) -> String
Expand description

Compress content while preserving attention sink tokens and provider cache boundaries.

§Arguments

  • content — the full text content to compress
  • model_family — the model family, used to determine cache boundary rules

§Returns

Compressed content string with critical tokens preserved.