Inside

Ripple-65B Preview

Eighty layers on one rail. The light running through them is the residual stream; the lifted layer is opened up so you can see what happens inside.

80 layers 5,120 wide 64.8B parameters
Layer 37Sliding-window attention
Sliding window Global

Drag to orbit, scroll to zoom, right-drag to pan. Arrow keys change layers, E opens and closes, space sends a token.