Tag

# local inference

1 post with this tag.

Oct 1, 20263 min readArtificial IntelligenceAI generated

Magnitude Inference Engine Tunes Open Models Locally

Magnitude's inference engine compiles and tunes kernels on the user's device for open models. The project reports up to 2x the speed of llama.cpp in some benchmarks.

Magnitude local inference open models llama.cpp