Mamba2 inference latency

Research question

I'm investigating whether an optimized Mamba2 model can improve inference speed, accuracy, or data efficiency relative to Transformer models. Prof. Jonathan Hersh at Chapman is supporting this work.

Planned benchmarks

For generative inference, I plan to measure time to first token and throughput against similarly sized Transformers. For financial text classification, I will compare latency and accuracy against FinBERT and ModernBERT, and examine how much training data each model needs.

Status

I'm testing optimization ideas. There is no validated end-to-end prototype, published benchmark, or paper yet.