Building the future of adaptive intelligence.

Read the research Talk to us
Scroll to explore

Built on the open research stack the field runs on

PyTorch NVIDIA A100 Hugging Face FlashAttention YaRN Zenodo DeepSpeed
Who we are

We're redefining how AI thinks.

SoLS AI is a research lab building the next generation of foundation models that think adaptively — allocating computation dynamically based on the difficulty of each token, instead of wasting fixed compute on every input.

Our flagship architecture, RecurrMoE, unifies temporal compute allocation (recursion) and spatial parameter routing (experts) into one hierarchical system — achieving what static transformers can't: intelligent, token-level compute budgeting.

  • ✓ Same quality at a fraction of the energy cost
  • ✓ Strict training–inference mathematical parity
  • ✓ Open, reproducible, peer-reviewable research
SoLS AI neural architecture visualization
Green AI Efficiency by design
Core technology

Six breakthroughs behind RecurrMoE

One architecture that addresses the fundamental bottlenecks of adaptive computation.

01
🔀

Hierarchical Dual-Level Routing

Separates local representation refinement (Inner MoR depth loops) from global token exiting (Outer MoR stage gates) for orthogonal optimization of depth and stage budget.

02
🧠

Embedded Sparse MoE

A Top-2 SwiGLU expert network inside the recursive block routes tokens through time and space — yielding 36 virtual expert paths from just 4 physical experts.

03
🗺️

Virtual-Layer KV Mapping

Resolves the recurrent KV-cache write-overwrite bottleneck with a unified virtual index V = p · d_max + d, maintaining strict training-inference parity.

04
🛡️

Exit-Stage Validity Masking

Prevents out-of-distribution logit corruption by applying dynamic validity masks that isolate early-exited tokens from deep attention gathers.

05
📐

Progressive YaRN Context Ramp

Scales the effective context window from 2,048 to 8,192 tokens via a three-phase curriculum with NTK-by-parts rotary interpolation.

06
⚡

93.7% Memory Reduction

Eliminates the quadratic O((T·D)²) overhead of 2D parallel masking, restoring standard O(T²) attention through autoregressive virtual-layer mapping.

AI in action

Adaptive compute, everywhere it matters

From research notebooks to production inference, RecurrMoE spends thinking time only where the work is hard.

A person interacting with an AI assistant on a laptop

Conversational reasoning

Hard prompts route through deeper recursion; easy tokens exit early — lower latency, the same answer quality.

Engineers reviewing model analytics on screens

Efficient inference at scale

Serve more requests per GPU with token-level early exit and MGQA cache savings.

Researcher analyzing data for AI experiments

Reproducible research

Open methodology, published benchmarks, and a fully documented training curriculum.

System design

The RecurrMoE architecture

A three-level nested adaptive computation framework: Outer MoR for stage routing, Inner MoR for recursive refinement, and MoR-based MoE for expert specialization.

RecurrMoE hierarchical computation graph

Three nested levels — Outer MoR (stage-level early-exit routing), Inner MoR (adaptive recursive refinement), and MoR-based MoE (sparse top-k expert selection per transformer block).

Performance

Real savings, measured

RecurrMoE-1B delivers genuine compute reduction in deployment while staying competitive on standard benchmarks.

0%
Compute reduction
Actual realized savings
0%
Memory reduction
vs. 2D parallel masking
0%
KV cache bandwidth saved
via MGQA 4:1 grouping
0
Virtual expert paths
From 4 physical experts

Benchmark performance (deployment config)

Benchmark Accuracy
MMLU24.83%
ARC21.08%
HellaSwag25.90%
WinoGrande49.41%
TruthfulQA8.57%
Average25.96%

Efficiency metrics

Total parameters
1.03 Billion
Mean exit stage
5.01 / 9 stages
Effective depth
10.0 / 18 layers
Inference mode
Token-level early exit
Built for

Who adaptive computation is for

Efficient foundation models unlock new economics for everyone shipping or studying AI.

Product team collaborating around AI features
Product teams

Ship AI features for less

Cut inference cost per request without sacrificing answer quality — make AI economics work at scale.

Researcher working with machine learning models
Researchers

Reproduce & extend

Open architecture, published curriculum, and clear benchmarks make RecurrMoE a foundation to build on.

Enterprise team in a meeting discussing AI strategy
Enterprises

Sustainable AI at scale

Lower energy footprint and GPU spend for the same workloads — a credible path to greener AI operations.

Voices

What the community is saying

"Adaptive depth is the right bet for efficient inference. RecurrMoE's virtual-layer mapping is an elegant solution to the recurrent KV-cache problem."
AR
ML Systems Researcher
Independent reviewer
"A 44% compute reduction at parity quality changes the cost equation for anyone serving models. This is the kind of efficiency work the field needs."
PE
Platform Engineer
AI infrastructure
"Clear methodology and honest benchmarks. Open research like this is exactly how green AI moves from slogan to practice."
DS
Data Scientist
Sustainable computing
Publication

RecurrMoE: Hierarchical Adaptive Computation via Nested Mixture-of-Recurrence and Sparse Expert Integration

Preprint Open Access v1.0 · June 2026

Authors: Bala Murugan, Deepak, Daniel Joe, Sudalai, Muthu Siva

A 1B-parameter architecture featuring hierarchical dual-level routing, embedded sparse MoE, virtual-layer KV mapping, and entropy-regularized exit masking. Pretrained on 12B tokens across a three-phase curriculum.

DOI: 10.5281/zenodo.20570332

Download PDF View on Zenodo
SoLS AI logo
Our team

The minds behind SoLS AI

A multidisciplinary team of researchers and engineers pushing the boundaries of efficient AI.

DJ

Daniel Joe

Project Manager
SoLS Research Group
D

Deepak

Project Leader
SoLS Research Group
BM

Bala Murugan

Lead Researcher
SoLS Research Group
S

Sudalai

Data Manager
SoLS Research Group
MS

Muthu Siva

Data Collector
SoLS Research Group
Vision

Where we're headed

From foundational research to production-ready adaptive AI systems.

Phase 1 — Foundation

RecurrMoE-1B architecture & pretraining

Designed the hierarchical MoR architecture, pretrained on 12B tokens across a 3-phase curriculum using 2× NVIDIA A100 80GB GPUs.

✓ Completed
Phase 2 — Publication

Research paper & open-access release

Published the architecture paper on Zenodo (DOI 10.5281/zenodo.20570332) with full methodology and benchmark results.

✓ Completed
Phase 3 — Scale Up

RecurrMoE-7B & enhanced training

Scaling to 7B parameters with extended pretraining on 100B+ tokens across multi-node GPU clusters with enhanced expert diversity.

● In progress
Phase 4 — Production

Inference engine & API platform

A production-grade inference engine with optimized virtual-layer KV cache, expert parallelism, and a developer API for adaptive model serving.

◇ Upcoming
Phase 5 — Ecosystem

Open-source toolkit & community

Open-source release of the RecurrMoE training framework, pretrained checkpoints, and evaluation tools.

◇ Upcoming
FAQ

Questions, answered

What exactly is RecurrMoE?

RecurrMoE is a 1B-parameter foundation-model architecture that unifies adaptive recursion (deciding how long to think about each token) with sparse Mixture-of-Experts routing (deciding which parameters to use). It produces the same quality output as a static transformer while spending substantially less compute.

How do you achieve 44.4% compute reduction?

Tokens that are easy to predict exit the network early, while hard tokens recurse deeper. Combined with sparse top-2 expert selection and virtual-layer KV mapping, the model averages an effective depth of ~10 of 18 layers — roughly 44% less computation per token in deployment.

Is the research open and reproducible?

Yes. The full paper is published open-access on Zenodo with a permanent DOI, including the architecture, the three-phase training curriculum, and all benchmark numbers reported here.

Can I use or build on this work?

The paper is open access today. An open-source training framework and pretrained checkpoints are planned for Phase 5 of our roadmap. Reach out if you'd like to collaborate sooner.

How can my organization partner with SoLS AI?

We're seeking compute sponsors, research collaborators, and partners aligned with efficient, sustainable AI. Email research@sols-ai.com and we'll get back to you.

Let's build the future of adaptive AI together

We're seeking visionary partners — investors, compute sponsors, and research collaborators — who share our mission of making AI fundamentally more efficient.

Partner with us Read our research
research@sols-ai.com