The Enigma Stepper: Mixture of Experts (MoE) as a High-Dimensional Rotor Machine

Ciphers

Aug 5

"Explore the striking structural parallels between WWII Enigma rotor mechanics and modern sparse Mixture of Experts (MoE) LLM routing. Includes a Python prototype."

---

## I. Introduction: The Art of Conditional Paths

An Enigma machine never processed two letters the same way twice. Press "T" once, and the electrical signal travels through one circuit; press "T" again a moment later, and it travels through a completely different one. The reason is mechanical: every keystroke advances at least one rotor by a notch, physically rewiring the path the signal takes on its way to the lampboard. Nazi Germany's military relied on this property — constant, deterministic-but-hidden change — to keep its communications unreadable to anyone without the day's rotor settings.

Look at how a modern sparse language model handles a sentence, and you'll notice a strangely similar idea at work. Models like Mixtral don't fire every one of their billions of parameters for every word. Instead, a small gating network looks at each token and decides, on the fly, which subset of "expert" subnetworks should process it. Token one might go to experts 2 and 5; token two might go to experts 1 and 4. The computational path shifts token by token, the same way the electrical path through an Enigma shifted keystroke by keystroke.

That resemblance is more than a cute analogy. It raises a real question: what happens when you look at sparse neural routing not just as a performance optimization, but as a dynamic cryptographic permutation function? And if it behaves like one, does it inherit the same weaknesses that classical cryptanalysts once used to break it?

## II. Mechanical Rotors vs. Gating Networks

**Enigma mechanics**

- **Input signal:** a keyboard press (say, the letter T)

- **Routing layer:** the stepping rotors (R₁, R₂, R₃) plus a fixed reflector

- **Dynamic state:** the rotors advance mechanically with every keystroke, so the circuit that processes token *N+1* is never identical to the one that processed token *N*

The Enigma's security came almost entirely from this stepping behavior. A single rotor is just a substitution cipher — trivially breakable. Three rotors that step relative to one another, combined with a plugboard, produce a keyspace large enough that Germany considered it unbreakable. It wasn't the substitution itself that mattered; it was the fact that the substitution kept changing according to a hidden, position-dependent rule.

**MoE mechanics**

- **Input signal:** a high-dimensional token embedding vector, **x**

- **Routing layer:** a gating softmax layer, `Gating(x) = TopK(Softmax(W_g · x))`

- **Dynamic state:** the expert assignment shifts token by token, based on the semantic content of each embedding, routing the input through sparse feed-forward networks

Where Enigma's routing rule was a fixed mechanical ratchet, an MoE gating network's routing rule is learned — a function of the content itself rather than a simple step counter. But structurally, both systems do the same job: they take an input, use an internal state to select one of several parallel transformation paths, and update that state as they go. Enigma's "state" is rotor position. An MoE's "state" is the sequence of routing decisions the gate has made.

## III. Side-Channel Analysis: Can MoE Routing Leak Information?

This is where the historical parallel stops being decorative and starts being genuinely useful.

Bletchley Park's codebreakers didn't just attack Enigma's ciphertext directly — they attacked its *metadata*. Traffic analysis, stereotyped message formats, and above all the statistical patterns in how rotors stepped over the course of a message gave cryptanalysts a foothold long before they recovered a single plaintext letter. The machine's operational behavior leaked information about its internal state, independent of the encrypted content itself.

Sparse MoE models create an analogous exposure. The sequence of which experts get activated for which tokens — the *routing log* — is itself a signal correlated with the input. Two prompts that route through very different sets of experts are, almost by definition, semantically different in ways an outside observer could try to infer. In deployments where routing telemetry, expert-load balancing metrics, or per-token activation logs are exposed (for debugging, billing, or infrastructure monitoring), that metadata can act as a side channel — leaking coarse information about prompt content without ever touching the model's actual output.

This has a direct practical implication for anyone building privacy-sensitive LLM infrastructure: **routing-level side-channel leakage is a real category of risk**, not just a thought experiment. Just as Enigma operators didn't realize their stepping patterns were giving away information, teams instrumenting MoE inference pipelines can inadvertently expose signal through telemetry they didn't think of as sensitive.

## IV. Hands-On Python Conceptual Prototype

To make the analogy concrete, here's a small Python simulation of a toy MoE router acting as an adaptive cryptographic rotor set. It doesn't use real trained weights — it's a conceptual model, not a production cipher — but it demonstrates the mechanism: a router selects an "expert" substitution matrix per token, based on both token identity and position, producing a rotor-like dynamic permutation.

```python

import numpy as np

class MoERotorEngine:

def __init__(self, num_experts: int = 4, top_k: int = 1, seed: int = 42):

"""

Simulates an MoE router acting as a dynamic cryptographic rotor.

- num_experts: Number of 'Expert' substitution rotors available.

- top_k: How many experts are activated per token.

"""

np.random.seed(seed)

self.num_experts = num_experts

self.top_k = top_k

self.alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ "

self.vocab_size = len(self.alphabet)

# 1. Initialize random gating weights (Router)

self.gating_weights = np.random.randn(self.vocab_size, self.num_experts)

# 2. Build Expert Substitution Matrices (Rotors)

self.expert_rotors = []

for _ in range(self.num_experts):

perm = np.random.permutation(self.vocab_size)

self.expert_rotors.append(perm)

def _gate_token(self, token_idx: int, step_offset: int) -> int:

"""Gating network: decides which expert (rotor) handles the current token."""

# Dynamic context combining token identity and stepping offset

input_vector = np.zeros(self.vocab_size)

input_vector[(token_idx + step_offset) % self.vocab_size] = 1.0

# Calculate gating scores (logits)

logits = np.dot(input_vector, self.gating_weights)

# Select Top-K expert (rotor) index

chosen_expert = np.argsort(logits)[-self.top_k:][0]

return chosen_expert

def encrypt(self, plaintext: str) -> tuple[str, list[int]]:

"""Encrypts plaintext by routing each token through a dynamically chosen expert."""

ciphertext = []

routing_log = []

for step, char in enumerate(plaintext.upper()):

if char not in self.alphabet:

ciphertext.append(char)

continue

token_idx = self.alphabet.index(char)

# Step 1: MoE Router selects the active expert (rotor)

chosen_expert = self._gate_token(token_idx, step)

routing_log.append(chosen_expert)

# Step 2: Route token through selected Expert Substitution Matrix

encrypted_idx = self.expert_rotors[chosen_expert][token_idx]

ciphertext.append(self.alphabet[encrypted_idx])

return "".join(ciphertext), routing_log

# --- Example Demonstration ---

if __name__ == "__main__":

engine = MoERotorEngine(num_experts=4, top_k=1, seed=1943)

message = "CIPHERGRID MOE ROUTER TEST"

encrypted_msg, routes = engine.encrypt(message)

print("--- MOE ENIGMA STEPPER DEMO ---")

print(f"Original Text : {message}")

print(f"Encrypted Text: {encrypted_msg}")

print(f"Expert Routes : {routes}")

```

Run it, and you'll see exactly what the historical analogy predicts: the same letter encrypts differently depending on where it falls in the message, because the "rotor" (expert) handling it changes with position — just as a repeated letter in an Enigma-encoded message was never guaranteed to produce the same ciphertext twice.

Try the fully interactive version of this simulator — type your own message and watch the routing happen token by token — in the widget below.

*[Interactive MoE Enigma Stepper widget embeds here]*

## V. Closing Thought

Enigma's designers assumed that constant, hidden state change was enough to guarantee secrecy. It wasn't — the machine's own operational patterns, not just its ciphertext, gave codebreakers a way in. As sparse MoE architectures become the default way to scale large language models efficiently, it's worth asking the same question of them: is the routing itself telling an attacker something the raw output never would?

---

*Have thoughts on routing-level side-channel risk in production MoE deployments? We'd love to hear from you — reach out via the CipherGrid newsletter or drop a comment below.*

https://books.apple.com/us/audiobook/portals/id679251155

Ingramspark.com

Reddit.com