Fern-1 — from-scratch transformer
Let's grow something.
A real transformer LLM — built from scratch in JavaScript.
No APIs. No pre-trained models. Just neural nets.
About Fern
Fern is a small but real transformer-based language model, built entirely from scratch in JavaScript. There are no AI APIs and no pre-trained models — every weight was learned by training the network on a baked-in English corpus using a hand-written backpropagation algorithm.
Architecture
- Decoder-only GPT-style transformer
- Multi-head causal self-attention
- Learned token + positional embeddings
- LayerNorm + GELU feed-forward layers
- Weight-tied output projection
- Byte-pair-style tokenizer with ~3.5k token vocabulary
What "from scratch" means
Every line of the model — the matrix multiplies, the softmax, the attention, the layer norm, the backprop, the optimizer — is written by hand in plain JavaScript. No TensorFlow, no PyTorch, no ONNX, no external inference engine. The model loads as a static JSON file and runs entirely in your browser, making it perfect for static hosting like Cloudflare Pages.
Honest expectations
Fern is small (about 1.3M parameters) and trained for only a few minutes on a curated corpus. It will not rival GPT-4 or Claude in coherence. But it is a genuine neural network generating text token-by-token through learned attention patterns — every word it produces comes from a real forward pass through learned weights. The quality gap between Fern and large commercial LLMs is a difference of scale (billions of params + weeks of GPU training), not of kind.
Try asking it
Ask about science, history, programming, or just chat. If a response seems off, regenerate it or rephrase — Fern's small size means its outputs will be inconsistent.