Popular on TelAve


Similar on TelAve

Solo Researcher Builds Three Novel AI Architectures From Scratch, Including Post-Transformer Model

TelAve News/10892573
ERIE, Pa. - TelAve -- Saiyan Corp, founded by independent researcher Dakuwon Moody, today announced the development of three novel artificial intelligence architectures, each representing a distinct approach to neural language modeling.

BulmaX is a self-modifying transformer equipped with 13 novel subsystems, including dynamic neurogenesis that allows the model to grow new attention heads during training. The model began training with 24 attention heads and autonomously expanded to 396 heads through a process inspired by biological neural development. BulmaX is currently scaling to 7 billion parameters on Google Cloud TPU infrastructure.

Goku is a 1 billion parameter dense transformer designed for maximum performance, featuring a complete inference engine written entirely in x86-64 NASM assembly language. The compiled inference binary is 42 kilobytes and contains zero C code across 23 hand-written assembly source files.

Beerus represents a departure from transformer architecture entirely. The model contains no attention mechanism, no query/key/value projections, and no positional encoding. Instead, it processes language through a swarm of competing micro-expert cells connected by stigmergic conductivity tubes inspired by Physarum polycephalum (slime mold) fluid dynamics. The network grows, clones, and kills its own cells during training.

More on TelAve News
All three models are trained on consumer and free-tier hardware, including a single Google Cloud L4 GPU VM, free Kaggle TPU access, a $200 Intel Arc A750 desktop GPU, and Google Cloud TPU Research Credits. The total infrastructure cost is near zero.

"I built these because I wanted to explore what's possible beyond the standard transformer," said Moody. "BulmaX proves that models can grow their own architecture. Beerus proves you don't need attention at all. And Goku proves one person can train a billion-parameter model from scratch on free hardware."

All code is original and all architecture decisions are novel. The models are actively training with plans for public release.

About Saiyan Corp:
Saiyan Corp is an independent AI research organization founded by Dakuwon Moody, focused on novel neural architecture design and efficient training on consumer hardware.

Contact: @YNSScarSaiyan on X (Twitter)

Source: Saiyan Corp INC
Filed Under: Technology

Show All News | Disclaimer | Report Violation

0 Comments

Latest on TelAve News