 | Hi everyone!! I really wanted to share my research what I've been working on. I wanted to build a nn that can simulate games, or at least start doing that Most video generators are too large to run on consumer hardware realtime, so I I designed a model that does this from scratch. No fine tuning bs or anything The core de noiser network is fully trained from scratch to support this goal. From image to games data. That video. above is on a RTX 5090. The nn is a small Transformer-like model and works in a causal way, just like LLMs. That lets us KV Cache all past information and do a simple autoregressive decode forward passes for every new frame we want. In the video shared, the model is a 0.4B variant with some SIGNIFICANT ISSUES like poor motion and some weird flashes, some context issues It's taking the keyboard actions I give it in realtime and utilising that in the forward pass. (no classifier free guidance though) Im training the next iteration , a 0.8B model now. Btw I haven't done quantisation yet, that can save a LOT more time. bf16 is slow. [link] [comments] |
Deep Neural Network that turns any Image into a Playable Game ! All on consumer GPUs and Not Datacenters
Reddit r/artificial / 5/30/2026
💬 OpinionDeveloper Stack & InfrastructureSignals & Early TrendsModels & Research
Key Points
- The article describes a newly developed deep neural network that can convert input images into playable game-like video sequences, aiming for real-time inference on consumer GPUs rather than datacenters.
- The author claims to train the “core de-noiser” network from scratch (without fine-tuning tricks) using image-to-game data.
- The model is described as a small Transformer-like, causal architecture similar to LLMs, enabling efficient autoregressive decoding with KV caching across frames.
- Early results are shown from an approximately 0.4B parameter variant on an RTX 5090, with noted issues such as poor motion, flash artifacts, and context handling problems.
- The system incorporates real-time keyboard actions into the forward pass, and the author is currently training a larger 0.8B iteration while noting that quantization has not yet been applied (bf16 is too slow).
Related Articles
AI Blog Writing Showdown: ChatGPT vs. Claude vs. Doubao vs. Qwen vs. Gemini vs. SEONIB
Dev.to
Summary - TerpreT: A Probabilistic Programming Language for Program Induction
Dev.to
What I learned building a debugger for PyTorch training loops and how it changed how I think about failure diagnosis [D]
Reddit r/MachineLearning
Claude Checks Agent Reputation: ERC-8004 MCP Tools for Trustless AI Validation
Dev.to
Gemini core part 3
Reddit r/artificial
