id,name,slug,company,lab,labSlug,year,category,subtype,worldModelType,primaryDomain,status,architecture,modality,trainingMethod,shortDescription,paperUrl,websiteUrl,tags,featured,lastUpdated
dreamer-v3,DreamerV3,dreamer-v3,Google DeepMind,Hafner et al.,deepmind,2023,Model-Based RL,Latent Dynamics Model,Imagination-based dynamics model,General RL,active,"RSSM with discrete representations, symlog predictions, KL-balanced objective",Visual + Proprioceptive,Self-supervised world model learning + imagination-based policy optimization,A general algorithm for mastering diverse domains with fixed hyperparameters through world model learning.,https://arxiv.org/abs/2301.04104,,"[""model-based-rl"",""latent-dynamics"",""imagination"",""general-purpose""]",true,2026-03-10
dreamer-v2,DreamerV2,dreamer-v2,Google,Hafner et al.,deepmind,2021,Model-Based RL,Latent Dynamics Model,Imagination-based dynamics model,Atari / Control,foundational,RSSM with categorical discrete latent variables,Visual,World model learning with discrete latent representations + imagination-based RL,The first model-based agent to achieve human-level performance on the Atari benchmark using discrete world model representations.,https://arxiv.org/abs/2010.02193,,"[""model-based-rl"",""latent-dynamics"",""discrete"",""atari""]",false,2026-02-18
planet,PlaNet,planet,Google,Hafner et al.,deepmind,2019,Model-Based RL,Latent Dynamics Model,Latent space planning model,Continuous control,foundational,RSSM with deterministic and stochastic paths,Visual,Variational inference for latent dynamics learning,Deep Planning Network: learns environment dynamics in latent space for image-based control without a policy network.,https://arxiv.org/abs/1811.04551,,"[""model-based-rl"",""latent-dynamics"",""planning"",""foundational""]",false,2026-02-05
rssm,RSSM,rssm,Google,Hafner et al.,deepmind,2019,Latent Dynamics,Latent Dynamics Architecture,Core dynamics architecture,World model backbone,active,Hybrid deterministic-stochastic recurrent model with encoder/decoder,Visual + Proprioceptive,Variational inference with reconstruction and KL objectives,Recurrent State-Space Model: the foundational architecture behind PlaNet and the entire Dreamer family of world models.,https://arxiv.org/abs/1811.04551,,"[""architecture"",""latent-dynamics"",""foundational"",""recurrent""]",false,2026-02-05
nvidia-cosmos,NVIDIA Cosmos,nvidia-cosmos,NVIDIA,NVIDIA Research,nvidia,2024,Foundation World Model,Video Generation World Model,Video world foundation model,Physical AI,active,Autoregressive and diffusion-based transformer models,Video + 3D,Large-scale video pre-training with physics-aware objectives,A platform of state-of-the-art generative world foundation models for physical AI development.,https://arxiv.org/abs/2501.03575,https://www.nvidia.com/en-us/ai/cosmos/,"[""foundation-model"",""video-generation"",""physical-ai"",""robotics""]",true,2026-03-12
genie-2,Genie 2,genie-2,Google DeepMind,DeepMind,deepmind,2024,Generative World Model,Interactive Environment Generator,Generative environment model,Environment generation,active,Autoregressive latent diffusion transformer with action conditioning,Image → Interactive 3D Environment,Large-scale video pre-training with action-conditioning,"A foundation world model that generates diverse, playable 3D environments from a single image prompt.",https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/,,"[""generative"",""3d-environments"",""embodied-ai"",""foundation-model""]",true,2026-03-14
muzero,MuZero,muzero,Google DeepMind,Schrittwieser et al.,deepmind,2020,Model-Based RL,Learned Model + Search,Abstract learned dynamics + MCTS,Games / Planning,foundational,Representation + Dynamics + Prediction networks with MCTS planning,Board states / Visual (Atari),Self-play with learned dynamics model and MCTS,Masters games without knowing the rules by learning a world model for planning via Monte Carlo tree search.,https://arxiv.org/abs/1911.08265,,"[""model-based-rl"",""mcts"",""planning"",""games""]",false,2026-02-20
predictron,Predictron,predictron,Google DeepMind,Silver et al.,deepmind,2017,Model-Based RL,Abstract World Model,Abstract internal dynamics model,Value prediction,foundational,Multi-step abstract model with λ-weighted returns,Abstract state representations,End-to-end supervised learning with multi-step abstract predictions,An architecture that integrates learning and planning into a single differentiable network via abstract world models.,https://arxiv.org/abs/1612.08810,,"[""abstract-model"",""planning"",""foundational"",""value-prediction""]",false,2026-01-28
unisim,UniSim,unisim,Google DeepMind,Yang et al.,deepmind,2023,Generative World Model,Universal Simulator,Action-conditioned video simulator,Robotics / Simulation,active,Video diffusion model with action conditioning,Video + Actions,Multi-domain video pre-training with action-conditioned generation,A universal simulator that learns to simulate real-world interactions from diverse data sources.,https://arxiv.org/abs/2310.06680,,"[""generative"",""universal-simulator"",""robotics"",""video-generation""]",false,2026-03-08
td-mpc2,TD-MPC2,td-mpc2,MIT / Meta,Hansen et al.,mit,2024,Model-Based RL,Temporal Difference Learning + MPC,Implicit dynamics + MPC planner,Multi-task control,active,Implicit latent dynamics model with TD-learning and MPPI planning,State + Visual,Joint TD-learning and latent dynamics model training with MPC-based acting,A scalable world model agent that combines TD-learning with model-predictive control across 104 diverse tasks.,https://arxiv.org/abs/2310.16828,https://tdmpc2.com,"[""model-based-rl"",""multi-task"",""planning"",""generalist""]",false,2026-03-05
ha-world-model,World Models (Ha & Schmidhuber),ha-world-model,Google Brain / IDSIA,Ha & Schmidhuber,deepmind,2018,Model-Based RL,VAE + MDN-RNN,Generative latent dynamics model,Game environments,foundational,VAE (visual encoder) + MDN-RNN (dynamics) + linear controller,Visual,Unsupervised VAE training + RNN dynamics learning + evolutionary policy search,The seminal paper that popularized the concept of world models: learning to imagine environments and training policies entirely in dreams.,https://arxiv.org/abs/1803.10122,https://worldmodels.github.io,"[""foundational"",""imagination"",""vae"",""dream-training""]",false,2026-02-10
v-jepa,V-JEPA,v-jepa,Meta,Meta FAIR,meta-fair,2024,Self-Supervised World Model,Joint Embedding Predictive Architecture,Self-supervised visual world model,Video understanding,active,Vision Transformer with joint embedding predictive objective,Video,Self-supervised prediction in abstract representation space (no pixel reconstruction),Video Joint Embedding Predictive Architecture: learns visual world models through self-supervised video prediction in abstract representation space.,https://arxiv.org/abs/2404.08471,,"[""self-supervised"",""jepa"",""video"",""representation-learning""]",false,2026-03-13
iris,IRIS,iris,Microsoft Research,Micheli et al.,microsoft-research,2023,Model-Based RL,Autoregressive World Model,Discrete token-based dynamics model,Atari games,active,VQ-VAE tokenizer + autoregressive Transformer dynamics model,Visual,"Discrete tokenization + autoregressive next-token prediction over (observation, action, reward) sequences","A world model agent that tokenizes observations into discrete tokens and models environment dynamics autoregressively, like a language model over game frames.",https://arxiv.org/abs/2209.00588,,"[""autoregressive"",""transformer"",""discrete-tokens"",""atari""]",false,2026-02-25
gaia-1,GAIA-1,gaia-1,Wayve,Wayve,wayve,2023,Foundation World Model,Autonomous Driving World Model,Generative driving world model,Autonomous driving,active,"Video diffusion model with multi-modal conditioning (text, action, video)",Video + Text + Actions,Large-scale driving video pre-training with multi-modal conditioning,"A generative world model for autonomous driving that predicts realistic driving scenarios from text, action, and video inputs.",https://arxiv.org/abs/2309.17080,https://wayve.ai/thinking/gaia-1/,"[""autonomous-driving"",""video-generation"",""foundation-model"",""multi-modal""]",false,2026-03-01
imagination-augmented-agents,Imagination-Augmented Agents (I2A),imagination-augmented-agents,Google DeepMind,Weber et al.,deepmind,2017,Model-Based RL,Imagination-Augmented Policy,Imagination-augmented model-free agent,Atari / Planning,foundational,Model-free policy augmented with rollout encoder over imagined trajectories,Visual,Joint training of environment model and imagination-augmented policy network,An agent architecture that augments model-free policies with learned imagination rollouts from an environment model.,https://arxiv.org/abs/1707.06203,,"[""imagination"",""hybrid"",""model-based-rl"",""foundational""]",false,2026-01-20
value-imagination-model,Value Prediction Network (VPN),value-prediction-network,University of Michigan / Google Brain,Oh et al.,deepmind,2017,Model-Based RL,Value-Targeted Dynamics Model,Value-focused abstract dynamics,Planning / Games,foundational,Abstract state transition model with value and reward prediction heads,Visual / Abstract states,End-to-end training with value and reward prediction targets,"A neural network that learns to plan by predicting future values and rewards through abstract state transitions, without reconstructing observations.",https://arxiv.org/abs/1707.03497,,"[""abstract-model"",""value-prediction"",""planning"",""foundational""]",false,2026-01-15
ami-world-model,AMI World Model,ami-world-model,AMI Labs,AMI Labs,ami-labs,2024,Foundation World Model,Multimodal World Foundation Model,Multimodal generative world model,Embodied AI / Robotics,emerging,"Multimodal transformer with cross-attention between vision, language, and proprioception",Vision + Language + Proprioception,Large-scale multimodal pre-training with physics-aware objectives,"A multimodal world foundation model designed for embodied AI, combining visual, proprioceptive, and language understanding for robot learning.",,,"[""foundation-model"",""multimodal"",""embodied-ai"",""robotics""]",false,2026-03-16
sora,Sora,sora,OpenAI,OpenAI,openai,2024,Generative World Model,Video Generation World Model,Text-to-video world simulator,Video generation / Simulation,active,Diffusion Transformer (DiT) operating on spacetime patches,Text → Video,Large-scale video and image pre-training with diffusion objectives on spacetime latent patches,OpenAI's video generation model that simulates the physical world by generating realistic videos from text prompts.,https://openai.com/index/video-generation-models-as-world-simulators/,https://openai.com/sora,"[""video-generation"",""diffusion"",""world-simulator"",""foundation-model""]",true,2026-03-18
world-model-on-million-timesteps,DIAMOND,diamond,Microsoft Research / University of Geneva,Alonso et al.,microsoft-research,2024,Model-Based RL,Diffusion World Model,Diffusion-based environment simulator,Atari games,active,Conditional diffusion model over observation sequences with action conditioning,Visual,Diffusion model training on environment transitions + imagination-based policy optimization,DIffusion As a Model Of the eNvironment in Deep RL: uses diffusion models as world models for reinforcement learning agents.,https://arxiv.org/abs/2405.12399,,"[""diffusion"",""model-based-rl"",""atari"",""imagination""]",false,2026-03-10
pandora,Pandora,pandora,Tsinghua University / ByteDance,Xiang et al.,deepmind,2024,Generative World Model,Interactive World Generator,Hybrid autoregressive-diffusion world model,Interactive 3D environments,emerging,Hybrid autoregressive-diffusion transformer with action and text conditioning,Text + Actions → Video,Large-scale video pre-training with hybrid autoregressive-diffusion objectives,"A general world model combining autoregressive and diffusion architectures for generating interactive, controllable video environments.",https://arxiv.org/abs/2404.16078,https://world-model.maitrix.org/,"[""generative"",""hybrid"",""interactive"",""video-generation""]",false,2026-03-15
oasis,OASIS,oasis,Decart / Etched,Decart,decart,2024,Generative World Model,Real-Time Interactive Simulator,Real-time playable world model,Real-time game simulation,active,Spatial autoencoder + latent diffusion transformer with real-time action conditioning,Actions → Video (real-time),Large-scale gameplay video training with action-conditioned diffusion,An open-source real-time interactive world model that generates playable game environments at 20+ FPS entirely from a neural network.,https://oasis-model.github.io/,https://oasis-model.github.io/,"[""real-time"",""interactive"",""open-source"",""game-simulation""]",false,2026-03-12
copilot4d,Copilot4D,copilot4d,Waabi,Waabi,waabi,2023,Foundation World Model,4D Point Cloud World Model,Spatiotemporal LiDAR world model,Autonomous driving,active,Discrete diffusion transformer over VQ-VAE tokenized LiDAR point clouds,LiDAR point clouds (4D),VQ-VAE tokenization of point clouds + discrete diffusion for spatiotemporal prediction,A world model for autonomous driving that predicts future LiDAR point clouds in 4D (3D space + time) using discrete diffusion.,https://arxiv.org/abs/2311.01017,,"[""autonomous-driving"",""lidar"",""4d"",""diffusion""]",false,2026-03-08
genie-1,Genie,genie-1,Google DeepMind,Bruce et al.,deepmind,2024,Generative World Model,Generative Interactive Environment,Action-controllable generative model,2D environment generation,foundational,Video tokenizer (ST-ViViT) + Latent Action Model + Dynamics Model (MaskGIT-style),Image → Interactive 2D Environment,Unsupervised learning from 200K hours of internet video with no action labels,"The first generative interactive environment trained from unlabeled internet videos, capable of generating action-controllable 2D worlds.",https://arxiv.org/abs/2402.15391,,"[""generative"",""unsupervised"",""interactive"",""foundation-model""]",false,2026-03-14
jepa,I-JEPA,i-jepa,Meta,Meta FAIR,meta-fair,2023,Self-Supervised World Model,Image Joint Embedding Predictive Architecture,Self-supervised visual world model,Image understanding,active,Vision Transformer with asymmetric masking and prediction in representation space,Image,Self-supervised prediction of masked image regions in abstract representation space,Image Joint Embedding Predictive Architecture: learns visual representations by predicting abstract image regions without pixel reconstruction.,https://arxiv.org/abs/2301.08243,,"[""self-supervised"",""jepa"",""image"",""representation-learning""]",false,2026-03-10
GameNGen,GameNGen,gamengen,Google Research,Valevski et al.,google-research,2024,Generative World Model,Neural Game Engine,Diffusion-based neural game engine,Real-time game simulation,active,Fine-tuned Stable Diffusion 1.4 with action and frame history conditioning,Actions + Frame History → Next Frame,"Two-phase: (1) RL agent generates gameplay data, (2) Diffusion model fine-tuned on gameplay sequences","The first neural model to simulate a complex game (DOOM) in real-time at high quality, making the game engine itself a neural network.",https://arxiv.org/abs/2408.14837,,"[""neural-game-engine"",""diffusion"",""real-time"",""doom""]",false,2026-03-11
emu-video,Emu Video,emu-video,Meta,Meta GenAI,meta-fair,2023,Generative World Model,Video Generation Model,Factorized text-to-video model,Video generation,active,Two-stage diffusion model: text-to-image + image-to-video,Text → Image → Video,Factorized training: image diffusion + video diffusion with image conditioning,"Meta's efficient video generation model using a factorized approach: first generate an image, then animate it into a video.",https://arxiv.org/abs/2311.10709,,"[""video-generation"",""diffusion"",""factorized"",""text-to-video""]",false,2026-03-05
3d-vla,3D-VLA,3d-vla,MIT / Tsinghua,Zhen et al.,mit,2024,Foundation World Model,3D Vision-Language-Action Model,3D-aware embodied world model,Robotics / Embodied AI,emerging,3D-aware vision-language model with integrated world model and action decoder,3D Point Clouds + Language + Actions,"Multi-task training: 3D understanding, world modeling, and action generation","A 3D vision-language-action model with a built-in world model for embodied AI, enabling 3D-aware reasoning, planning, and action generation.",https://arxiv.org/abs/2403.09631,,"[""3d"",""vision-language-action"",""robotics"",""embodied-ai""]",false,2026-03-16
rt-2,RT-2,rt-2,Google DeepMind,Brohan et al.,deepmind,2023,Foundation World Model,Vision-Language-Action Model,Web-knowledge transfer model for robotics,Robotics,active,PaLI-X / PaLM-E backbone fine-tuned with action tokenization,Visual + Language + Proprioceptive,Co-fine-tuning on web VLM data + robot demonstration trajectories,A vision-language-action model that transfers web-scale knowledge directly to robot control.,https://arxiv.org/abs/2307.15818,https://robotics-transformer2.github.io/,"[""robotics"",""vision-language-action"",""foundation-model"",""embodied-ai"",""transfer-learning""]",true,2026-04-07
lwm,Large World Model (LWM),lwm,UC Berkeley,Liu et al.,uc-berkeley,2024,Foundation World Model,Long-Context Multimodal Model,Million-length video-language world model,Video Understanding,active,LLaMA-based transformer with RingAttention for million-token context,Visual (Video) + Language,Progressive context extension on interleaved video-text data,A foundation model trained on 1M+ interleaved video and language tokens for long-horizon world understanding.,https://arxiv.org/abs/2402.08855,https://largeworldmodel.github.io/,"[""long-context"",""multimodal"",""video-understanding"",""foundation-model"",""open-source""]",false,2026-04-07
stable-video-diffusion,Stable Video Diffusion,stable-video-diffusion,Stability AI,Blattmann et al.,stability-ai,2023,Generative World Model,Video Diffusion Model,Image-to-video diffusion model,Video Generation,active,3D UNet latent diffusion with temporal attention layers,Visual (Image → Video),Multi-stage training: image pretraining → video fine-tuning on curated dataset,"An open-source foundation model for video generation, producing temporally consistent video from a single image.",https://arxiv.org/abs/2311.15127,https://stability.ai/stable-video,"[""video-generation"",""diffusion"",""open-source"",""image-to-video"",""foundation-model""]",false,2026-04-07
mile,MILE,mile,Wayve,Hu et al.,wayve,2022,Foundation World Model,Driving World Model,Model-based imitation learning for driving,Autonomous Driving,active,Variational autoencoder with spatial-temporal transformer for dynamics prediction,Visual (Multi-camera) + Ego-state,Model-based imitation learning with future imagination rollouts,"A world model for autonomous driving that jointly learns dynamics, perception, and planning through model-based imitation learning.",https://arxiv.org/abs/2210.07729,https://wayve.ai/thinking/mile/,"[""autonomous-driving"",""imitation-learning"",""end-to-end"",""world-model"",""planning""]",false,2026-04-07
steve-1,STEVE-1,steve-1,UT Austin,Lifshitz et al.,ut-austin,2023,Generative World Model,Instruction-Following Game Agent,Text-conditioned generative agent in open-world,Game AI / Open World,active,VPT backbone + MineCLIP goal encoder + latent-conditioned policy,Visual + Language (instructions),Hindsight relabeling on unlabeled gameplay + CLIP-based instruction conditioning,An instruction-following agent for Minecraft that uses a generative world model to execute open-ended text commands.,https://arxiv.org/abs/2306.00937,https://sites.google.com/view/steve-1,"[""minecraft"",""instruction-following"",""open-world"",""generative-agent"",""video-pretraining""]",false,2026-04-07
gen-3-alpha,Gen-3 Alpha,gen-3-alpha,Runway,Runway Research,runway,2024,Generative World Model,Video Generation Model,Controllable video generation with scene understanding,Video Generation,active,Proprietary multimodal transformer with temporal consistency mechanisms,Text + Image → Video,Large-scale video-image joint training with human feedback alignment,"Runway's next-generation video model with fine-grained control over motion, style, and composition.",,https://runwayml.com/research/gen-3-alpha,"[""video-generation"",""creative-ai"",""controllable-generation"",""text-to-video""]",false,2026-04-07
genie-3,Genie 3,genie-3,Google DeepMind,DeepMind,deepmind,2025,Generative World Model,Interactive 3D World Model,Real-time interactive 3D environment generation,3D World Generation,active,Autoregressive latent world model with spatiotemporal transformer,Text → Interactive 3D Environment,Large-scale internet video and 3D data pre-training with action-conditioned generation,Google DeepMind's general-purpose world model that generates interactive 3D environments from text prompts in real time at 24fps.,https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/,https://deepmind.google/models/genie/,"[""3d-generation"",""interactive"",""real-time"",""text-to-world"",""foundation-model""]",true,2026-04-10
v-jepa-2,V-JEPA 2,v-jepa-2,Meta,Meta FAIR,meta-fair,2025,Self-Supervised World Model,Video Prediction Model,Joint-embedding predictive world model for video understanding and robot planning,Physical Reasoning & Robotics,active,"Vision Transformer with joint-embedding predictive architecture, latent space prediction",Video → Latent Predictions,"Self-supervised learning via latent space prediction from video, no pixel reconstruction",Meta FAIR's self-supervised video world model achieving state-of-the-art visual understanding and enabling zero-shot robot control.,https://arxiv.org/abs/2506.09985,https://ai.meta.com/research/vjepa/,"[""self-supervised"",""jepa"",""video-prediction"",""robotics"",""open-source"",""physical-reasoning""]",true,2026-04-10
leworldmodel,LeWorldModel,leworldmodel,Mila / NYU / Samsung SAIL,Mila / NYU / Samsung SAIL,mila-nyu-samsung-sail,2026,Self-Supervised World Model,Compact JEPA Model,Compact joint-embedding predictive world model for physical understanding,Physical Reasoning,active,Compact Vision Transformer with JEPA and SigReg regularization,Visual → Latent Predictions,"Self-supervised JEPA training with SigReg collapse prevention, single-GPU trainable","A compact 15M-parameter JEPA world model that learns real-world physics on a single GPU, solving the notorious representation collapse problem.",https://arxiv.org/abs/2603.19312,,"[""jepa"",""compact"",""efficient"",""physical-reasoning"",""sigreg"",""single-gpu""]",false,2026-04-10
pixverse-r1,PixVerse R1,pixverse-r1,PixVerse,PixVerse Research,pixverse,2026,Generative World Model,Real-time Multiplayer World Model,Real-time interactive world generation with multi-user participation,Interactive Entertainment,active,Real-time generative world model with multi-stream rendering,Text/Image → Interactive Shared World,Large-scale video and 3D data training with real-time streaming optimization,The first real-time world model supporting multi-user shared worlds with personalized avatars and no session limits.,,https://pixverse.ai,"[""real-time"",""multiplayer"",""interactive"",""shared-worlds"",""avatars""]",false,2026-04-10
marble,Marble,marble,World Labs,World Labs,world-labs,2026,Foundation World Model,Spatially Consistent 3D World Generator,Persistent multimodal 3D world model,3D World Generation,active,Proprietary multimodal 3D world generation system,Text/Image/Video/360 -> Persistent 3D World,Large-scale multimodal world understanding and 3D scene generation,"World Labs' multimodal world model for generating spatially consistent, persistent 3D environments from text, images, video, and 360 inputs.",https://www.worldlabs.ai/,https://www.worldlabs.ai/,"[""3d-generation"",""multimodal"",""persistent-worlds"",""creator-tools"",""foundation-model""]",true,2026-06-12
1x-world-model,1X World Model,1x-world-model,1X,1X,1x,2025,Foundation World Model,Humanoid Robot Video World Model,Physics-grounded action-conditioned video world model,Humanoid Robotics,active,Physics-grounded action-conditioned video model for humanoid control,Robot Video + Actions -> Future Observations,Embodied video prediction grounded in robot interactions and failure-aware learning,1X's physics-grounded video world model for anticipating the outcomes of NEO's actions and supporting generalization to unseen household tasks.,https://www.1x.tech/ai,https://www.1x.tech/ai,"[""humanoid-robotics"",""video-world-models"",""physical-ai"",""embodied-ai"",""foundation-model""]",true,2026-06-12
playworld,PlayWorld,playworld,Princeton University,Princeton University,princeton,2026,Generative World Model,Action-Conditioned Robot Video Model,Robot manipulation world simulator learned from autonomous play,Robot Manipulation,active,Action-conditioned video world model trained on autonomous robot play data,Multi-view Robot Video + Actions -> Future Observations,Autonomous self-play data collection with action-conditioned video model training,A Princeton robot world model trained from autonomous self-play to simulate contact-rich manipulation and support policy evaluation and RL fine-tuning.,https://arxiv.org/abs/2603.09030,https://robot-playworld.github.io/,"[""robotics"",""self-play"",""video-world-models"",""manipulation"",""policy-evaluation""]",true,2026-06-12