Generative AI Architectures logopeterindia.net
Generative AI

Generative AI Architectures

The main architectures behind generative AI, Transformers, diffusion models, GANs, VAEs and more, with how they work and where they fit.

36 topics•6 categories•Updated November 2026

Prominent Generative Architectures

5 entries

The five families behind most of today's generative systems.

01TransformersAttention-based autoregressive models behind large language models; they generate one token at a time and scale well with data and compute. 02Diffusion ModelsLearn to reverse a gradual noising process, generating images, video and audio by denoising step by step. 03Generative Adversarial Networks (GANs)A generator and a discriminator train against each other; fast sampling, but training can be unstable. 04Variational Autoencoders (VAEs)Encode data into a probabilistic latent space and decode samples from it; stable training and a useful latent representation. 05State Space Models (SSMs)Linear-time sequence models such as Mamba that offer an alternative or complement to attention.

Other Generative Families

4 entries

Further model classes with distinct ways of learning a data distribution.

06Autoregressive Models (PixelCNN)Factorise the data distribution and generate one element at a time, here pixels. 07Normalizing Flows (RealNVP, Glow)Invertible transformations that give exact likelihoods and fast sampling. 08Energy-Based ModelsAssign low energy to likely data; LeCun et al.'s tutorial on energy-based learning. 09Flow MatchingTrains continuous flows by regressing vector fields; used in recent image and video generators.

Hybrids & Scaling Architectures

6 entries

How the families combine and scale in current systems.

10Latent Diffusion (Stable Diffusion)Runs diffusion in the compressed latent space of an autoencoder to cut cost. 11Diffusion Transformers (DiT)Replaces the U-Net in diffusion models with a transformer backbone. 12Mixture of Experts (Switch Transformers)Routes each token to a few expert sub-networks to grow capacity without proportional compute. 13Hybrid Transformer-Mamba (Jamba)Interleaves attention and state space layers in one model. 14Diffusion Language ModelsApply diffusion-style generation to text instead of left-to-right decoding. 15Vision TransformersTransformers applied to image patches, a backbone for vision and generation.

Architectures by Modality

5 entries

Which architectures dominate for each kind of output.

16Text and CodeMostly autoregressive transformers, with state space, mixture-of-experts and diffusion variants. 17ImagesDiffusion and latent diffusion lead, with GANs and VAEs as components or earlier approaches. 18VideoDiffusion transformers operating on spacetime patches. 19Audio and SpeechAutoregressive models such as WaveNet, now joined by diffusion and codec language models. 20MultimodalModels that accept and generate several modalities, usually transformer based.

Learn More

7 entries

Courses and explainers for going deeper.

21Attention Is All You NeedThe original Transformer paper. 22Generative Adversarial Networks (Goodfellow et al.)The paper that introduced GANs. 23Auto-Encoding Variational BayesThe original VAE paper. 24Denoising Diffusion Probabilistic ModelsThe paper that made diffusion models practical for image generation. 25MambaLinear-time sequence modelling with selective state spaces. 26What are Diffusion Models? (Lilian Weng)A widely read explainer. 27Stanford CS236: Deep Generative ModelsCourse notes covering autoregressive models, VAEs, flows, GANs, EBMs and diffusion.

Related Pages

9 entries

Related pages on this site.

28Generative AIHub for generative AI. 29Top Deep Learning AlgorithmsANN, CNN, RNN, GAN, Transformer and more. 30Language ModelsArchitectures and types of language models. 31Large Language Models (LLMs)Directory of LLMs. 32Diffusion ModelsDiffusion models directory. 33Swin TransformerHierarchical vision transformer. 34Datasets for Generative AITraining data for generative models. 35AI ArchitectureAI system architecture. 36Deep LearningDeep learning resources.

Frequently Asked Questions

What are generative AI architectures?

The neural network designs used to learn the distribution of data and generate new content such as text, images, audio, video or code. The prominent families are Transformers, diffusion models, GANs, VAEs and state space models.

Which generative architecture is best?

None is best everywhere. Transformers dominate text, diffusion models lead image and video generation, GANs offer fast sampling, VAEs give structured latent spaces, and state space models target long sequences efficiently.

How do Transformers generate content?

Most generative transformers are autoregressive: they predict the next token from the preceding ones using self-attention, and repeat until the output is complete.

How do diffusion models work?

They are trained to remove noise from corrupted data. At generation time they start from noise and denoise it step by step, often guided by a text prompt, to produce an image, video or audio clip.

What is the difference between a GAN and a VAE?

A GAN trains a generator against a discriminator and tends to produce sharp samples but can be unstable. A VAE learns an encoder and decoder with a probabilistic latent space and trains stably, though its samples can be blurrier.

What is latent diffusion?

Diffusion carried out in the compressed latent space of an autoencoder rather than on raw pixels, which makes high-resolution generation far cheaper. Stable Diffusion uses it.

What are state space models and Mamba?

Sequence models that process data with a recurrent state and scale linearly with length. Mamba adds selective state spaces and is an alternative or complement to attention.

Are these architectures combined?

Yes. Diffusion transformers use a transformer inside diffusion, latent diffusion uses an autoencoder, and hybrids like Jamba mix attention with state space layers.

How were these entries chosen?

They are the architectures most often cited in the literature, linking to this site's directories where they exist and to the original papers otherwise. Check current research, as the field changes quickly.