Entropy logopeterindia.net
Knowledge Hub

Entropy Web Resources

Entropy as uncertainty, disorder and randomness — three related ideas across information theory, physics and computing.

21 topics•5 categories•Updated October 2026

Information Theory

5 entries

Shannon entropy and its applications.

01A Mathematical Theory of Communication (Shannon)Shannon's 1948 paper that introduced information entropy. 02Entropy (information theory) (Wikipedia)Definition, properties and examples. 03Information Theory, Inference, and Learning Algorithms (MacKay)Free book linking information theory, probability and learning. 04MIT OpenCourseWare 6.441 Information TheoryGraduate course. 05Visual Information Theory (Christopher Olah)Visual explanation of entropy, cross-entropy and KL divergence.

Thermodynamics & Physics

5 entries

Entropy in physical systems.

06Entropy (Wikipedia)Overview across thermodynamics and statistical mechanics. 07Entropy (HyperPhysics)Concept map entry from Georgia State University. 08Energy and Entropy resources (The Physics Hypertextbook)Resources on the second law of thermodynamics. 09Landauer's principle (Wikipedia)The minimum energy cost of erasing a bit of information. 10Entropy, Environment and Resources (Springer)Springer book on entropy in resource and environmental contexts.

Entropy in Computing

4 entries

Randomness for security and software.

11Entropy (computing) (Wikipedia)Randomness collected by an operating system for cryptography. 12/dev/random (Wikipedia)How Unix-like systems provide random numbers. 13NIST SP 800-90B: Entropy SourcesNIST recommendation for validating entropy sources. 14Sources of Entropy (Javamex)Tutorial on where computers get randomness.

Journals

1 entries

Research on entropy.

15Entropy (MDPI)Open-access journal on entropy and information.

Related Pages

6 entries

Neighbouring pages on this site.

16MathematicsMathematics hub. 17Probability Theory for AIProbability for AI. 18CryptographyCryptography resources. 19PhysicsPhysics resources. 20Complexity TheoryComputational complexity resources. 21Theoretical Computer ScienceTheoretical computer science hub.

Frequently Asked Questions

What is entropy?

The word covers related ideas: in information theory, a measure of uncertainty in a random variable; in thermodynamics, a measure of how energy is dispersed among microscopic states; and in computing, the unpredictability collected to seed random number generators.

How is entropy used in machine learning?

Cross-entropy is a standard loss function for classification, and KL divergence measures the difference between probability distributions. Both are built on Shannon entropy.

What is Landauer's principle?

It states that erasing one bit of information in a computer must release at least a minimum amount of heat, tying information to thermodynamics.

How current is this page?

Links were checked in October 2026. Publisher and journal sites that block automated checks are kept and may need a manual click. This page lists resources rather than teaching the subject.