Random Pertubations

Imagine M&Ms divided into four groups by color on a sheet of graph paper. The brown M&Ms sit on the positive side of the Y axis, the reds sit on the negative side of the Y axis, the blues sit on the positive side of the X axis, the greens sit on the negative side of the X axis. All other colors are absent.

This model represents any structured system. A human cell, a car engine, a software microservice. Pushing any of the M&Ms deeper into its own color group would increase the inherent structure, pushing any of the M&Ms further away from its color group would decrease it.

Most of the things we humans call “the world” depend on a large degree of structure to function properly. The most amazing devices — microchips with billions of individual transistors per square inch for example — tend to rely on the largest degree of structure.

If you were to randomly perturb an M&M, pushing it in any of the four possible directions on a 2D graph, one of those directions would push it deeper into its cluster, in the direction of increased structure, three would push it away from its cluster. Thus there is a 75% chance this maneuver would decrease the structure of the overall system.

The number “4” comes from the dimensionality of the system. The number of dimensions times two, in this case for a two-dimensional graph. The real world is far higher-dimensional. As a very rough proxy, CLIP embeddings, representing all possible images, have 1024 dimensions. In such a space, one direction pushes deeper into the vector for a given M&M color, 1023 push away, for a 99.902% chance of a random pertubation decreasing the structure of the system.

In our world, any powerful entity can push things around. When things are pushed, they are far more likely to be pushed in the direction of decreased structure than in the direction of increased structure. This is countered by the fact that the most important human decisions tend to be made with significant care, and these decisions don’t all have to be right. The higher dimensional the system, the more carefully the direction must be chosen to maintain or increase structure, as the lower the number of options that increase structure. When structure decreases in our world, it tends not to go well.