Astronomers think the icy objects of the Kuiper Belt are remnants left over from the formation of the solar system. Similar to the relationship between the main asteroid belt and Jupiter, it's a region of objects that might have come together to form a planet had Neptune not been there. Instead, Neptune's gravity stirred up this region of space so much that the small, icy objects there weren't able to coalesce into a large planet.
The amount of material in the Kuiper Belt today might be just a small fraction of what was originally there. According to one well-supported theory, the shifting orbits of the four giant planets (Jupiter, Saturn, Uranus and Neptune) could have caused most of the original material, likely 7 to 10 times the mass of Earth, to be lost.
The basic idea is that early in the solar system's history, Uranus and Neptune were forced to orbit farther from the Sun due shifts in the orbits of Jupiter and Saturn. As they drifted farther outward, they passed through the dense disk of small, icy bodies left over after the giant planets formed. Neptune's orbit was the farthest out, and its gravity bent the paths of countless icy bodies inward toward the other giants. Jupiter ultimately slingshotted most of these icy bodies either into extremely distant orbits (to form the Oort Cloud) or out of the solar system entirely. As Neptune tossed icy objects sunward, this caused its own orbit to drift even farther out, and its gravitational influence forced remaining icy objects into the range of locations where we find them in the Kuiper Belt.
Today the Kuiper Belt is slowly eroding itself away. Objects that remain there occasionally collide, producing smaller objects fragmented by the collision, sometimes comets and also dust that's blown out of the solar system by the solar wind.