So the weights don't literally contain pictures of perfect …

So the weights don't literally contain pictures of perfect cats - instead they encode the mathematical relationships and patterns that define "catness" in abstract informational space. It's like the network discovers the platonic form through learning statistical regularities across many examples.