Hierarchies occur widely in evolving self‐organizing ecological, biological, technological, and social networks, but detecting and comparing hierarchies is difficult. Here we present a metric and technique to quantitatively assess the extent to which self‐organizing directed networks exhibit a flow hierarchy. Flow hierarchy is a commonly observed but theoretically overlooked form of hierarchy in networks. We show that the ecological, neurobiological, economic, and information processing networks are generally more hierarchical than their comparable random networks. We further discovered that hierarchy degree has (...) increased over the course of the evolution of Linux kernels. Taken together, our results suggest that hierarchy is a central organizing feature of real‐world evolving networks, and the measurement of hierarchy opens the way to understand the structural regimes and evolutionary patterns of self‐organizing networks. Our measurement technique makes it possible to objectively compare hierarchies of different networks and of different evolutionary stages of a single network, and compare evolving patterns of different networks. It can be applied to various complex systems, which can be represented as directed networks. (shrink)