I think our only real hope for survival and prosperity is learning to cooperate nonviolently — not just without violence and threats in day-to-day life, but without even condoning state violence such as war, arrests, and imprisonment. Historically, state violence has been far from sufficient to organize society; society's functioning rested primarily on nonviolent day-to-day interactions, what Gandhi called satyagraha, primarily between people who know each other in small communities.
Mostly starting in the 20th century, although with antecedents in historical episodes like the Roman Dominate, we've seen a dangerous set of experiments in organizing society primarily around the state, which is to say around violence. While these experiments have produced some promising results (the extermination of polio, space travel, dramatically improved agricultural yields, the internet) I hope we can move beyond them to a peer-to-peer global society. Otherwise, it seems that we are doomed either to live under a global state — which we can hope against hope will not be particularly despotic — or perish in a nuclear holocaust or similar tragedy.
Nevertheless, it remains true that through most of history, we've been under the threat of violence from other communities. Warfare and warriors have been a constant plague on humanity since prehistory, and historically, we have had no hope of living without violence, only of prevailing in it.
And, historically, this violence has been mostly two-dimensional: warfare and warriors moved around on the ground. This gives rise to a minimal stable size for an agricultural settlement, because the circumference of a two-dimensional shape increases more slowly than its area as the shape grows.
A farmer cannot defend his own field from looters, armed or otherwise. It might require 100m² or 1000m² of land to feed one farmer, depending on the climate, your cultivars, your ability to trade, and so on; but even 100m² of land has a minimal circumference of some 35 meters. One farmer might be able to defend one or two or ten meters of circumference, even while he's awake, but not 35.
But a circle with a circumference of 70 meters has an area of 400m², enough to feed, hypothetically, four farmers. That leaves each of them only needing to defend 18 meters of circumference instead of 35. Still too much, but perhaps moving in the right direction.
If each farmer can defend ten meters, your minimal size is almost twice that, or 130 meters of circumference, and 13 farmers, with their wagons or houses circled in a 20-meter circle around 1300m² of land.
If each farmer can defend two meters of circumference, you have 315 farmers on a circle 200 meters across with a total area of 31500 square meters, about three city blocks, three hectares, or eight acres. This is beyond Dunbar's Number, and so at this point you start needing institutions, formal hierarchy, and so on. Two meters is small enough that you can have night sentries who wake everybody else up if they see or hear anything, and dense enough that you can't steal the farmers' crops by yourself — you need a raid by an armed and organized band of bandits.
100m² is a really small farm. It's not enough to support a person except in the most fertile parts of the world, with a lot of luck. 1500m² might be a more realistic estimate in most of the world. If you need 1500m² per person, but each person can defend only five meters, your minimum community size is 754 farmers, 600 meters across, with an area of a million square meters (100 hectares).
Of course, in real life, the farmers don't live on the outer borders of the farmland; they only go there when there's a raid or the threat of a raid from another tribe, and they keep what is most precious to them — their lives and those of their families — in a much smaller area that's easier to defend than their entire fields. But there still need to be enough of them to keep a watch on the border, and to repel the raiders when they raid.
I think this is why the birth of agriculture led to the stratification of society. It's not, as many have said, that agriculture makes it possible, for the first time, to produce enough surplus to support a richer class; observations of contemporary hunter-gatherers show that, even though they're living on the most marginal lands, they still only work for survival a small amount of the time. And it's not just that agriculture makes you store up your harvest in a granary where it can be stolen. It's that keeping hunter-gatherers from gathering what's in your field requires you to organize into groups that are bigger than Dunbar's Number, and the more organized the hunter-gatherers or pastoralists or invading agriculturalists are, the bigger your community needs to be to repel them — even at the cost of enabling parasitic warrior-kings.
To put it more plainly, in such a world, if your farmers aren't willing to die for the sake of people they don't really know, if they aren't willing to sacrifice their lives for an abstraction, if they aren't willing to be sent to their deaths by a general, then raiders will take their food and their children will starve. But the general or king who chooses who to send to their death will not send his own sons first, and if you do not pay him tribute, he can send his loyal subjects to take it from you by force. So stratification is born.
If this explanation holds water, we'd expect to see:
I think these do exist, although I'm not sure — many enormous empires with great stratification have surely existed in very fertile places.
(The explicatory power of this area-to-perimeter ratio thing is perhaps somewhat dubious: surely in most of history the greatest determinant of the size of any given state has been the size of neighboring states, no?)
What about new technology?
People only need to eat 100–120W of food, or 2000–2500kcal/day, and the solar resource in most of the world is greater than that per square meter — in the US, it's mostly 3–6kWh/day/m², which is 120–250W/m². The reason you need hundreds or thousands of square meters per person for agriculture is that, first, natural photosynthesis is only about 3% efficient even when not resource-limited; second, plants spend most of their energy on growth and reproduction, not on feeding you; and, of course, you need micronutrients.
But what if you had a 60%-efficient electric solar collector, powering a 30%-efficient synthetic sugar-and-protein plant? Then you could turn 18% of the sunlight you caught into edible calories. If you had access to a 250W/m² solar resource, you could get by on 2⅔m². Remember how, before, the need for one defending farmer per five meters of perimeter forced the farmers to band together into groups of 754? Now you can get that same level of security with 2 "farmers" guarding a 1.3-meter-diameter circle. You don't hit Dunbar's Number until you have 135 "farmers" guarding their shared 10.7-meter-across solar plant, half a meter apart.
One person per 2⅔m² is about 370 000 people per square kilometer, which is more than ten times the density of the densest cities in the world, such as Delhi, or Manhattan, or Friendship Village, Maryland.
That is, people eating synthetic macronutrients out of vats could establish population densities that exceed the population densities of our current agriculture-based society by as huge a ratio as ours exceeded the density of the pastoralist and hunter-gatherer societies it has defeated. And, perhaps, they would have no need to establish hierarchical states with unaccountable rulers, simply in order to be able to protect their crops from raids; they could do it with tribes of hunter-gatherer scale.
None of this is quite feasible yet, neither 50%-efficient solar-energy collection (the current state of the art is about 40%, or 5% at the lowest cost per watt) nor efficient macronutrient synthesis. However, they're both clearly technically achievable.
(You might think that it would be pretty uncomfortable to have only 2⅔ m², 29 square feet, per person, like hanging out in a crowded supermarket for your entire life; but you could build buildings many stories tall, to provide each person with ample living space.)
(Another caveat: as Charlie Manson, David Koresh, and the like have amply demonstrated, living in a hunter-gatherer-sized autonomous band is no guarantee of living in peace or hunter-gatherer-like egalitarianism.)
The United States contains 9.8 million km². If it were populated at 370k people/km², it would contain 3.6 trillion people, 500 times the current population of the Earth. At a twentieth of that density, it would contain 180 billion people, all living with a material standard of living comparable to current US society. The corresponding numbers for Argentina are 2.8 million km², 1.04 trillion people, and 21 billion people. For Earth, 150 million km², 56 trillion people, and 2.8 trillion people, about 7900 or 400 times the current population.
At the current population growth rate of 1.1% per year, we'd reach those population benchmarks in the years 2561 and 2835; but if the world population were to grow at Qatar's 4.9%, which we know is possible, we'd reach them in years 2138 and 2201. In-vitro gestation would make much higher population growth rates possible.
(Why a twentieth? Because it brings the total solar power available to each person to some 5–10kW, or 2½-5kW after conversion to electricity, putting them on par with modern US consumption of some 10.4kW per person.)
Given the extent to which modern cities are tolerant of pluralism, and the compatibility of growing vat food with city life, it's probably not realistic to imagine the eruption of such a new lifestyle in separated communities; rather, people inside of cities will buy, separately, solar energy harvesting devices and macronutrient-growing vats, probably in both cases mostly as emergency fallbacks; and vat food won't become popular until a generation comes to maturity that grew up eating it, probably due to a political and economic catastrophe that forced them to grow up in poverty, like Spam and horrible boiled vegetables in England after World War II.
Furthermore, vat-growing food won't become a popular thing to do as a hobby, like bean sprouts or yogurt-making, unless it becomes popular first as an industrial-scale product, or unless there are regulatory and political reasons people can't do it on an industrial scale.
Quite aside from the purely factual questions considered above — how vat food could become established, what population it could support, where it would likely take root, and so on — there is the normative question to consider: is a vat-food future dystopic?
Hierarchical or not, it sounds dystopic to me.
Instead of increasing agricultural productivity per square meter, consider the possibility of extending each farmer's defensive powers. Today drone pilots in the US armed forces routinely control four drones at once, each capable of surveillance and unaccountable targeted violence over a very large geographical area.
If each farmer has at his command some kind of robots, perhaps he could use the robots, instead of his neighbors, to guard his garden against raids. In a sense, this is what happened with the enclosure of the Old West: instead of robots, ranchers installed barbed-wire fences ("bob wahr") to keep their cattle from wandering off, converting their rustler-killing, Native-American-battling pastoralism into a kind of agriculturalism. It didn't stop rustlers, but it slowed them down enough, and it did stop the buffalo.
Now, it seems eminently plausible that one person with a bunch of video cameras and, say, remotely triggered Tasers, chains, salt-water squirters, mines, and so on, could police 1500m² of land, which after all has a perimeter of only 138 meters, against raids. His neighbor, or his neighbor's robots, wouldn't be able to step onto his land to dig his potatoes without his permission.
But could one person really put all of that in place? Maybe not — but it seems clear that one person with modern surveillance, computation, and weapons could defend a larger perimeter than one person without.
So far, though, we've seen a countervailing trend: the increasing division of labor, and thus specialization, necessary to produce the computing devices is deeply bound up with the current world system of enormous states with millions to hundreds of millions of citizens or subjects. And while current machinery may not make it particularly easier to steal potatoes, it certainly makes it a lot easier to cut chains and blow up mines, enabling attackers as well as defenders — especially attackers using the power of large states with hundreds of millions of members.