It is probably no coincidence that so many of the games we play today, particularly the team spectator sports that are now big business and cultural institutions, were codified within the span of a few decades during the second half of the 19th Century. An increase in the amount of leisure time for the people of Great Britain and the United States, one result of the Industrial Revolution taking place at the time, might have something to do with it. I'm sure that some study on the subject has attempted to narrow it down through historical study, but it's more likely that it was a complicated coming together of several social factors. Wikipedia credits the lawnmower. Because all of these sports coalesced into their recognizable forms around the same period, their respective history in the countries in which they took hold (or narrowly missed doing so) is easily traceable to a few simple decisions, crucial moments, or quirks of fate.
As a soccer fan, you're probably aware that the game's first effective set of formalized rules were laid down in the 1860's with the creation of The Football Association, a body formed out of a longstanding desire to standardize the various games called "football." Out of that event evolved what we now know as soccer (despite that first set of rules being somewhat rudimentary). At the most basic level, the decision taken in 1863 came down to whether or not the ball could be carried. Soccer was born out of choice to ban running with the ball.
Luckily for Americans and lovers of rugby, not everyone agreed with the decision.
Those that preferred carrying of the ball continued as they pleased, eventually leading to the codification of what is known as rugby. Although the two sports hardly seem related when viewed in their modern incarnations, the universities and clubs of England once treated them as the same basic game and took seriously their consideration of which rules to adopt so that matches could be played against others. With that simple decision in 1863, the two sports of rugby and soccer were sent on divergent paths; while we can't guess what might have happened had the FA founders chosen the rugby-style version, it's possible soccer's ascent to its current status as the world's game would have differed in some significant way. Additionally, had backers of the rugby-style rules simply accepted the predominance of association football and put down the ball, that sport might not exist.
And without rugby, the development of American football on our side of the Atlantic would have been impossible.
Ten years after English clubs attempted to standardize their football and the association game was officially born, Americans tried to do the same. Representatives from four universities met in New York City, decided on a set of rules for intercollegiate competition, and began to contest matches amongst themselves. Had it not been for Harvard abstaining from this meeting, continuing with their "Boston game" version that allowed for carrying of the ball (more accurately it was a hybrid of soccer and rugby), and eventually playing Montreal's McGill University in a match that involved rugby regulations the Massachusetts school subsequently fell in love with and then popularized with other universities, American football might never have come into existence. Had Harvard not been so stubborn, the people of the United States could have easily followed the rest of the world into a passionate love affair soccer in the closing quarter of the 19th century, rather than striking out on their own with a new version of football that, one hundred and twenty years later, no other country has taken to on a large scale.
Unlike in England, where the rejected code retained a strong measure of popularity, association football mostly disappeared as a sport in America.
Choices like the one made by Harvard and its peers repeated itself at schools across the country. The vast United States birthed colleges and universities at a rapid rate, and the students of those institutions took to playing competitive sport as a means of testing themselves or of bringing glory to their alma maters. In a few specific instances, soccer was the sport of choice before the wave of other schools playing American football forced a swap of codes. American football spread rapidly, became the ubiquitous intercollegiate sport so quickly that those playing soccer switched in short order to give themselves the opportunity to play against others. Sticking with soccer was a solitary pursuit.
Soccer lost out because a stubborn Harvard liked to play with their hands, a bunch of Canadians traveled down to give them a game where such a thing was encouraged, and other schools subsequently emulated the famous universities of the East. In a nation where distances between schools could be massive, choosing the same sport as your neighbors was a matter of practicality.
Such was the case at the University of Colorado. Founded in 1876, the school was an outpost of higher learning in a sparsely populated region, far from the Harvards and Princetons. When a group of young men formed the school's athletic association in the early 1890's, they did so at a time when American football's dominance was only just becoming a reality.
Below is an account from one of the founders of Colorado's football team, John C. Nixon. It's striking how Nixon makes clear that the group was much more inclined to soccer than rugby/American football, but that it was their desire to test themselves against other schools that pushed them towards the latter.
Forty-five years after Nixon and co. formed the Colorado football team, my grandfather played for the Buffaloes alongside future US Supreme Court Justice Byron "Whizzer" White. "Football" meant many things in late 19th century America, so much so that "football teams" prior to the turn of century probably played more than one kind of game carrying the label.
At the beginning of the first semester in the fall of ’90 the boys rooming at the dormitory on the campus of the U. of C. being afflicted with a super-abundance of penned up energy, or perhaps having recently drifted from under the parental wing and delighting in their newly found freedom, decided among other wild schemes, to form an athletic association. Messrs Carney, Whittaker, Layton and others, who at that time constituted a majority of the male population of the University, called a meeting of the campus boys in the old medical building. Nixon was elected president and Holden secretary of the association.
It was voted that the officers constitute a committee to provide uniform suits in which to play what was called “association football”. Suits of flannel were ultimately procured and paid for assessments on the members of the association and generous contributions from members of the faculty.
The University at that time had about the best base-ball team in the state composed of such star players as Ingram, Blake, Carnahan, Rust, Neighoff and others. But it was a reflection upon us that although it was styled the University base-ball team, it was composed mainly of outsiders, who had no connection with the U. of C.
It was the object of the newly organized association to ultimately absorb the base-ball team, even if it was mercilessly whipped in that, as it was afterwards scourged in football.
Whatever may have been said of their playing, the boys were made of that stuff called “American Grit,” and had no conception of what the word failure meant. To them defeat was simply a severe lesson which was to teach them how to win in the future.
Taking up the game of association football the boys practiced quite regularly and under the instruction of Hosford, who was an Englishman of extraordinary proficiency in the game, they soon became adepts in the art of running at full speed and keeping the ball almost fairly between the feet. I called this an art, as I consider excellence in any athletic sport an art. The graceful swing of a trained foot-runner is as fair a sight as the rhythmic waltzer, or as creditable a performance as that of an opera singer. After becoming proficient in this game we learned that all other football teams of the state played what was called the Rugby game, and as we could not attain state or national renown playing among ourselves, it became a question whether we would make a change or not. Only two in the association had ever seen a Rugby game and it was reported as fiendish beyond comprehension. The rush of a Rugby team was likened to the stampede of a herd of Texas cattle: nevertheless the boys decided to throw themselves in front and die a la Leon idas, and succeeding events showed that they had not over estimated the opposition.
The association had two factions—no live organization is ever harmonious—and the lesser faction always opposed the propositions of the majority. In this instance they had secured the individual promises of the eleven of the twenty four members to oppose a change of the game, and as it required the assent of two-thirds to accomplish this the remaining members decided to adopt the tactics of wearing out their opponents in the meeting. The session lasted one entire afternoon until darkness came upon them, during which, bursts of orator, pro and con, and imminent danger of pugilistic strife pervaded the atmosphere. The vote stood 16–8 in favor of a change and the University football team was launched on a broader basis. This vote was followed by our throwing the association open for membership to the whole student body, (and a limited number of outsiders, if we should need them), but time showed that the vim and snap of the college man was not possessed by those outside.
You can read the rest of Nixon's account here (.pdf).