In Europe, nationalization of the rail networks, and the national carriers' passenger focus, helped hide this. European freight services were often loss-leaders, competing head-to-head against trucking.
Meanwhile, in the United States, a move was made to transport bulk goods, and transport them effectively. The advent of containerization allowed general freight to be shipped cross-country as bulk loads, originating in Los Angeles and terminating in New York. Coal unit trains, trundling between mine and power plant, often never need classification. Nationalization of passenger service, relaxation of shipping-rate regulations, and consolidation in the railroad sector -- a process that had begun during the Long Depression, but stalled in the cabal era before it began again apace postwar -- all had a hand to play in this process, and today the United States is often cited as the most efficient freight railroad network in the world, its companies posting record profits.
Contrast this with Europe. The development of high-speed rail has opened more space on the classic lines, but often freight operators -- usually the national rail companies' red-headed step-children -- have failed to take advantage of the improvements. Part of this is, of course, historical. Sealanes have historically controlled Europe's bulk-shipment sector, and until the end of the Cold War, most of the overland bulk routes would have run through Warsaw Pact territory. But the Cold War is twenty years over, and most of the countries along the best bulk routes have been integrated into the European Union, with accession work being done for the remainder (most importantly here: Serbia).
Actually, the biggest structural problem Europe now has is the legacy of rail nationalization. Picture, for a moment, if you will, an alternate United States where every individual state nationalized its internal railroad network. Today, a container bound from Los Angeles to New York requires all of one interchange*, but over its journey it'll cross through eleven states**. If all of those states had their own rail networks, this same shipment would have had to interchange no fewer than ten times, and most likely have to be reclassified multiple times, despite intermodal trains' crack nature.
That is the structural problem, the reason why European freight rail is less efficient and less profitable than American freight rail. Only in a small handful of corridors where reclassification needs are limited (if not outright nonexistent) -- say Calais-Marseille, entirely in SNCF territory, or Rotterdam-Genoa via the Gotthard Pass or Hamburg-Trieste via the Brenner Pass (three countries each) -- can bulk freight begin to become possible. And while each of these routes offers a significant advantage to running around the Bay of Biscay and through the Strait of Gibraltar -- Trieste most of all, as ships would also have to head around the boot of Italy and up the Adriatic Sea -- greater advantages accrue the further east you go, as the maritime detours become longer.
Rotterdam-Istanbul, as an intermodal-freight mainline, makes all kinds of sense. The overland route is around 2.5 times shorter than the maritime one***, most of the mainline (by and large, the old Orient Express route -- yes, that Orient Express) lies in EU member states' territories, and Istanbul's historic and modern role as global entrepot makes it an obvious terminal for Suezmax container ships from China -- a much shorter journey than going all the way around Africa! -- as well as the principal port-of-call for Turkish manufacturing. Rotterdam is, obviously, Western Europe's Port of New York.
But the problem is that -- to get from Istanbul to Rotterdam -- you have to go through Turkey, Bulgaria, Serbia, Hungary, Austria, Germany, and the Netherlands -- seven countries in all! All of which have their own national rail networks! And so even a crack express intermodal is stalled by interchange after interchange after interchange before it gets where it needs to go. The enormous efficiencies of Europe's mainline rail network -- large-scale mainline electrification, for example -- are completely gobbled up and wasted despite rail's natural geographical advantage.
There are a few ways to solve this problem. One possible solution would be a return to the common-carrier model, where a given railroad is expected to offer equal passenger and freight service. For example, Britain's London Northeastern Railway (LNER) made most of its money -- about two-thirds of it -- shipping coal. In a common-carrier scenario, railroads would privatize, and then begin the consolidation process into common-carrier systems.
The problem with this is, of course, that the common-carrier model failed. The developed world's highly unbalanced railroad networks, favoring freight in North America and passenger in Europe, are an artifact of this failure, one which occurred because common-carrier railroads, in practice, used profits from freight operations to cross-subsidize passenger operations, a practice which remained effective only as long as railroads held a monopoly on overland travel.
The other possible solution, which began to be experimented with in British Rail's privatization, is the open-access model. Unlike under the common-carrier model, in which a railroad is responsible for both the infrastructure and services offered on that infrastructure, in an open-access model, one entity is responsible for the infrastructure -- the train tracks, bridges, tunnels, cuts and embankments, electrification, signalling, dispatch, etc. -- and it sells slots, or time windows for operations, to various service providers (what are, in the UK, called "train operating companies" or TOCs). In a way, the open-access model functions not unlike a toll road: you pay to access and use the infrastructure provider's network.
The problem with is is that -- as with many forms of for-profit infrastructure -- profit comes from not providing quite what you said you would provide. This is the issue with American healthcare Obamacare set out to remedy, why Enron was able to deliberately underpower Southern California to jack up rates ... and why, less than a decade after British Rail's privatization, the for-profit infrastructure provider Railtrack was partially renationalized as the nonprofit Network Rail.^
When tracks are owned, and services provided, by the same entity, the pressure to skimp on maintenance is moderated by a need to avoid accidents and deliver to the customer exactly what one said one would -- because, if you don't, you lose your customers and hence your revenue. Nevertheless, American mainline railroad maintenance (never mind branch lines!) is inferior -- by an order of magnitude -- to its European counterparts, lagging behind not just in obvious things like electrification but less obvious things like signalling. American trains are exceptionally heavy, and seem to have spent the past thirty years adding more and more weight in an effort to increase long-haul tractive power at speed at the cost of acceleration: trains so heavy rail spreading is a surprisingly common occurrence, as are the derailments it causes.
(Nevertheless, the partially-private (it is publicly-traded but majority-owned by the Canton of Bern) BLS Railway is making a game attempt to revive the common-carrier model in Europe.)
The British experience shows that the actual provision of rail infrastructure most certainly is in the public interest -- not just because rail disasters tend to be low-frequency high-impact events, but also because the frequency of rail accidents (and hence disasters) is, for obvious reasons, negatively correlated to increased infrastructure spending. In fact, I'd go as far as to say that the organization or agency charged with delivering rail infrastructure must be nonprofit but also must be removed from the public sector enough to not be interrupted by meddling politicians looking for pork for their constituents over that of the state's needs (in this, Network Rail's increasingly cozy relationship with its "public" side is worrisome).
But even if you accept the British model -- of a nonprofit infrastructure provider selling slots to TOCs -- as the closest thing we have to an "open access" paradigm, and seek to implement it across Europe, you've set up a situation where bulk freight can work: schedule a cascade of freight slots for your trains up and down the line. But it's a fragile system. What happens if your train runs late (say because of transshipment irregularities at the port)? If you miss your slot, that's it, somebody else's got the next slot, and either you've got to pay a hell of a premium to schedule the next empty slot (assuming it exists) or wait for the next one you've scheduled, which means you've also got to run two trains through one slot, which may or may not be possible, depending on train length and slot density. And all this time your supposedly crack intermodal train is waiting in an interchange yard in
Clearly, no matter how you structure it, you have to decrease the count of infrastructure providers as much as possible. Thirty different providers in Europe is just as bad as fifty different ones in the United States would be. If every state in the Northeast provided its own rail infrastructure, every single Acela run would have to involve the successful cascade of nine service slots (DC, Maryland, Delaware, Pennsylvania, New Jersey, New York, Rhode Island, and Massachusetts). The same would hold true for an express freight route. Indeed, the unwieldy nature of such a construction was recognized so early that the Pennsylvanian, Delawarean, and Marylander companies allied with each other to construct a Philadelphia-Baltimore railroad had already merged together by 1840.
For the sake of all users, it turns out that, when providing moderate- to long-distance freight and passenger rail^^, political borders are sort of a mirage. Far more effective, for our intermodal railroader, would be a infrastructure provider that sells a single slot all the way from Istanbul to Rotterdam (or from LA to New York, for that matter). But at the same time, that is far less effective for passenger services, which, it seems, generally need a relatively compact infrastructure operator responsive to their needs.
And suddenly we run into a strange optimization quandry. Infrastructure providers at the state level^^^ are generally better for passenger rail, save in places like the Northeast where the conurbation has gobbled a bunch of states up, while those at the level of large regions favor freight rail. It is telling there are only seven Class I railroads in the US: two serving the West, two the East, two Canada (both with significant US holdings), and ... the Kansas City Southern. Contrast this to the height of the common-carrier era, when there were around a hundred distinct Class I railroads*^. Even today, providing rail infrastructure at the regional, rather than the state level, makes more sense; the Class I's were all large regional carriers, serving multiple distinct regions: the Northeast and Midwest, for example, or California and the Far West.
But if you broke the United States' rail infrastructure up into its natural provision regions -- the Northeast, Midwest, South, Greater Texas, Interior West, California, and the Cascades -- you suddenly have the problem of an increased interchange count and hence an increased fragility w/r/t slot timing for transcontinental bulk shipments. This is less of a problem in Europe, where regional providers would remove some of the interchanges and hence headache, especially in the Balkans.
No matter how you slice it, though, it is clear the open-access model has both advantages and disadvantages. Already, DB Schenker has acquired half of Western Europe's freight providers, locking down all the ones east of the French border between the North and Adriatic seas. The stirrings of transcontinental-scale freight rail are beginning again in the EU, just as surely as the stirrings for better passenger rail are beginning again in the United States.
The only real questions are: Who will own and maintain the physical infrastructure? And will their control over it be a help or hindrance?
* It is fair to point out that early intermodal services from the Port of San Francisco to New York via the Alphabet Route would have required up to six interchanges (e.g. WP-D&RGW-CBQ-NKP-WM-RDG-CNJ), but the vast majority of transcontinental American shipments have never required more than one interchange (usually in Chicago) ... except to/from San Francisco, which until very recently would have required two interchanges west of the Mississippi.
** Say, via the BNSF to NS: California - Arizona - New Mexico - Colorado/Oklahoma (forget which, it's in the damn corner) - Kansas - Missouri - Illinois - Indiana - Ohio - Pennsylvania - New Jersey.
*** And, for those of you who want to point out you can run a barge up the Rhine and down the Danube instead, bulk rail tends to compete toe-to-toe with river shipments. If that weren't the case, the Illinois Central Gulf, whose mainline duplicates the Mississippi, wouldn't have been so profitable the Canadian National bought it!
Incidentally, the land route between Los Angeles and New York is only about twice as long as the sea route via the Panama Canal.
^ A less conscientious observer might point out that an awful lot of this sounds a lot like fraud.
^^ And rail service of any sort, really.
^^^ I've been using the term "state" with the tacit understanding that, for the purposes of this discussion, an EU member "state" is only moderately more sovereign than a US "state". Or, not to put too fine a point on it, at the transcontinental level, for infrastructural purposes, EU member states and US states are more alike than they are not.
*^ That said, many of the Class I's of the era wouldn't count today. None of the New England railroads then would have had the inflation-adjusted income figures (just as they don't today), for example; trying to figure out which 1950s Class I railroads would still count as such in 2010 would be a tedious and largely pointless process. But, no matter how many would get kicked down to Class II status, it is clear that there were an order of magnitude more then than there are today.