There are many reasons for this and among them is the rising cost of land and the development of that land. State and federal requirements for environmental controls as well as local planning are leading to an inflated price for build able lots. This gives developers an incentive to place the largest house possible on the smallest lot possible. They have to maximize the land use to maximize profitability. This means that buyers seeking the classic big ranch on a huge lot will either have to dig ridiculously deep into the bank account or seek out an older resale home. Even the latter however will be expensive.
Young people however are more likely to feel forced to accept the notion of a townhouse, condo, or a smallish house on a really tight lot. Millennial buyers are among this group and they a by far the largest buying block in the country today.
Vancouver, Washington is among the many US cities that is actively promoting the infill style. It is good for cities as they can concentrate public services and associated private commercial development to create a more manageable land use program with less infrastructure and better traffic. In the picture at top left, a snapshot from Google Earth shows a typical infill development of large homes stuffed onto small lots in and older established neighborhood. There are 16 homes with sizes of around 2500-3200 SF on about 2 acres of land. Adjacent is the old 1970s neighborhood
with 8 houses on roughly 2.5 acres.
Vancouver, WA has a large urban growth boundary that wraps around the existing city and totals roughly 85-90 square miles of area. Within that area are approximately 300,000 people already. With infill the area could easily support another 50-100,000 more over the next few decades.
Although this style of land use has many benefits to community planning and urban living, it has a major side effect. It tends to drive land values and development costs higher. This has certainly been the case in the Portland Metro Area over that last few decades. Restrictive lands use policies have lead this area to go from having homes price at roughly the national average in the 1980s to nearly double the national average today. Some may argue that land use planning has only been part of the issue, and they may be right, but there is no doubt a direct correlation.
So this trend is likely going to continue and in likely to get even worse as the side effects of sprawl are measure against the side effects of infill. Infill will almost certainly win the war.