Monthly Archives: August 2008

House prices: Statistics vs Reality

The chattering classes profess confusion over discrepancies in the various house price indexes. There are several indexes published regularly and closely watched, with plausible claims to statistical validity:

The Rightmove index represents asking prices, and is unsurprisingly much higher than any of the others – which represent actual sold prices. No problem there.

The Nationwide and Halifax are our long-term biggest mortgage lenders, and their indexes represent the sale price of houses purchased with a mortgage. Reassuringly, these indexes are closely correlated (graph: BBC News), so we can infer that any differences between the two lenders’ markets (who they lend to) are not important. That might not hold for smaller, specialist lenders, but we can surmise that these fairly represent the mainstream.

What seems to have the pundits baffled is why the Land Registry differs from the Nationwide and Halifax. There is a time lag, said to be around 3-4 months, built in to the Land Registry compared to the others. But that doesn’t explain the divergence we now see, as noted today by the FT.

It seems to me there is a perfectly simple explanation, and that we can extrapolate from it what will happen at a hypothetical turning point where confidence returns to the market. The issue is that the downturn is affecting different parts of the market in different ways.

Let’s consider a hypothesis with some plausible assumptions:

  • Tighter mortgage conditions have a disproportionate effect at the lower end of the market, particularly first-time-buyers. Higher up the market, rich people are less reliant on mortgages.
  • Therefore the reported 60%[1] drop in numbers of transactions is concentrated primarily at the lower end.

That’s enough. Let’s put some representative numbers to these assumptions[2]. The numbers themselves don’t matter: feel free to vary them, consistent with the assumptions. Your results will differ from mine, but they’ll still demonstrate why the observed discrepancy exists. Just to emphasize the point, we’ll take one figure way in excess of what any of the indexes tell us (of now): an actual drop of 20% across the entire market!

Applying that to some representative price points, we hypothesise:

  • First time buyer, down from £150000 to £120000
  • Mid-market, down from £250000 to £200000
  • High-end, down from £500000 to £400000

In “normal” times – before the crash – there’s little doubt that the majority of transactions were in the lower price ranges. Let’s say the above price points represent 60%, 35% and 5% respectively of the market. That gives us a pre-crash average of (0.6 * 150000 + 0.35 * 250000 + 0.05 * 500000) = £202500, which is roughly consistent with published figures (for sale prices, not asking prices).

Now we know that the crash has affected the bottom end disproportionately, and left the top end relatively intact. Let’s put some figures to that too, bearing in mind that the overall drop in activity is reported as being at least 60%[1]. Suppose activity levels are down by 70%, 50% and 10% at our three price points. That gives us an overall percentage drop of (70 * .6 + 50 * .35 + 10 * .05) = 60%.

Now, here’s the crux. Let’s calculate the average post-crash price with the above figures, bearing in mind that we have assumed each individual house is down by 20%. Our post-crash distribution of market segments have changed:

  • First time buyers: 60% reduced by 70% = 18% of the pre-crash market
  • Mid-market: 35% reduced by 50% = 17.5% of the pre-crash market
  • Top-end: 5% reduced by 10% = 4.5% of the pre-crash market.

That’s a total of just 40% of the pre-crash market (the 60% reduction). So what we see is different shape of post-crash market:

  • First-time buyers: 18% * 2.5 = 45%
  • Mid-Market: 17.5% * 2.5 = 43.75%
  • Top-End: 4.5% * 2.5 = 11.25%

So with the 20% drop in price of each individual house, we get an average of

(45 * 120000 + 43.75 * 200000 + 11.25 * 400000) / 100 = £186500

as compared to our pre-crash

(60 * 150000 + 35 * 250000 + 5 * 500000) / 100 = £202500

That’s a percentage fall of 100 * (202500 – 186500) / 202500 = 7.90%.

While individual houses have fallen by 20%, the market average – and hence the published statistics – has lost a mere 7.9% in our model. That’s actually smaller than the current falls reported by the Nationwide and Halifax!

Now my gut feeling is that the figures I’ve used may be conservative: the market skew may be much bigger than that (bearing in mind reports about first-time-buyers being near-eliminated, rather than 45% or the market as above). That would indeed be consistent with the mere 2% fall recorded in the Land Registry index.

Now it’s not hard to see how the discrepancy arises. The mortgage lenders see a different market profile to the land registry. We could perform a similar analysis with some more numbers: say 95% of first-time-buyers, 75% in the mid market, and 50% at the upper end have mortgage, we see their figures are biased towards the market sector that’s been most affected. I’ll leave it as an exercise to the reader to calculate hypothetical Nationwide/Halifax indexes based on those numbers (or choose your own).

Side-Effect: Rise of the Rental Market

A well-documented fallout from the crash is the rise of the rental market:

  • People unwilling to sell at current prices are letting their houses instead.
  • People waiting for further falls are choosing to rent for the time being, even those who could afford to buy.

So suddenly the rental market has changed. The quality has risen – with lots more houses than before that the owners thought good enough to live in themselves! And the status of tenants has risen too: it’s no longer so heavily dominated by those too poor to get a mortgage (and too honest to lie for one). And because the UK rental market is traditionally small (most people own their own home), the effect on it is disproportionately large. Even if the traditional rental market (the rich exploiting the poor) were little-changed, the overall market has risen with the coming of the new landlords and tenants.

Predicting the bottom of the market

Supposing this month, we were to hit the bottom of the market. Confidence suddenly returns. All the prospective buyers who are currently renting decide it’s time to buy.

  • The profile of the market returns to “normal”. The statistics catch up with the individual houses, so a 7.9% drop suddenly becomes a 20% drop in the published indexes.
  • Corollary: the sharpest adjustment to the Land Registry index. If it happens after more than a year of falls (so the indexes aren’t measuring from the top of the bubble) it will not merely catch up with, but overshoot, the Nationwide and Halifax indexes in terms of year-on-year percent falls.
  • Even so, the market doesn’t return to bubble-level, because the mortgage lenders have got burnt giving out pyramid-scheme money willy-nilly.
  • The top end falls off the rental market, leaving only the poor as tenants for all those buy-to-let landlords.

Clearly the key to that is the first point. Such a sudden fall in the indexes is going to kill of that returning confidence. Corollary: there will be no sudden return of confidence, now or anytime: it’ll be a gradual thing, with several years in the doldrums after the sharp falls have gone. That fits the pattern of past house price corrections, including the 1989-97 one[3].

The third is also interesting, as it could mean (far) more buy-to-let landlords in trouble with falling rents and far-more-fallen sale prices. That’ll be the point Bradford&Bingley (the specialised buy-to-let mortgage lender who just raised money on very unfavourable terms in a rights issue) will be in real trouble. With any luck, the Northern Rock fallout will be so visibly horrendous by then that the government of the day will have the guts not to pour in yet more taxpayers money to do the same for B&B.

So what will the house price statistics look like as the market bottoms? Well, the key is that activity has to return to the lower end of the market, and that’ll be gradual. But from the above analysis, we’ve got a visible sign. So long as the Land Registry index trails the Nationwide and Halifax (over and above the 3-4 month difference in reporting time), we can infer that real prices are falling ahead of any of the indexes. When the Land Registry starts catching up could be a good time to look for a bargain. Once it’s caught up, we’re back to a saner, and lower, market, and we can finally start to take the statistics at something closer to face value again.

[1] From memory, and very probably wrong. Not important – you can get a similar analysis with a different (large) percentage drop.

[2] This is the kind of analysis mathematicians do all the time: take a complex problem and get a handle on it by making simplifying assumptions. It’s useful in that it can provide a good insight into “what if” questions – how does it affect the overall picture if different inputs vary, or if our working assumptions are incorrect. My degree was in Maths, and my first professional job after graduating involved precisely this kind of operational analysis.

[3] Mathematicians will often prove a result by an approach of assume the contrary, and show that implies a logical contradiction. Mine isn’t a mathematical proof of anything, but it’s a similar kind of argument.

Fine fruits, Impulse buying

In the supermarket today, a watermelon caught my eye.  They’re a taste I acquired in Italy: a fine dessert for those mediterranean summer days!  Haven’t had one yet this year: like a Weizenbier or a Pinot Grigio wine, it only really comes into its element in that hot weather that just hasn’t happened this year.

Back home, I tried to put stuff in the fridge.  Including the watermelon, which is basically a solid drink, and nice when chilled.  Won’t fit on either main shelf.  Not even if I move them around to create the maximum space.  Bah.  Comes of having a single person’s fridge.  Oh well, there’s an excuse to chop it in half, and eat a slice.  And lo, it’s the best watermelon I’ve had since returning from Italy to the UK!

We seem to be having a great season for some of our native produce, too.  In the past week or two I’ve hugely enjoyed the first corn-on-the-cob and the first of our native victoria plums of the season.  The main disappointment is the blackberries, many of which are not good, possibly due to the exceptional amount of rain.  But a better September could still rescue them 🙂

Pot – kettle

Our foreign secretary is reported as warning Russia off starting a new cold war.  Or words to that effect.

Someone remind me: which two governments have been in the vanguard of every new international war since the end of the cold war?  And the underlying reason: which two economies are most dependent on their armaments industries?  Clue – it’s not Russia.

Someone remind me: who harbours and protects a billionaire from Yeltsin’s kleptocratic gangster era, who openly supports the violent overthrow of the russian government?  Clue: it’s not Afghanistan.

Someone remind me: who appears to be trying to present an either/or choice to countries like Ukraine, and making it politically hard for them to maintain friendships with both Russia and the West without the one prejudicing the other?

Pot – Kettle.

Note – I’m not making any comment for or against anything Russia may be doing (that’s the business of russian bloggers, not mine).  Just on the shameful hypocrisy of some nearer to home.

Cheese Fair

It’s that time of year again.  The Tavistock Cheese Fair at the town hall, organised by Country Cheeses.  A great chance for cheese-lovers to meet the best producers, and vice versa.

I went today, and encountered the expected mixture of old favourites and interesting new tastes.  Together, of course, with others I won’t be buying.  But I think I’ve accomplished what I set out to do: expand the list of cheeses I’ll make my regulars over the next year.


The downside is that because it’s popular, it’s way too crowded.  Something of a true Everyman[1] event, enjoyed by small children, oldies in wheelchairs, and everyone in between.  Still, this is one event that’s worth facing the crowds for.


[1] a term that encompasses both sexes – lest anyone with more politically-correct sensibilities than English comprehension think otherwise.


The term SWMBO normally signifies one to whom a chap has voluntarily given away his freedom. But have you noticed how a woman with whom you have no contract nor emotional relationship can have the same effect? Perhaps we should use a different term: SWWTNFAA (She who won’t take No for an answer). Just as deadly, but without the benefits.

So it is that this weekend I shall be singing with a quartet. Unrehearsed. And a quartet that isn’t really up to performing in public even if we’d practiced. We haven’t really had an ensemble fit to perform since our tenor, who was also (more importantly) the driving force behind the group, moved to another part of the country.

At least it’s just a couple of pieces to pad out what’s really an organ recital. Or so I’m told.

Bah, humbug.

BTW, if this post appears twice, it’s because I wrote it yesterday evening. Today WordPress seems to have lost it, but had most of it saved as a draft, so I’m just re-publishing.

p.p.s.  WordPress seems to let me edit the publication date of an article.  Just tried it here to put it at the right time.  But I’m a little concerned that it worked, as it rather diminishes the blog’s value as a historical record of who said what, and when.


Today I have, as ever in the autumn season, stung and shredded hands.  For I have been out blackberrying and collected a decent crop.  Now that the season’s started, I expect to be out there regularly over the next couple of months.  As of now they’re suitable for cooking rather than eating, but this lot should be sufficient for both a crumble now and a batch saved in the freezer.  Not quite enough as yet to make chutney, unless I forgo the crumble.  That comes in a few weeks when the season is at its height.  Or when I can be arsed to go further afield to collect them (as I shall have to do by about mid-September, as the stupid council chop them down locally).

Actually these aren’t quite the first of the season.  I collected a few for my visit to the family.  But the amount available then was a token rather than a serious pick of ’em, and they fell just short of filling what had originally been a pickle jar: about a pint.

Last year we had a good but early season, helped by warm and sunny weather for about three months of autumn after the very wet summer.  This year there’s no sign of that, just a continuation of the wet season.  I think we may’ve had even more summer rain this year than last.


I blogged about mod_sed a couple of months ago.  At the time, I think I explained in outline why it’s an advance on its predecessors, including mod_line_edit and mod_substitute.  But one key part was missing: any discussion of how to take advantage of sed’s capabilities, beyond the simple string or regexp search-and-replace of the earlier modules.  That matters because few of us are power-users of sed who know what we can do with it, and even those who are may not realise how original-sed translates to mod_sed.

Now Basant has filled that gap, with a blog entry describing advanced usage.  He shows a range of programmatic usage of sed within the filter, starting with branches and conditionals, and moving on to block-structuring and advanced buffer manipulation.  For those who want advanced text manipulation in an Apache filter, this is a great kick-start to learning what sed can do for you.

Lots of family

My brother miq and his wife Sock-Cheng are in Blighty for a visit.  Since they live in Auckland (S-C being a kiwi), this is a rare event.  So a family reunion, c/o our parents, was quite a big thing.  I travelled down on Friday, and my other brother (no webpage, but he’s best known for XeTeX) with his wife and three children joined us a couple of days later.  It’s a long time since I’ve seen them, too.

Great to see them all, of course.  At the same time, ten people in the parents’ shoe-box of a three-bedroom house (2+attic) is challenging.  I got the privilege of sleeping out on the balcony.  Yes, it was my choice (the alternative being less attractive), and yes, it rained heavily.  Fortunately the awning over it kept me mostly dry.  And it was just the one night with everyone there.

Unsurprisingly, the adults present (my generation and our parents) are much as I remember them, but the children are changed.  I was pleasantly surprised by the latter.  Partly from a purely selfish point of view, they’re no longer of an age to do things like come and bounce on their long-suffering uncle at some ungodly hour of the morning.  But also because they seemed bright and mature.  Both the boys (17 and 14) show strong geekish tendencies, and the older looks ripe for the student life at (hopefully) one of our best universities next year.  The youngest (11) is not just fractionally taller than her aunt-by-marriage S-C, but also plenty mature enough to treat as an equal rather than talk down to as a child.

No doubt they were on best behaviour.  It worked.

All the best to miq and S-C for the remainder of their time in Blighty.  And to everyone.

Solar Energy: Handouts for the Rich

Today’s junkmail: someone providing solar energy for the home.  Oh, and the private swimming pool.  With big subsidies: up to 100% on selected promotional properties, and government subsidies on others.  Great!  Clean energy on the very-cheap.

Unless of course you’re stuck with renting your home, and any such investment would be in the hands of your landlord.  Yep, like so many benefits, this one excludes the poorest 30% (or whatever) of the population, who just get the privilege of subsidising richer folks.  OTOH, rather pay for their solar power than their excessive consumption of dirty energy.  Except – we get to do that too, through a range of payments, as well as tax-breaks like low-rate VAT.

How’s this for a radical suggestion.  Increase solar-energy microgeneration while at the same time benefitting poorer folks by legislating for its installation in rented property.  Add ever-tougher energy standards as mandatory requirements when letting a property.  To include basic solar panels for any property with a south-facing area of roof.  In the case of flats, these should feed communal (but metered) hot water supplies.  Nah, dream on …


My slice is up-and-running, and all major services appear to work, though there’ll doubtless be glitches.  Yesterday I updated DNS to point to it, after Richard contacted me to say they’re clearing out the old datacentre over the weekend.  This morning, DNS has propagated to my ISP and no doubt much of the ‘net, so next time you contact anything I run, it’ll be on the new slice.

There were a couple of minor panics in setting it up, when things didn’t compile first time.  libhtnorm (the backend for AccessValet) was an unexpected scare when it showed a bunch of unresolved C++ symbols.  But it turned out to be just the linker that was different, so it worked fine when I explicitly loaded  Other Site Valet tools required some very minor troubleshooting, but only at a sysop level (no programming).  My main fear proved unfounded as mod_validator required nothing more exciting than the latest Xerces package and an OpenSP build with the right options.  ApacheTutor also needed some trivial work, to compile mod_xmlns against the expat version installed on the new slice.

I’m still thinking about how best to add a note to pages served, and invite users to report anything that’s broken in the move.  Of course I can use mod_publisher to insert a notice, and the stumbling block is to work out the page design with the notice in for each site affected.  All my sites need an overhaul anyway.

Anyway, it’s farewell to Openia, who have done a great job hosting the server over several years.  A special thanks to them for sponsoring it when WebThing was struggling with no money.