sed in apache

We have a history of general-purpose sed-like filtering in apache.  In chronological order:

1. sed-like filter for Apache 1.3.
2. sed with mod_ext_filter
3. mod_line_edit
4. mod_substitute
5. mod_sed

This represents a genuine progression. The first is limited by the apache 1.x architecture, which means it can’t in general be used with dynamic or proxied contents. The second is not thus limited, but incurs a big performance penalty. The third and fourth are similar, and support general-purpose search and replace in apache’s output (mod_line_edit is designed primarily for use in a proxy but works anywhere; mod_substitute has no such preference).

The fifth, mod_sed, is new, and appears to be another big advance on any of its predecessors. Whereas mod_line_edit and mod_substitute are described as sed-like, mod_sed is the real thing: sed itself embedded in an apache filter. mod_sed developer Basant Kukreja (who I am privileged to have as colleague at Sun) has updated original sed code to be thread-safe and reentrant and to use APR pools, and has hooked it in to a filter. That alone means it can run much more complex operations than basic search-and-replace. But it has yet more to offer: mod_sed, unlike its predecessors, can filter input as well as output.

So what’s the cost of this extra power? Well, it’s bleeding-edge, with all that implies. And it’s bigger than its more limited competitors. But in terms of performance it appears to hold its own comfortably against any competition. Neither is it any more complex to configure. Well, I’m suitably impressed!

The question is, what next? I think for general-purpose filtering, mod_sed may be as good as it gets[1]. I’m wondering if this can now move in new directions:

• Can we come up with a framework to plug syntax modules into mod_sed (as we can, for example, make vim syntax-aware)? And if so, could it move into the space of markup-aware modules like mod_proxy_html and mod_publisher, or indeed mod_highlighter?
• Can we usefully apply mod_security-like rulesets with mod_sed to make a powerful untainting and information-disclosure filter? If so, the fact that it streams I/O will offer major performance advantages over scanning request and response bodies with mod_security, for situations where filtering is considered sufficient. This is something I’ve had at the back of my mind for years, but mod_sed offers a more powerful startingpoint than has hitherto been available.

[1] Yeah, right, you could probably do the same thing with perl and have truly the ultimate text processor, at the cost of much more bloat. But mod_perl does no such thing. Ditto other scripting modules, as far as I know.

[UPDATE] Since the above was written, mod_sed has been donated to the ASF and will be included as standard in Apache HTTPD 2.3/2.4 releases.

Tavistock jobsworth

I always thought Tavistock’s indoor market had quite a lot to recommend it.  Bustling, lively, and with some (though by no means all) good stalls.

My most regular calling point there is the wholefood stall, where I buy herbs and spices, grains and pulses.  There’s also a wholefood shop in town, but I prefer the stall for one main reason: the happy and friendly service I get from the lady on the market stall, as compared to a rather depressing atmosphere in the shop.

On Friday, that suddenly changed for the worse.  I headed down to the market to buy a couple of things from this stall.  It was 4pm, which is closing time for many stalls, but she’s always open after some of the others have packed up.  And indeed, I could see she was open.  But, alas, at the door to the market building, I was confronted by a miserable jobsworth, who insisted the market was closed and I couldn’t enter.  And furthermore, claimed to be the management of the market, in charge of the place.

I went to the shop, which has suddenly become a happier experience than the market.

Of course I know and accept that shops close.  I think no less of the greengrocer on the corner of the market because he closes at four and I’m occasionally too late for him.  But a jobsworth blocking me for no good reason is a different story, and gives the whole market bad karma.

Bah, Humbug.

Clueless?

RIP Humph.

If you’re a brit, you can’t’ve missed the news. For others, they’ve just announced the death of Humphrey Lyttleton, chairman for over 30 years of the popular radio show “I’m sorry, I haven’t a clue” (ISIHAC), among other things.

ISIHAC was a cult, introducing into our culture (or should that be subculture?) the game of Mornington Crescent, the esteemed Mrs Trellis of North Wales, the ever-lively Samantha, etcetera. It was characterised more than anything by the banter of a team of veteran comedians, coordinated (in the manner of expert cat-herding) by Humph. Banter that was at once witty and lighthearted, outrageously rude, and just like banter within the family. I think that last may have struck the strongest chord with many listeners, including Yours Truly.

I shouldn’t think they’ll try to revive ISIHAC under another chairman (if they do, they’ll surely re-brand it). But the good news for fans is that there is another show where the spirit of ISIHAC lives on, with its own brand of irreverent banter, often only slightly gentler than ISIHAC. Radio 4’s Gardener’s Question Time stands out as rather good entertainment even for those of us with no garden, nor knowledge of or serious interest in gardening. And its chairman Eric Robson has a definite touch of the Humph about him.

Try it!

Those who can, do …

… those who can neither do nor teach, run the NUT.

Today I’m glad I don’t have children, as the National Union of Teachers (one of several teaching unions) goes on strike over pay, and (according to the meeja, at least) lots of parents have to make one-off arrangements to take care of their kiddies.

Nothing new in that: they were messing us all about in my schooldays back in the 1970s[1]. That’s also when I firmly convinced myself of the truth of George Bernard Shaw’s famous quip (albeit out-of-context), when it was clear that most of my teachers were frankly not very bright at all. They remained amongst those who whinged the loudest throughout the Thatcher years, and … well, I haven’t paid so much attention of late, but they clearly haven’t grown up yet.

The NUT and others tell us that teaching is a profession, and teachers are professionals. Yet the NUT itself is, and always(?) has been, a complete antithesis to all that professionalism stands for. A professional may detest and despise the boss (c.f. Dilbert), but doesn’t take it out on the customer. Whereas a professional does the job and expects his/her hours will routinely exceed what he/she is contracted for, the NUT whinges and “works to rule” or even strikes. Nowadays our local bus drivers are clearly more professional than the NUT.

This time they’re striking about pay. Unlike the police – who have legitimate (albeit exaggerated) cause to be aggrieved – teachers have not been short-changed: it’s the full award of the independent review body they’re striking against. Historically, teachers salaries were below average for graduates (though their starting salaries were pretty good) but they made that up with excellent pensions and far more holiday than anyone else. Nowadays they still have the benefits, but pay has also risen very substantially in recent years. Those who don’t own their own homes even qualify as “key workers” for subsidised housing, which can leave them better-off than almost anyone in the private sector apart from a handful of celebrities and “city fat-cats”.

Guess who doesn’t have my sympathy.

[1] Even in my day, it was (some of) the inadequate teachers who participated in union-inspired disruption. The good teachers very definitely didn’t.

Getting to know the new mount

Today I went out for the third time on my new steed. I’ve still yet to do any longer distance (like a full or even half day ride): today was just under 30Km to go and view a cottage (it was mostly nice – I’m thinking about it). But I’ve tried a fair range of terrain, and (as is inevitable in this area) some fair-size hills.

It’s a pleasure to ride. Well, it jolly well should be! On-road it’s faster and smoother than other bikes I’ve had in recent years. On moderate off-road (grass, stones, gravel), it’s also very nice, with a firm and positive ride, though it lacks the ground clearance of a mountain bike. I didn’t miss knobbly tyres at all. But I did find myself getting off and pushing on the hardest off-road section I’ve yet done, both uphill and downhill on the steep sections at the top of Pew Tor.

It’s good to be back on drop bars, and these are right for me. I’m still adjusting to it (that should probably be adjusting back to it, though the paunch has grown since I last had drops). So on the steep uphill the reach feels slightly long, while on the flat the hoods are right, and on the downhill I have the luxury of the drops. The saddle is a comfy little number, too.

Some slight niggles: changing gear upwards on the chainring wants adjusting, and the toeclips are crap. But the LBS will give it an initial service in a couple of weeks, and can adjust things that need it.

One of these weekends, I’ll find time to give it a proper outing.

Bare Protectionism?

Heathrow, London’s (and Britain’s) main airport, has been a disgrace for as long as I’ve been around, and I first used it as a child back in the early 1970s. Terminal 5 is just the latest episode, and frankly I have to admit to a certain degree of schadenfreude about it: people who would fly for a journey like London to Edinburgh or Paris deserve what’s coming to them. But I digress.

That situation could justify changes in the management of the airports, which since privatisation in the 1980s has been in the hands of BAA. And indeed, there are suggestions flying around (pardon the pun) that the regulators may split up BAA, and force them to sell some airports. Maybe more competition would indeed be an improvement.

But right now, a change in management has only just happened! BAA has been taken over, and is under new management, who have scarcely had time to get their feet under the table. They might be embarking on a programme of doing everything right! So why the talk of splitting it up just now rather than, say, a year ago, or ten years ago? Or in a year or two’s time, if things don’t improve.

Could it have anything to do with the fact that the new owners are Spanish, and someone doesn’t like foreigners in control of such high-profile “national” assets? If we don’t want to accept a foreign owner, we shouldn’t have privatised it in the first place!

Meanwhile, I shall personally continue to do my best to avoid both Heathrow in particular and air travel in general.

Drop the “10p” rate

Right now there’s a media and political storm about dropping the so-called 10p[1] bottom rate of income tax.

There is merit in the central complaint: that it penalises people on low incomes (though not the lowest, who pay no income tax). There is little merit in the government’s response that many of those who lose will gain in other ways from the same budget, because those who gain are the same groups who have consistently gained from Brown’s budgets over the past ten years, and yet again there’s nothing for those who work hard for low pay but are childless. Indeed, the emphasis on ever more money for families may well be pushing us back to something like a 1970s “get pregnant to get housed” ethos.

But all that is of little relevance to the opportunistic humbug that’s filling the meeja. Politicians (on all sides) and others who are now screaming about it would be more credible if they’d taken it up a year ago when it was first announced. And the simplification is the Right Thing to do, insofar as simplification really happened. The obvious thing to do would have been to raise tax thresholds at the same time, though that does nothing about the poverty trap, where means-testing leaves people paying effective tax rates of over 100% on some of the very lowest incomes.

The point noone seems to have made is that the government must be able to change the burden of taxation over time, and that change cannot always be in one direction. The fuss about penalising some low-paid people is an argument that you can only ever adjust taxation in one direction. That leaves perpetual losers, and they won’t be happy.

Having said all that, in this instance I think the government should have done the obvious thing, and upped tax thresholds. And paid for it with less waste (like all those wars, and paying off the whole wunch of bankers) and/or a much-needed shift of taxation away from incomes and towards activities that pollute. A plague on all their houses!

For the record, most people are now taxed at 44%, while incomes over £43000/year are taxed at 54% on earned income[2]. Unearned income is taxed at much lower rates, and often not at all. And I’m not one of the losers from this budget: I think I’m marginally better off, though mostly for the wrong reasons.

[1] Actually a rate of up to 34%[2].

[2] That’s including the employment tax called “national insurance”, paid by both employers and employees.

Sanitising user-contributed markup

At ApacheCon, I once again encountered the argument sanitising markup is difficult, with an explanation of how easy it is to evade pattern-matching filters with tricks like reordering, whitespace, and embedded comments. I protested that this kind of difficulty comes from using the wrong tools, and the problem largely goes away if you use markup-aware tools.

On April 10^th I promised a note on this (though that promise came from a separate conversation at apachecon, and in a different context to the security issue). Today I’ve just delivered on that promise, with a brief technical note. I expect to use it in future when the subject arises.

Taxing times

A big downside to working for Sun is that it puts me onto bog-standard PAYE for all my main income. So I lose the tax-efficiencies I’ve hitherto used, and pay income tax in a country where taxes on earned income dwarf all other taxes, so that hard workers get to pay for the idle rich (grrr) as well as for government waste and folly (grrr) and for more deserving causes.

Today i invested the annual maximum (now £3600) in a cash ISA. That’s something I’ve been doing each year since my income from WebThing grew sufficient to support it. And I’m looking at what seems to be the only other way left to avoid Very High Tax: putting a very large amount of my income into a pension.

Now a pension is only any use if it is significantly above the government’s means-tested benefits. If I’m still renting a home when I retire that’s a non-starter, because means-tested benefits that include housing costs will leave a good deal more net income than I’m accustomed to or could save for. So to save tax, I also have to buy a house. Fortunately I can combine the two, as a pension can (in part) be used to buy a house. That is, if house prices continue downwards long enough …

The other thought that someone at Apachecon mentioned … what about moving about so much that I’m not resident in any country for tax purposes? That means living in three countries, each for clearly under 6 months a year. Or something like that. For the tax saved, I could rent somewhere adequate in each of three European countries (or indeed elsewhere, though permission to live&work outside Europe would presumably be a nightmare of red tape). Sounds like a lot more faff than I want to deal with, but it’s an interesting thought …

New steed!

I’ve just made it to my local bike shop to pick up my new steed.  Unfortunately I don’t have time today for a ride, beyond the short stretch home from the shop, so that pleasure is yet to come.

One thing I haven’t used before is these gearshifts based around the brake levers. The super galaxy has the brake lever itself shift up, and a smaller lever shift down, on both sides, so that’s not too confusing. The other common new feature – fix your feet to the pedals – doesn’t come as standard, and I didn’t ask for it, so I have old-fashioned toeclips instead. I know it’s much less efficient than special-purpose cycling shoes, but it’ll go nicely with my old trainers.

I still need to make some adjustments. I think I’ll want to move the saddle forward a little, and I may want to move the bars. But great to have a good-quality touring bike: one that’s great on-road and will also survive off-road!

Whee!