Randomly Selected Article or Link

On the occasion of his visit to Iraq on a "fact finding mission" the BBC report that "Gordon Brown has said lessons must be learned on use of intelligence in the run-up to war" and that "in future intelligence analysis must be kept independent of politics."

One nice idea might be that when the international community appoints a civil servant with no particular political axe to grind to look into the whole business, rather than agreeing with your ally in a stand up row about the guy's conclusions, you try hard to believe what your appointed on the ground investigator has been telling you.

Technorati Tags: dodgy dossier, intelligence, gordon brown, iraq

A couple of days ago I picked up on a report from the Times and Bloomberg about growth in M4 money supply. I've been trying to do a little research on relative growth of different types of assets and the money supply and retail prices.

I don't know if this chart makes any sense, but I found figures from various sources for M0 and M4 (from the Bank of England statistics service), for house prices (from the Department of Communities and Local Government), for Retail Prices (from the National Statistics Office) and for the FTSE Actuaries All Share Index (from Finfacts.com an Irish financial statistics organisation) most of which can go back to the late sixties.

I rebased all of them to start from 100 in all but one case in Q2 1968 (M0 I could only find from Q2 1969) and charted the results. No doubt something is glaringly wrong and people will soon yell at me what it is. I need to add GDP growth and one or two other things as well. But for now I thought I'd let people see what I had come up with so far:

Though this is not a techno-blog, I do do a fair amount of work behind the scenes myself in php, css and html to get it looking like it does (not a winner I notice!). But one thing has bugged me for some time...

The observant amongst you may have noticed that it is developed using the Drupal content management system. This has lots of add-on modules contributed by nice people all over the world. One of these is supposed to process "trackbacks". You'll have seen these on others' blogs - where if someone on another site posts something about a story you've written, you get a magical little line saying "Such and such a site or blog refers to this post".

Well with the Drupal trackback module this only appear to work if people have explicitly "pinged" the trackback URL for the post of mine they are referring to. And nowadays it seems that very few people actually do this. When I post on my blog it does ping each site I've linked to in the posting, I think (it certainly does at least the first site I refer to) without me having to explicitly tell it to.

So others see when I have referred to them and can put a link back in to my blog if they want. But I don't see automatically when they do, so they don't get a link back to their post underneath mine. But I know I am being linked to - I've been mentioned in the Lib Dem Voice "Golden Dozen" a few times, and in the Brit Blog Round-up and so on.

The trackback module claims to be able to discover a site that is referring to mine just from someone clicking on the link in another blog's article that refers to a post of mine. But it doesn't appear to work. So, while you've all been to conference, I've been trying my hand at rewriting that module to enable my blog to discover when someone refers to my blog just from click-throughs. And that's why I've been "blogging lite" this week.

I've not actually got very far yet. Just setting up a debugging environment took long enough. And now that I sit down to think about how to do it, I find it much harder than I first thought. I can see why the module authors have not implemented it yet! For example - how do I distinguish between a referrer that is a search engine results page, a "real" article that refers to me, or just a "blogroll" type link in a sidebar. Incidentally - I notice that Lib Dem Voice picks up these sidebar links, as I've seen my pages listed as referring to some of their articles when actually they seem to have picked up the feed from the Lib Dem Blogs aggregator I put in my sidebar.

However, if you've stumbled on this because you are also looking for this feature in the Drupal Trackback module, you might be interested in having a look at the logic I think I have finally settled on in the following specification to achieve this:

Note: I am going to do this as a Drupal cron job in the trackback module using entries in the Drupal access_log database table. This is because the processing of each one to check whether it is a real "referral" or a search engine page or a link outside an article like a blogroll will take a little processing time for each page load if I do it when each page is requested by a user agent. So I guess I'm doing something similar to what Technorati does when it indexes your site when it has received a ping. Except it won't be triggered by a ping, but a referrer record in the access_log table. So here's the logic in crude "pseudo code".

1. START processing:
2. When cron runs,
3. For each row in the access_log table that accesses a Drupal node and has a referrer URL since the last time cron ran
4. Try to fetch an RSS type feed url from the referrer site.
5. If the site doesn't provide a feed url, then it's likely it's not a blog or news type site and we can stop processing.
6. If it does provide a feed url, fetch the feed and parse it.
7. For each article in that feed check whether it contains a reference to the Drupal node referred to in the access_log record.
8. If it does not contain a reference to said Drupal node we can stop processing.
9. If it does, extract an excerpt and title from the article in the feed and save the whole lot into the trackbacks_received table. It should then appear under the node on my site when it is viewed.
10. STOP processing.

As a slight aside, I'm wondering how to check comments on others' posts as well. I'm not quite clear whether all types of feed have a way of discovering the comment feed, if one is available, for each article in the main feed. If so it can probably be done.

If you didn't believe that Orwell's 1984 was inexorably becoming a reality you should read this:

BBC NEWS | England | Kent | Local groups discuss terror signs:

Neighbourhood Watch co-ordinators across Canterbury have been finding out how they can prevent terrorism.
It was one of the main issues discussed at an annual conference held at the Kent city's Christ Church University.

Co-ordinators were told how to spot signs of bomb-making and other preparations for terrorist acts.

What an awful prospect. If this is even necessary, which I frankly doubt, the best way to be watchful in our own communities is to be better communities, to know and if possible be friends with your neighbours. Neighbourhood Watch is a worthy campaign, and best placed to spot strangers coming in to burgle our homes and so on. But to spy on the very neighbourhoods they are set up to serve? No thanks.