Randomly Selected Article or Link

Though this is not a techno-blog, I do do a fair amount of work behind the scenes myself in php, css and html to get it looking like it does (not a winner I notice!). But one thing has bugged me for some time...

The observant amongst you may have noticed that it is developed using the Drupal content management system. This has lots of add-on modules contributed by nice people all over the world. One of these is supposed to process "trackbacks". You'll have seen these on others' blogs - where if someone on another site posts something about a story you've written, you get a magical little line saying "Such and such a site or blog refers to this post".

Well with the Drupal trackback module this only appear to work if people have explicitly "pinged" the trackback URL for the post of mine they are referring to. And nowadays it seems that very few people actually do this. When I post on my blog it does ping each site I've linked to in the posting, I think (it certainly does at least the first site I refer to) without me having to explicitly tell it to.

So others see when I have referred to them and can put a link back in to my blog if they want. But I don't see automatically when they do, so they don't get a link back to their post underneath mine. But I know I am being linked to - I've been mentioned in the Lib Dem Voice "Golden Dozen" a few times, and in the Brit Blog Round-up and so on.

The trackback module claims to be able to discover a site that is referring to mine just from someone clicking on the link in another blog's article that refers to a post of mine. But it doesn't appear to work. So, while you've all been to conference, I've been trying my hand at rewriting that module to enable my blog to discover when someone refers to my blog just from click-throughs. And that's why I've been "blogging lite" this week.

I've not actually got very far yet. Just setting up a debugging environment took long enough. And now that I sit down to think about how to do it, I find it much harder than I first thought. I can see why the module authors have not implemented it yet! For example - how do I distinguish between a referrer that is a search engine results page, a "real" article that refers to me, or just a "blogroll" type link in a sidebar. Incidentally - I notice that Lib Dem Voice picks up these sidebar links, as I've seen my pages listed as referring to some of their articles when actually they seem to have picked up the feed from the Lib Dem Blogs aggregator I put in my sidebar.

However, if you've stumbled on this because you are also looking for this feature in the Drupal Trackback module, you might be interested in having a look at the logic I think I have finally settled on in the following specification to achieve this:

Note: I am going to do this as a Drupal cron job in the trackback module using entries in the Drupal access_log database table. This is because the processing of each one to check whether it is a real "referral" or a search engine page or a link outside an article like a blogroll will take a little processing time for each page load if I do it when each page is requested by a user agent. So I guess I'm doing something similar to what Technorati does when it indexes your site when it has received a ping. Except it won't be triggered by a ping, but a referrer record in the access_log table. So here's the logic in crude "pseudo code".

1. START processing:
2. When cron runs,
3. For each row in the access_log table that accesses a Drupal node and has a referrer URL since the last time cron ran
4. Try to fetch an RSS type feed url from the referrer site.
5. If the site doesn't provide a feed url, then it's likely it's not a blog or news type site and we can stop processing.
6. If it does provide a feed url, fetch the feed and parse it.
7. For each article in that feed check whether it contains a reference to the Drupal node referred to in the access_log record.
8. If it does not contain a reference to said Drupal node we can stop processing.
9. If it does, extract an excerpt and title from the article in the feed and save the whole lot into the trackbacks_received table. It should then appear under the node on my site when it is viewed.
10. STOP processing.

As a slight aside, I'm wondering how to check comments on others' posts as well. I'm not quite clear whether all types of feed have a way of discovering the comment feed, if one is available, for each article in the main feed. If so it can probably be done.

It is generally accepted I think that the Wall Street Crash and the subsequent depression started with a major sell-off on the New York Stock Exchange on Thursday 24th October, 1929 - the so called "Black Thursday". The day of the worst one day fall on the New York Stock Exchange was the following Monday, "Black Monday", October 28th 1929, when it lost 13% of its opening value.

If all this does prove to be the start of a period of economic turbulance that will live in memories as long as the Wall Street Crash, I wonder which day will be identified as the start of it all. It seems to have been continuously bad news for weeks. Today, Russia's stock exchange index lost virtually 20%, having suspended trading twice. But what about the day the US bailed out AIG? Or when Hank Paulson got down on his knees to beg Nanci Pelosi to sign up to the $700bn bailout? Certainly that day I thought for the first time George Bush actually looked chillingly, almost menacingly sincere, much moreso than at any time during the Iraq conflict. The nationalisation of Fannie Mae and Freddie Mac perhaps? Or the failure of Lehmans? Or maybe just the day Peter Mandelson returned to the government!

It's not that I am usually a luddite. Nor do I necessarily mourn the fact that workers have priced themselves out of a job. But there is something very sad about the Telegraph's story that the Symington port family is to phase out crushing grapes by gangs of human feet:

Centuries of port heritage ended by family firm

The world's oldest and largest port producer is finally trampling on 2,000 years of agricultural history.

The Symington family, which has been making port since 1652, has announced that it will no longer crush its grapes under foot.

The saddest part is surely that:

While the robots are an expensive investment, they can do the job at any time of the day or night - and don't need the encouragement of an accompanying musician.

Will port wine ever be the same without the local folk tunes of Portugal being instilled into it at birth I wonder?

If the world is a more dangerous place, it's as much because of people like Richard Armitage: US threatened to bomb Pakistan as it is because of people like Bin Laden.

I hope personally that we will choose to stand shoulder to shoulder with our Commonwealth brothers and sisters who have provided us with many of our residents and not a few friends and colleagues than a regime that even remotely thinks it's acceptable to make such threats.

Interestingly though, this is a similar phrase to one that was alleged to have been used as an ultimatum to the Taliban themselves, BEFORE 9/11, two months before, when they were stalling in talks over a Unocal pipeline from the central asian republics to the Arabian Sea coast.

If you threaten what to most of us seemed like a basket case state and their friends respond with a 9/11, would you really want to threaten a more sophisticated populous and relatively much more influential military one with nukes, even small ones, like Pakistan?