Archive

Posts Tagged ‘whipuptitude’

Importing Radio posts to MT

January 5th, 2003 No comments

A couple of people have asked for more pointers on exactly how I got my Radio posts into MT. So here’s some more detailed information.

Firstly, I ran into lots of problems with Radio seemingly caching macros. When I edit radio macro files and resave them, Radio doesn’t seem to notice the changes for about 5 minutes. Which is far ideal for the programming approach I use (particularly in a language I’m not that familiar with, such as UserTalk), which basically involves keeping the code running at all times (code a line, save it, test it still works). I would never survived as a programmer in the old days of coding ‘offline’.

I originally thought this was to do with my set up (I have my PC’s C: drive mounted via samba onto the linux box on which I do most of my programming, as I find it much easier to code in that environment), but using Notepad didn’t seem to make any difference, and there are a few scattered references to this problem littering the Userland noticeboards.

Anyway, I ended up having to code in the scratchpad instead. The script is below. It was cut-n-pasted from Radio’s outline editor so the formatting is a little strange, but it should do the trick. This can either be used as a macro, or, as I did, or by adding it to the scratchpad and calling <% workspace.showAll ()%> in a page (I just created a new page in my Radio Userland/www/ directory that had that in it.

This outputs all my posts in XML, with the titles and bodies wrapped in CDATA tags. I then wrote a simple Perl script to turn this into the MT input format. You then import this into MT following the instructions in the manual.

I’m sure this could all be tidied up some more, but hopefully it’s of some use to someone as is!

Read more…

Migrating from Radio to Movable Type

January 4th, 2003 No comments

I finally managed to move to MT from Radio. As much as I’ve enjoyed learning all the ins and outs of Radio, I’m just finding it too painful. Whilst the memory leak which slowly kills my computer (unless I restart Radio each time I want to do something and then shut it down again straightaway) is merely irritating, the fact that I can’t post when I’m not at my desktop machine is becoming a major problem.

In particular, I’m heading off to Boston for a week soon and I want to be able to post whilst I’m there. (Anyone who reads this who’s going to be in Boston in the next few weeks and would like to meet up, please drop me a line at tony@tmtm.com)

Actually setting up MT was trivial, but I’ve spent a few days trying to minimise the disruption to readers. I’ve set Apache up to redirect most requests to the relevant page on the new site, but I’ve almost certainly missed some.

There were all sorts of interesting issues with the migration:

1) Export / Import

I wrote a radio macro to export all my posts as XML, and a Perl script that then turned that file into MT’s input format. This was easier than trying to get Radio to export to that format, as I couldn’t find an easy way to output only the information I wanted on a Radio page. It also gave me the chance to make a few other changes in Perl, where I’m much more comfortable.

2) Post Titles

In Radio you can have untitled posts. MT gives those titles – some of which are very strange …

3) Title Links

In Radio I often used the ability to add a link to a post’s title. I couldn’t find a sensible way to import this into MT. I’ll gradually work my way through my historic posts to fix this.

4) Auto-formatting

Both Radio and MT attempt to auto-format your posts. But they do it in subtly different ways. I had to turn off ‘convert line breaks’ in the imported posts by performing an update on the database. This makes most posts look a little strange (mainly because they lose the paragraph breaks). Again, I’ll tidy those up over time.

5) Template and Stylesheets

I’d heavily customised my Radio templates and stylesheets. I had to do the same with MT. It took me a while to work out where some of the templates actually were with MT (particularly the seach template which by default applies across the entire MT installation, rather than per blog, and thus can’t be edited in the same way as all the others…), and then a lot longer to modify them all. Again, I’ve almost certainly missed some.

6) Blogroll / Linkback / Google-links

The blogroll and link-back were done via xbit-hack includes, and so could be brought in trivially. However, MT clobbers the file permissions every time it rebuilds the page, making the SSI no longer work. For now I have to remember to chmod it by hand. The google-links used to be created automatically as I wrote my posts and stored in the Radio object database. I still haven’t investigated how to replicate this in MT.

7) Posts on Home Page

With Radio I could say show the last 7 days of posts on the front page. In MT I can say the same. However, with Radio it shows the last 7 days on which there were posts, whether they were actually the last 7 days in real time. In MT it shows the last physical week. So I’ve had to tell it to show the last 10 posts instead.

8) Navigation

I set MT up to have daily archives, but still haven’t found any useful way to navigate around those beyond the back-a-day / forward-a-day links on each page. I quite liked the ‘Month of Posts at a Glance’ calendar that was on Dive Into Mark at some point, so I’ll probably try to find that and use it.

If there’s anything else strange about the new version, please let me know!

HTTP Conditional Get

October 22nd, 2002 No comments

My “last updated” blogroll on the left hand side of the page is now much friendlier than before. I’ve made a fairly trivial changed my script so that to now use LWP::Simple‘s mirror() function, rather that it’s get() one. This uses the If-Modified-Since header to only fetch the body of people’s RSS feeds if they’ve changed. It should not only cut down on bandwidth usage, but also makes it much faster to update my blogroll, which was becoming quite slow, as this is all running on a fairly old machine.

Upload problems

August 30th, 2002 No comments

For the last few days I’ve been having ADSL upload problems, which may cause subtle problems with this site. At first I thought it was the server, which had to be rebooted a few days ago for the first time in about 2 years. But, although it’s having a few teething problems readjusting to life in the modern world, I’ve discovered that Radio also has problems uploading to my company server as well. For a while I thought that maybe something had happened with Radio, but then we discovered that any upload of over about 8k is having problems (the problem doesn’t affect small files, and thus tools like ping weren’t showing any difficulties). It also doesn’t cause any problems with downloads.

So, in the meantime, I’ve had to change Radio to post to an internal server, then then manually gzip and split the files, and scp them up to here by hand. I’m only maintaining my index.html and rss.xml like this. Daily archives, and category archives will reappear once normal service is resumed.

Sorry for the inconvenience.

googleTitleSearch redux

August 23rd, 2002 No comments

The googleTitleSearch macro caches the results each time it searches Google in Radio Userland’s inbuilt database. This means that it doesn’t have to rerun the search every time you publish your blog. However, it also means that the See Also: becomes fixed at the point you publish, which isn’t always what you want – particularly if you’re commenting on something recent that Google doesn’t know much about yet, or if the links in question are no longer relevant, months later.

So, I changed the script to store the date that the query was run, and then rerun it if once a day for the first week, and then again if we’re republishing in more than a month’s time.

I also decided to refactor the code quite a bit, so I could learn more about Radio’s macro system. I learnt a lot about nested functions, and arrays, and how much I really miss map-style functions for dealing with lists!

The new code:

1
2
3
4
5
6
7
8
9
10
11
on googleTitleSearch (adrpost) {  if not defined (adrpost^.title) and not defined (adrpost^.link) { return "" };
  on htmlForResult (adr) {    on chooseToolTip(adr) {      if sizeOf (adr^.summary) > 0 { return adr^.summary};      return adr^.snippet};
    on chooseTitle (adr) {      if sizeOf (adr^.directoryTitle) > 0 { return adr^.directoryTitle };      return adr^.title };
    on tidyTooltip (tip) {      return string.replaceAll(searchEngine.stripMarkup(tip), """, "&quot")};
    on tidyTitle (title) {      title = searchEngine.stripMarkup (title);      return string.trimWhiteSpace(string.replaceAll(title, "...", "")) };
    local (url = adr^.url);    local (title = tidyTitle(chooseTitle(adr)));    local (tooltip = tidyTooltip(chooseToolTip(adr)));    return "<a href="" + url + "" title="" + tooltip + "">" + title + "</a>"};
  on resultsAsHTML (theResults) {    local (adr, htmlHTML = {});    for adr in theResults { htmlHTML[0] = htmlForResult(adr) };    return htmlHTML };
  on daysAgo(aDate) {    return (long(clock.now()) - long(aDate)) / (60 * 60 * 24) };
  on haveResults (result) {    if not defined (result^) { return false };    if result^.searchComments == "Sorry, no content found for this URL" {      return false };    if not defined (result^.searchDate) { return false };    local (lastCheck = daysAgo(result^.searchDate));    if (lastCheck > 1 and lastCheck < 7) or lastCheck > 30 { return false };    return true };
  on join (aString, anArray) {    local (i, returnString, arraySize = sizeOf(anArray));    for i = 1 to arraySize - 1 { returnString = returnString + anArray[i] + aString };    return returnString + anArray[arraySize] };
  local (seeAlso = "");  try {    local (adrSearchResult = @adrpost^.googleTitleSearchResult);    if not haveResults(adrSearchResult) {      if defined (adrpost^.link) {        adrSearchResult^ = google.search ("related:" + adrpost^.link)};      if not haveResults(adrSearchResult) {        adrSearchResult^ = google.search (adrpost^.title)}};    adrSearchResult^.searchDate = clock.now();    seeAlso = join(" | ", resultsAsHTML(@adrSearchResult^.resultElements)) };  return seeAlso };

googleTitleSearch

August 15th, 2002 No comments

For a while now, I’ve been using Jake’s googleTitleSearch macro to auto-populate the “see also” links after my posts. But I’ve been noticing more and more that all the links that Google is offering for some titles are the same – particularly when I link to articles at sites which operate on multiple URLs. I’m sure Google will sort out this problem eventually, but in the meantime I’ve made a few changes to the macro so that if I create a link for a post, it will do a google search on what is related to that, instead.

The macro, with my changes highlighted:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
on googleTitleSearch (adrpost) {
  local (htmltext = "");
  if defined (adrpost^.title) or defined (adrpost^.link) {
    try {
      local (adrSearchResult = @adrpost^.googleTitleSearchResult);
      if not defined (adrSearchResult^) {
        if defined (adrpost^.link) {
          adrSearchResult^ = google.search ("related:" + adrpost^.link)};
        if not defined (adrSearchResult^) or adrSearchResult^.searchComments == "Sorry, no content found for this URL" {
          adrSearchResult^ = google.search (adrpost^.title)}};
      local (adr);
      for adr in @adrSearchResult^.resultElements {
        try {
          local (url = adr^.url);
          local (title, tooltip);
          if sizeOf (adr^.directoryTitle) > 0 {
            title = adr^.directoryTitle}
          else {
            title = adr^.title};
          title = searchEngine.stripMarkup (title);
          title = string.replaceAll (title, "...", "");
          title = string.trimWhiteSpace (title);
          if sizeOf (adr^.summary) > 0 {
            tooltip = searchEnging.stripMarkup (adr^.summary)}
          else {
            tooltip = searchEngine.stripMarkup (adr^.snippet)};
          tooltip = string.replaceAll (tooltip, """, "&quot");
          htmltext = htmltext + "<a href="" + url + "" title="" + tooltip + "">" + title + "</a> | "}}};
    if sizeOf (htmltext) > 0 {
      htmltext = string.mid (htmltext, 1, sizeOf (htmltext) - 3)}};
  return (htmltext)}

Client- vs server-side scripting with Radio

July 30th, 2002 No comments

Jon Udell talks about client- vs server-side scripting with Radio: Radio pages, for example, are dynamically generated but statically served — a strategy that I like very much on the whole, but sometimes chafe at when nifty features like TrackBack and your search extender show up in MT.

For some time now I’ve been considering moving away from Radio. I like a lot of its features, but whilst I can muddle my way around Radio’s macro system, I find it much easier to do things in Perl. And, as the pages are generated, rather than calculated, there’s lots of nifty things I’d like to do, but can’t.

And then it struck me, that actually I could solve both of these problems in one simple step – SSI.

So, I decided to try to implement one feature I’ve been wanting for quite some time: a “further reading”-type feature, like that at dive into mark etc, that tracks referrers back:

I turned SSI on in Apache for my blog directory, and wrote an Apache Logging Handler that throws the referrer information into a MySQL database. Then I wrote a simple CGI script that tallies up the referrers for a given page, and outputs them as HTML.

Then in my Day Template, I added the line:

<!--#include virtual="/cgi-bin/getlinks?date=<%longDate%>" -->

Now when Radio generates my page, it will include the SSI command in the static page, which in turn will get called at page request time, calculating the HTML to insert in the final document.

Hey presto! Server-side scripting, in Perl, with Radio.

(My only nit is that I have to pass a date with spaces in it as an argument to the CGI script. If anyone knows how I can reformat the ‘longdate’ macro output in Radio, I’d love to hear …)

Browsing Google

April 21st, 2002 No comments

Dave Winer has created a Google Outliner Browser: It’s a different way to crawl through The Mind of Google

For those without outliners, I’ve knocked up a simple web-based version at http://www.kasei.com/google/browse.

Tags: