Journal of an Avocational Web Project

BSV 1.0 Released

Mon, 28 Apr 2008 19:23:14 +0000

Release Notes

I released the next version of BSV over the weekend. It satisfies all my criteria so this release is 1.0 (finally). Here are the release notes:
changelog.txt

What Next

Now that there is sufficient functionality for the website to be really useful without taking up too much of my time, it is time to take it to the next level. By that I mean to make sure I have a few more users than I currently have. So I need to concentrate on a bit of e-marketing.

Firstly you may have noticed from the release notes that I now have an about us page, where I unashamedly ask for people to link to the site and donate to the site so that I can invest the money in some e-marketing. In addition, there is a useful sites page which has a list of links to sites that I think are useful for my users. They are all sites that require a reciprocal link and have passed quality criteria (they have to be useful and well used).

I'll keep you posted on the traffic but some preliminary stats are looking good!

Link Farms

Let me explain something about linking to other sites.

Google's early success was largely attributable to the accuracy of the results that it returned when you did a search. It soon became clear that the algorithm that was used to prioritise results was absolutely critical. Some may think intellectual property as valuable as that would be a closely guarded secret although there is an argument that you should be as open as possible, especially when a massive number of consumers are involved. Anyhow, I'm not sure why or exactly when but Google decided to divulge the concept behind their code.

The idea is pretty simple, every web page that has been indexed is assigned a number that represents how popular it is (the page rank). For a given search the results are ordered by popularity. So how can you possibly calculate how popular a web page is without access to exhaustive usage statistics of all web pages? You can't but Google believe that with the information they have, they are able to approximate popularity. They do this by looking at how many inbound links there are and the contents that is wrapped in the links.

All good and well, as long as no one exploits this information. But this is the real world so of course the same group of marketeers that exploited older search engine technologies to improve their listings used the algorithm information similarly, to the detriment of all web users. One of the tactics they used was creating link farms. Link farms are websites or areas of a website which has the sole purpose to improve the page rank of a site by exchanging links indiscriminately. Ultimately link farms are bad because they offer no value to a user at all, they simply artificially (and therefore incorrectly) increase a website's popularity which means Google's results aren't as accurate as they used to be.

Google obviously are aware of link farms and probably are taking/have taken steps to deal with them. There is one rumour going around that when they discover a link farm they simply strip the page rank of the offending site, personally I don't think they would be quite so drastic. They certainly have been presented with a problem which isn't easily solved: how do you identify indiscriminately exchanged links with truly valuable ones in a automated manner? You either need a highly sofisticated heuristic approach which will no doubt be constantly changing or you do it manually (at least some of it).

All this is also a problem for website owners because if we choose to increase our popularity using links (and when you have no budget like me there pretty much is no alternative) how do you do it without being classed as a link farm? The answer is not to link indiscriminately and that is what BSV's useful sites page does.

BSV 0.8 Released

Wed, 09 Apr 2008 10:01:14 +0000

Software Releases

Since I am writing a journal about this I may as well formalise a couple of things. Firstly I need to give the software a name, BSV is good for me so that's what I called it.

Next on the agenda is versioning. I have decided that version 1 will have to satisfy the following criteria:

Full CRUD functionality for property management
Working satisfactorily in IE 6 and Firefox

This release is missing part of the photos update functionality in IE due to the browser not adhering even closely to W3C standards, so I'll randomly give this version 0.8 (you have to start somewhere).

Release Notes

Here are three quick definitions that I will use in the release notes:

enhancement - refers to new functionality
change - refers to modification of existing functionality
defect - refers to correction of erroneous functionality

I know they are a bit fluffy around the edges but they are only intended to be an indication. Here are the release notes:
changelog.txt

Image Caching In IE

Mon, 31 Mar 2008 11:18:26 +0000

Caching Problem

The problem is that IE does not cache images at all if they are referenced in DOM javascript. The effects of this are obvious: pages are slower to load and bandwidth is consumed unnecessarily. When I came across this, I didn't try and change IE settings so it may be that you can fix the problem by playing with Tools > Internet Options. But more to the point I wouldn't want to fix this by tweaking my settings, I'm operating a public website and it should work well with out of the box settings.

Sadly there is no quick fix and no workaround. I have confirmed it to be a problem in IE 6 and IE 7.

Scenario

Here is a scenario to reproduce

create multiple image nodes iNode = document.createElement('img')
set the same source attribute on the images iNode.src= uriSrc
append the images to an existing DOM node domNode.appendChild(iNode)

IE will download every image as a distinct object, although they have the same src attribute. Maybe I'm going slightly mad but to make it worse it seems to take way longer to do this than it ought to. I'm not going to waste time verifying that so I'll give it the benefit of the doubt.

Real Life

What I've just described is all a bit abstract and you're probably thinking 'yeah, but why would you want to do that? Give me real scenario.' Well, here it is. Have a look at this snapshot:

There is a main image and below it thumbnails, each with mini icons above and below. The mini icons are used to manage photos, the arrows allow you to move them left and right while the A and X icons allow you to edit the photo title and delete the photo. When you click on any mini icon the meta data for this widget changes and communicates this to the back end asynchronously.

There are two situations where this widget needs to create image nodes and they are when it initialises and when meta data changes (since it synchs with the database). Obviously, when this happens the browser has to render each mini icon. In firefox one loads and they all appear at the same time. In IE one loads, then another, then another. You get bored go and grab a coffee, still loading, come back, still loading, take a sip, the next one appears... OK that's exaggerated but you get the picture.

What To Do?

So what should I do about this on bulgariaseaview.com? This is where my earlier decision to work on this project independently begins to pull it's weight. There is no one to answer to here, so I have opted to leave it working slowly in IE and added a note recommending the use of firefox.

It's a bit naughty, after all who am I to tell anyone what browser they should use? But on the other hand imagine how completely I would be ignored if I asked Microsoft to fix their code and maybe by explaining why it works better in a more compliant browser it will convince users to stop using inferior ones.

Styles Starting With Underscores

Mon, 17 Mar 2008 10:51:22 +0000

Quick Fix

The quick fix is this: do not use CSS styles that begin with an underscore (_) if you want them to be rendered in IE 6. By the way, the bug in IE has been fixed in version 7.

IE CSS Underscore Bug

I was sort of aware of IE's CSS underscore hack but I was doing my best to forget it, since it's a piece of information which is as mundane as it is common with IE, "yet another problem with IE". However, this issue must be related to that hideous hack.

This is a screenshot of what I wanted to achieve in IE 6:

Simple enough, so I put together this HTML which should be absolutely fine:

"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

IE 6 - styles starting with underscore

Tab 1

Tab 2

However IE 6 renders this:

Can you spot what's wrong? Most people would and I would argue that the difference is not acceptable for a public website, even though styles are subjective.

To fix it you simply change all the styles defined in the

Tab 1

Tab 2

Setting the Scene

Mon, 10 Mar 2008 22:42:03 +0000

An Afterthought

This blog is an afterthought, one that came to me like so: I was happily doing some web 2.0 development, testing in Firefox with those very useful add-ons and it was all looking good. Then I thought I'd better make sure it worked in IE which is always slightly annoying since the computer that has it installed isn't mine.

Having successfully negotiated use of the test machine, "I'll prepare you a hot chocolate if you let me use your lappy", I was ready to do my 5 minutes of validation before calling it a night. There was no way it was going to take longer than that, after all my code was W3C compliant. What could possibly go wrong?

IE bombed out with it's usual ironic error message (you know, the one that is supposed to be helpful but is actually almost completely pointless). It soon became apparent that the only reliable way to fix the problem was to bombard the search engines with questions until they revealed the answer. On this occasion it came in the form of a tech-site written in that unique form of hispano-germano-cyrillo english that you are probably familiar with.

Anyhow, this got me thinking. Wouldn't it be great to write down all the issues I encounter for the duration of the project with details of how I solve them so that others can benefit? What a unique and genius idea! Then I realised that actually it wasn't unique, that hispano-germano-cyrillo-english guy had already thought of it (along with millions of others). Nonetheless, I concluded that it was still a good idea to document the project since there was bound to be at least one person who would find it useful.

That is why this journal is an afterthought and because it is an afterthought, it has come into existence just over 1 year since the project began. So, let me give you a brief run down of what has happened so far.

My Approach

Firstly, I decided that this would be purely my own work. No partners, no clients (like I can just choose to have clients), no other developers, no time schedules, no one to answer to: fantastic. Embarking on a project with that mindset makes a huge difference.

Being my own client I told myself that all I wanted was a quick brochure site that I could start e-marketing within a few days. I toyed with the idea of using java but then I decided if I wanted to do things quickly it would have to be done in perl. Perl development is fast anyway and on top of that I had tried and tested libraries that I had built up over the years which would make it even faster.

I set to work to get a couple of pages with fresh content, good usability, nicely styled (isn't that subjective?), targeted meta data and compliant mark-up. You know, all the elements needed for SEO if you believe in that particular form of occultism. Then I used a bit of web 2.0 magic so that I could display my holiday snaps in a single compact page. Finally, I implemented the google maps API so that users could get a bearing of where the apartment is, and an AJAX control so that they could send me coords of areas of interest. Next I went shopping for a domain name with my favourite registrar, and set up hosting with them as well. An hour after that, I had deployed my new web app and registered it with the search engines.

A final step was needed: approval from one of the world's greatest consumers and avid on-line reader. I showed the site to my girlfriend and watched her expectantly. She stared at the monitor for a couple of seconds, then her hand moved tentatively towards the mouse. She retracted it before it got there and turned to face me.

"Shall we go dancing tonight?" she blurted out.

Success!

How The Website Has Grown

The website has changed a lot since then. Without going into great detail this is what I did on the technical front:

I packaged it up properly so that it could be reliably deployed in various (linux and FreeBSD) environments by referencing an appropriate configuration file.
I improved my publication technology so that it was able to publish in static or dynamic mode.
I modified my templating technology so that code was naturally divided into cohesive units.
I improved my core javascript library so that it had inherent support for AJAX calls.
I added a whole load of custom UI tags.

As far as what is considered creative in the traditional way, even though personally I consider software development just as if not more creative, this is what I got up to:

I added guides with photos (my holiday pictures). For the guides I did some on-line research, had conversations with people from the area and added my personal experience.
I picked up a few other pictures with appropriate creative commons licenses and in doing so I had to improve my image widget so that I could properly attribute the photos.
I integrated skyscanner mini.

The Big Decision

By this time the site was attracting enough traffic to get a couple of enquiries about the property. More importantly, it felt like the code was solid. I had adequate unit test coverage, was working on integration tests using selenium IDE and a proper error notification framework made sure I fixed defects pretty quickly.

Also at this time, it came to be that an acquaintance who owns an apartment in Bulgaria as well, demanded that I add a page for his property to my site. My defences had been weakened by the effect of beer and I responded enthusiastically "Yeah, no problem. Get me another drink, will you?"

So I was faced with this question. Should I

create another static HTML page for his property
- or -
enhance the web app so he could add and manage his property and in doing so allowing anyone to do the same with theirs?

The answer to this dilemma is obvious and as far as I can remember these are the steps (more or less in order) that bring us to today:

Requirement 1: set up a straight through processing form that,

creates a user account with contact information,
allows the user to enter property address, descriptive details, select from a list of communal and private facilities, property meta data, season dates, rental rates, rental and payment terms, photos with meta data and of course map coordinates.

I used the above requirement with bit of agile modelling (scribbled on a piece of paper which was ultimately destined for the bin) to design the relational object model.
From the object model I derived a database schema.
I set up a data system for my web app which hooks into mysql, and coded up repeatable and reversible scripts for

creating the schema
populating standing data
validating (at a basic level) data integrity

Then I wrote a load of perl modules which together forms the data access layer of my web app.

With that my first requirement was satisfied and I deployed code to see if anyone would add their property.

Requirement 2: use the data collected from requirement 1 to create a single HTML page per property.

I had most of the technology for this, all I needed was to code up a component that extracted the data and then used my publishing and templating technologies to create the HTML.
I added support for scheduled publication of property pages (with cron) and the ability to publish a single page or the whole lot.

With the second requirement completed, I replaced my old static property page with my new database generated one and added the page for the second property as well.

Requirement 3: the property owner should be able to manage the content related to their property.

I dug out an old library that I wrote for an authenticated web session. I improved it (a hell of a lot) until it was a proper MVC system using the command and front controller patterns (I know, it's not a J2EE project but it's a good pattern) as the core framework.
I added support to the MVC framework so that I could plug my publishing and templating technologies into it.
Then for each object model I coded create, read, update and delete functionality.

Well, for almost each one. I have the photos model left and if I hadn't started this blog that would be ready as well.