Forget long-term access, we’re struggling with the short-term!

On Friday I attended a one-day conference at the e-Science Centre at Edinburgh University, entitled “Maintaining Long-term Access to Geospatial Data“. The quote above came from one of the speakers and, to me, it was the key point that came out of the day. Having said that, some interesting themes emerged that align nicely with ongoing discussions amongst the geospatial crowd.

Firstly- the overwhelming trend in all the discussions was a move towards a service-based decentralised architecture for geospatial data storage and sharing. This requires good and standard metadata, and of course a transparent Spatial Data Infrastructure (SDI). The title quote refers to the fact that we haven’t really suceeded in either of those two areas yet, for data sharing now, let alone 50 or 100 years into the future.

Steve Morris, from the North Carolina Geospatial Archiving Project (NCGDAP) said that the establishment of an SDI should be seen as a catalyst for future discussion, and presumably experimentation/innovation within the geospatial community. I find this encouraging- as trying to create something that suceeds right from the word go is daunting, if not totally off-putting. In a later discussion, James Reid from Edina asked whether we should be trying to create global or national SDIs, or whether more localised or sector-based approaches might not be better. At the risk of over-using a phrase, should we be Thinking Global, but Acting Local? Given recent discussions about the lack of success of many current SDIs, perhaps this approach might be the way forward. Certainly I would like to see both an archaeological SDI, which could be global or national in scope, and a UK SDI focussing on datasets with national coverage. In particular, I would like to see the establishment of an SDI as an aim of the OSGEO Archaeology Special Interest Group (follow the link if you’re interested).

Clearly a good SDI is totally reliant on good metadata. Without going into a long discussion on the merits and pitfalls of metadata, there is a tendency to overdo the amount required, making it unlikely that people will produce any at all. There is also a tendency to invent entirely new standards, making it more difficult for data producers to know what they should be complying with. Tony Mathys, again from Edina described something emminently sensible in his discussion of the Go-Geo! Project, which is the creation of sector-based profiles, all of which contain a small number of required core elements, and additionally only those elements that are relevant in that sector. Again, I would like to see something like this created for archaeology, particularly in the UK, but I don’t want us to reinvent the wheel when we could create an “archaeology profile” from existing standards.

Finally, there was a suggestion that producers of geospatial data are generally “temporally impaired”. In other words, we rarely consider future uses or interpretations of our data as we are concentrating on fulfilling a particular objective in the present. We rarely maintain archives of old versions of files, and our metadata tends to refer to a snapshot in time. At OA we try and get around these issues by maintaining an archive of geospatial files, when storage allows it, and we try to keep a “lab notebook” detailing the work process (and hence any changes made) throughout a project. Storage space is an issue though, as is educating people about the need to do this.

Overall, this was a very useful conference to attend (the food and facilities were great too), and it provided a very useful insight into how larger institutions such as the UK Higher Education bodies are addressing issues of data sharing and archiving. For OA, the main thing is to keep moving forward, get our own SDI running smoothly, get sharing data with our colleagues elsewhere, and produce the right metadata to go with it all. Phew!

Minor triumphs, Major Hassles

In my spare time/lunch times I’m in the middle of a major project at the moment, to update our site database. Without going into the gory details of how it ended up in three separate, totally unlinked databases, it is supposed to document the archaeological sites we’ve worked on since the 1970’s, and to help with the administration of project archives, the location of finds within our finds store and so on.

For a long time I’ve wanted to sort this out, pull everything into one place, display the site locations on a web-based GIS and so on. This is not rocket science, but the data had ended up in such a bad state that I couldn’t see past fixing that before getting to the good and fun stuff. Attempts a few years ago to use Mapserver as a mapping interface were abandoned because the positional data for the sites was wildly inaccurate (we seemed to do a lot of work in the Scilly Isles, which are as close to 0, 0 on the British National Grid as you can get), and connecting to Microsoft Access was quite difficult and unstable. If anyone using a Windows XP machine tried to open the database, my map would not display. None of those problems are Mapserver’s fault of course, but you can’t roll out a map that has inaccurate data on it, and might not alwats work.

So, after a few years of muttering about this, I have finally bitten the bullet. I am working, in stages, towards an integrated database for all of our finds and archive information, in a PostgreSQL database, with both an Open Office base front-end for querying, and a web-based map interface. I’ll blog about each stage in this process, starting with integrating and data-cleansing in Microsoft Access.

The title of this post refers to the incredible feeling of achievement that I had when I got all this data together, followed by the difficulty I’m having moving that data into PostgreSQL. More later…

Jobs for alien archaeologists

The science might be a little dodgy (No more methane at all? What about all that decaying plant matter?), but I think this image is great. You’ve got to wonder what alien archaeologists will make of our heavy metals and particularly stubborn plastics though, in a time as far ahead of now as we are ahead of the palaeolithic…

(Goes off in a fit of navel-gazing about the lack of permancy of anything…)

Are the big guys squaring up?

There have been a few heads-up over the last week about 52°North, who have just announced an initiative for geospatial open source software. Of course, we’ve heard this before, but this time ESRI are on board.

So, we have Autodesk supporting OSGEO, and ESRI supporting 52North’s initiative (it needs a snappy title or acronym). The approach seems different though, because Autodesk chipped in right from the word go by open-sourcing one fork of MapGuide, which they freely admitted was not a core product. It appears however as if ESRI are not looking to open-source one of their existing products, but to help to develop new technologies, currently in the areas of Sensor Web Enablement (SWE), Security and Digital Rights Management (to quote from the press release).

Personally, I can’t decide if the big guys are squaring up for an opensource fight, or if we’re just seeing much more involvement in the opensource movement. Following on from the recent discussion about the use of opensource alternatives to ESRI’s main product lines, and further back, ESRI’s decision to include support for PostgreSQL as a database, hopefully this is all a sign of a gathering momentum. At first I thought this 52 North initiative was a bad thing- a dilution of focus, but maybe it’s just a sign that the snowball’s getting that bit bigger. More products, and some healthy competition has to be good, right?

Going Analogue

About three weeks ago I decided to give my Sharp Zaurus a well-earned rest and try going back to a paper-based approach to project planning and time management. Well, I say “going back” but in all honesty I’ve never tried the paper-based approach, it’s simply that I have never managed to find exactly what I want in a PDA to-do list/calendar, and there is always that low-level worry of data loss and breakage.

I’d been interested in the Hipster PDA/DIY Paper Planner approach for a while, and decided to give it a go, since the total investment (see below) was less than £20, so if it all went horribly wrong then I hadn’t lost much! So, from Waterstones I purchased a Moleskine Pocket Memo Folder, from my local generic stationers I purchased some 3×5 inch blank index cards, from my local art store I brought a really nice propelling pencil and fine-nibbed biro, and then I downloaded the HipsterPDA DIY Planner template.

Initital impressions:

The Moleskine had too many pockets for the number of cards I wanted initially. To get around this, I cut some of the card dividers out (sorry Moleskine). There is no loss of integrity with this approach because the dividers are separate card inserts, stuck at the sides but not at the bottom. This left me with three large pockets, each of which held a reasonable-sized stack of paper/card (see next point).

Index cards seemed too thick for what I wanted, so I ended up using them for the items that needed to be robust and long-lasting, such as the calendar and important contact details. The rest I printed out on A4 paper, and spent a happy hour or so cutting the individual cards out with a craft knife. Initially I printed way more cards than I needed, and didn’t print on both sides, but when I get around to reprinting I will use both sides (and only print the sections I want).

After a few weeks:

I think I like it!

I find it easier to write ideas down freely on paper rather than on a PDA, even using a sketchpad programme, so am tending to write down far more “speculative”ideas than I would with my zaurus.

There are some great day-planner templates, of the type recommended in Time Management for System Administrators. I’m trying to get into the habit (after reading that book) of arriving at work a few minutes early and planning my day before I even check my email. The only thing I allow myself to do first is change the data backup tape in case I forget later on. The day-planner is a great tool for combing a daily todo list and time planner (hence the name, I guess) and it really works for me. On one side is a list for tasks and spaces to assign priorities (and a nice tick box to check when it’s completed, which is always a bonus) and on the other is the work day laid out in hours, so it’s a small job to map out roughly how the day should pan out.

Other useful templates are the project planner, agenda, notes (obviously), shopping list and weekly time-tracker. This in particular translates very well at the end of the week to my work time-sheet. I haven’t really used any of the others, and consequently when I refill my Moleskine I probably won’t include them.

Is it the be-all and end-all?

Not sure. I find the 3×5 cards slightly too small (and the bigger ones too big), although this may be mitigated by printing on both sides of the page and only including the templates that I really need. I also like having my todo list and calendar integrated, and that’s more of an effort to do on paper than digitally, or online.

Is it the end for my zaurus?

Absolutely not! This frees up my zaurus for other things, like experimenting with other operating systems or software (and filling it up with ebooks and mp3s). It also saves me from the worry of having to ensure everything is backed up, particularly if I am playing around with other packages.

In conclusion, I’m still trying to find the best approach for time/project management and todo lists but this does give me some flexibility and peace of mind that my previous pda-based approach did not. We’ll see…