IronMan. The project and the hack day.
IronMan. The project and the hack day.
IronMan was and continues to be a project to promote the Perl programming language by encouraging a community of bloggers to post on a regular basis and offering a token reward for their efforts (link needed here). Each of these blogs is then agregated to a single point for the community and the world at large to read, and of course for the assorted search engines to index.
The IronMan hack day 2009.
On Saturday 12th December 2009, the North West England Perl Mongers gathered together at the Shadowcat offices in Lancaster for our first hack day to work on the IronMan project.
The Enlightened Perl Organisation established the IronMan project and were very keen to see an archive so that all the posts that were being aggregated could be viewed retrospectively.
It was felt that the current organic architecture wasn’t shaping up in the best way for the direction that the project needed to take so a new approach was envisaged.
Who turned up?
mst, mdk, idn, iain, fade, grim, acid2, Leigh, epitaph
By the magic of the intarweb (IRC):
Sorry if I’ve missed anyone, it wasn’t intentional. If I’ve missed you shout up and I’ll add you on.
Someone had the inspired idea to draw a small diagram of the room and who was sitting where to enable us to look like we knew everyones names
What are we working on?
mst talked to us about the existing and proposed architecture making extensive use of the conference room white boards as we went along.
The legacy used Plagger to collect blog feeds and aggregate them into a statically generated planet. New feeds are signed up via a basic web form. The feeds data was stored in a SQLite database file via DBIx::Class, while the post data was stored in CSV files by Plagger.
The new would replace Plagger with Perlanet, and the statically generated content with a dynamic Catalyst web UI. The CSV files would move into the SQLite database and be stored in a table to enable the dynamic generation and archive ability.
Who was doing what?
mst took the overall role of technical architect and had a good idea of where he wanted to go with the project flitting from one place to another as needed.
acid2 picked up Perlanet & Perlanet::IronMan and proceeded to hack it into shape for our particular needs.
fade and iain continued working on the IronMan::Web aspect which they’d already started working on prior to the hack day. Some might say this is cheating, I say that it’s a good way to know what you’re doing in advance.
grim & castaway took the original DBIx::Class IronMan::Schema and modified it to support holding the blog posts. Alongside this, they began working on tools to import the legacy Plagger CSV data into the new schema.
Leigh sat amongst us and worked on her NaNoRiMo, and why not.
Refreshments and the essential caffine, pizza and beer.
Tea and coffee from Lancasters premier coffee shop Atkinsons fuelled a coding haze that saw us write some Good Shit(tm). Sadly, being coding geeks means that your supply of coffee needs to be plentiful and strong. mdk had to make a trip to fetch more coffee when we drained the office supplies dry!
Towards the end of the day, Shadowcat shelled out to further fuel the session with pizza, beer, nibbles and awesomeness.
What did we learn from the IronMan hack day?
Whilst much has been written about software development both in the commercial world and in the open source community, I’m taking this opportunity to reflect on my own view of the only open source project that I’ve been involved with.
So why bother to stop and reflect? It mostly went well!
The Prince 2 project management framework refers to a “Lessons Learned” log, it’s something you keep as a project progresses so you have something to review at the end. Problems encountered to avoid in future, that kind of thing.
We all examine our own code, if we’re lucky any documentation and such things, but do we take enough time to examine the organisational aspects of the process that brought us to this point?
These are my notes and ramblings on how we might have done things better.
Supporting infrastructure – laying the foundations.
Whilst it’s certainly possible to do these tasks on the day, it’s dead time for almost all of your developers. Do what you can to prepare as much as possible in advance and keep everyone in the zone for as long as possible.
Publicity and marketing.
There’s lots of information online on how to market events. Yes. I said the M word.
It’s important. If people don’t know about your event, they definitely won’t come even if it’s something they’d want to help out with.
Start pimping your event early and often. We had a good physical and electronic turnout. Even though we were all in the same room, we tended to use IRC for communication anyway, which allowed extensive contributions from outside the room.
Scope is all important as we all know. I’ve said already you can probably do all the suggestions here on the day and that’s certainly simple if you have four people.
What if you have two hundred? Hack days are a wonderful thing, but organising for two hundred and then having four turn up would be as bad as organising for four and having two hundred turn up. This could be really bad for you and for your project.
Knowing who you’ve got showing up allows you to think ahead and discuss aspects of your project that you want to split up into manageable chunks. With a little planning, the required information can be given to the right people who can then talk to others on the day. Think scalability and resilience, try to eliminate your single points of failure The more others know about the plan, the less the chance of a Guru Meditation Error.
Remember that physical presence is not required and extend your invitation to those who may not be able to physically attend but still wish to contribute.
Network and connectivity.
Now you know how many you’re expecting, you can think about the capacity of any network that you need to provide for the event. Can you get away with a sinlgle domestic wireless access point for your four people, or do you need something more substantial for your two hundred?
Despite his best efforts, epitaphs fight with the wireless and LAN through the course of the day resulted an a few outages stopping us all working.
I don’t care to argue which one you use, but you should use a VCS of some kind. We chose to use subversion. Regardless, the following points are still valid.
Create your repository or use an existing one.
Set appropriate permissions on the newly created or existing repo.
Find out who’s coming so you can create user accounts in advance.
Make sure its something your target group may be familiar with!
We lost time to repository creation and user setup that we could have reasonably been done beforehand.
Our venue not only provided very nice refreshments, but a large number of white boards. Never underestimate the power of explaining things with a whiteboard! It not only tells everyone else what you’re thinking, but helps to solidify your own thoughts.
Don’t underestimate how many 13A sockets you’re going to need for laptops.
What are we working on?
Turnout was good, we had a number of physical attendees along with people joining us via IRC. Everyone was fired up and ready to go. There was one major problem. We didn’t all understand why we were there, what we were working on or what the objectives for the day were.
Several hours were spent hashing out design, which might well have been needed, but some of the other fundamentals were missing.
1. What are we working on?
IronMan is a blog agregation service to promote Perl. A shop window for the world.
2. Why are we working on it?
We want to streamline the management of the existing service by replacing plagger with a new model based around Perlanet for feed collection.
We’re going to replace the existing static page creation with a Catalyst based web application that will display the content dynamically and permit people to view the blog post archives for specific days or months.
3. What are we hoping to achieve in the timeframe?
In the day, we’re going to:
* Get a good fundamental understanding of the existing architecture so that everyone knows how things work.
* Review and implement changes to the database model.
* Begin working on the Perlanet collection code.
* Begin working on the Catalyst web UI code.
It’s easy to list these things in retrospect, but we could have saved a large amount of time and effort if we’d have thought about these things before we all got together.
During the course of the day, we didn’t have a defined checkpoint to deploy what we had so that everyone else could see it.
Take the time in advance to configure a staging system so that at pre-defined points during the day, everyone commits their changes and the staging system updated and restarted as required.
Try not to break the build…
Beyond the hack day.
Upon these sage words of wisdom, your hackday went well, right? Remember to look back at what you’ve learned at each step of the process, what worked and what didn’t.
Once your hackday has been and gone, you’re left with the inevitable job of continuing to support the project you’ve been working on. This is a whole new set of challenges in much the same way as the organisation of the hack day itself.
Nobody likes writing it, everybody likes reading it. Well, most people.
To quote castaway: “documentation is hard”. She’s right of course.
If you’re lucky you’ll have a documentation whiz like castaway on your team, but if you don’t you either need to find one or try your hardest to make do with the people you have. In order to get people interested, you need to have a basic level of documentation in place or it will put people off from contributing to your project.
Document your version control system structure. Doesn’t matter how you lay things out as long as it’s clear what’s going on.
Once you have your repository structure documented, then document how you move through the different stages of release. I’m a simple bloke and generally like to roughly document the commands I use to ‘svn copy’ things from one area of the repository to another. It’s a simple thing, but it’s something that will save you time and keep you consistent in your naming.
If you don’t do the above, your repository may end up looking like a dogs breakfast and people will be put off contributing patches simply because they don’t have time to try and work out what’s going on. Post deployment of the IronMan project, it took around a mans day worth of work to get things into a sensible state, in the mean time several people were unable to contribute because they couldn’t work out which files they should be working on.
I’ve mentioned staging your development for the period of the hack day, you should do this all the time. Not only does it allow you to check that things will work on a machine that isn’t yours, it checks other peoples committed chanmges and gives the project visibility. If you don’t have visibility before you deploy to your live service, then you can’t get feedback from your user community.
Organising your next hack day.
So following your last successful hack day, you’re mad for organising another, right!?
I’m making one major change for this year. The North West England Perl Mongers haven’t decided what hack day 2010 will be yet, we’re asking the membership to present lightening talks for their project at the September technical meeting. This does a few very important things:
You can’t present on your project and sell it if you haven’t thought about it and what you want to achieve
If you want your project worked on this year, you need to sell it to the membership who will be voting at the end of the evening.
It might solve some of the preparation issues, it might not. Perhaps this time next year I’ll finally finish some more reflections…
Armed with your reflections and thoughts, bring free software to the masses, because we can and because it’s the socially responsible thing to do. Plus all th ecool kids are doing it….