It was a misty Saturday morning, to ward off the evil “Customer Spirits” the systems admin was wearing a “Hachimaki”. Today he knew was going to be a long day and the two other systems admins would be a great asset. There was a great deal of work to do, there had been lies told to ensure that the work would go ahead, there had been some subtle diversionary work first and more than one plan had been prepared. All this just to re-install some servers to the standard desired by our European colleagues, still if all went according to plan it would be worth while. One of the initial concerns was ensuring that there was no possibility of someone who understood what we were about to do finding out that we were about to do it, on that count – “Give the Systems Admin a Cigar”. We made the BOFH look like a total amateur, the customer had not a clue about what was going to happen to all of the production systems. Today we would tackle all of the easy stuff, simply moving containers from one server to an other along with the associated disk. By early evening we were well on track, in fact we were almost exactly on the timetable for the move and upgrades if you looked at the plan.
As the second day dawned we all met at the office, here we started the more complicated part of the work. We had decided that we’d do the easiest bit first and how well it had gone, but as we discovered it was remarkably easy to snatch disaster from the jaws of total success! Gradually as the day progressed we fell further and further behind the plan, although we were making progress there just didn’t seem to be enough of it. Still not to worry too much as we had a full day in hand for contingency, we would probably need some of that I thought to myself as the day ground inexorably onward. By the end of the day, it was clear that we may even be struggling to finish by the end of the full contingency day that we had. It was decision time, so in the best traditions we went to the pub for a planning meeting. A great deal of planning was discussed in the pub and over dinner, during the course of the evening we did consider killing one of the other systems admins. He had single handedly caused most of the problems that we encountered, apparently without effort.
By the next morning these carefully considered plans had all been forgotten, with the exception of the one where the other systems admin dies a horrible death! So we started using the contingency day that we had set asside, in the morning the main question was how much of it would we need. By early evening the question was would be be able to deliver any thing close to what was required, as it turned out yes. Although there were teething problems, these were all resolved during the first actual working day. As the third day drew to a close, the check list became shorter and shorter and it was clear that we had scored a victory. Albeit there had been a certain amount of lost service on some of the servers, my estimate of the value of the work is that it will ensure that the project has a good chance of being completed more or less on schedule. Had we not done the work in this window of opportunity then it is likely that it would have been impossible to do in the future, so there we would be with the same old crap to deal with going forward.
On the first working day there was some flak to deal with, the awkward questions asked were easy to deal with. There were questions which all seemed to be looking to find a culprit, well I can tell you who the culprit was – it was you with your easy attitude to the proccess! You were delinquent in your duties, you obviously didn’t pay attention at school, you only can see as far as the weekend as it’s the next major event in your working life. So you should remember – when you see a systems admin with a head band on, there’s probably cause to ask a few questions………