One more commit

Last week I posted an article on the future of Java build tools, which received lots of positive feedback both here on this blog and on JavaLobby. That post focused more on the differences between various build tools and advantages of one over another. This week I decided to follow up with an article, which instead will talk about something that all kinds of software have in common: the risk of feature creep (in the context of build tools).

Feature creep in action

Build scripts are very fragile pieces of software. They are treated differently than other programs that we write. The companies coding rules rarely apply to code written in XML, and refactoring or testing build scripts is something that’s hardly ever done. Because of that, it’s often easier to break the build by changing the build logic, than it is by changing application logic (the other one happens more often, but that’s because the application code is changed way more frequently).

The other problem, which is sometimes mentioned as a main headache with build scripts, is their complexity. Scripts can get thousands of LOC long, split (chaotically) over a dozen or more files. Can you imagine the pleasure of debugging those, when something starts failing?

In my opinion fighting complexity of build scripts can be done two-fold:

• by using a better build tool
• by reducing the scope of the script

Compared to regular application development, picking a build tool (Ant, Maven, Gradle, etc.) is more or less like choosing a programming language: you consider IDE support, friendliness of the syntax, sometimes performance characteristics (rarely ever done for build tools, though). There’s also your team’s qualifications, and more often than not, your companies standards (so the choice can be completely out of your hands).

On the other hand, there’s reducing scope, or managing feature creep if you will. The more your build script does, the longer and more complex it becomes. The more features you add over the time, the less maintainable it comes to be. This leads to conclusion that the smaller the scope, and the rarer the changes to it are, the better.

Build scripts can range from the most ascetic, which only build and package the application, to the most byzantine, which will download the applications sources, compile, package, test, generate reports, generate and upload documentation, install all prerequisites on the build and deployment machines and finally deploy and run the built application. As the CI servers grow in popularity, it is possible to do more and more with appropriate plugins instead of home-made build scripts, but while they help reducing the scope a little, there’s still a lot of stuff that they can not do.

It seems reasonable to ask: what should a build script actually do then? Should it stick to the very basics? Does it make sense to have A-Z build scripts which can do everything with a single command? I wish I could give you an answer to those questions right here and right now, but I can’t. The feature list of a build script is in my opinion heavily dependent on the application itself, and the technologies in use.

One thing that could possibly help fighting the feature creep, is defining what the build script should do in advance. What reports are relevant to this project? Should the script be able to deploy the application, and if yes, which deployment scenarios should it support? Should it just generate, or also upload the build artifacts, like .jar files or documentation somewhere? What kind of notifications should be implemented, and so on. With this, a build script once created, should be left relatively unchanged. However, this seems rather restrictive, and definitely not “agile” at all.

The more agile approach could be by mimicking what has been done in the Grails framework, which has a very friendly integrated build system (based on Gant), which can be easily extended. As Grails is very keen on “convention-over-configuration”, the commands reside in appropriately named files, thus helping you find the code you’re interested in easier. The separation is cleaner, and adding your own command is unobtrusive and straightforward. The problem here is mostly time, because rolling a build system like that on your own is definitely harder than using something “out of the box”.

So the best practices I can name are: do as little as possible, but as much as necessary and keep your build code in bite-size, easily identifiable pieces - not one 10k LOC build file, but maybe dozens of small files named by convention. Not much, huh? That’s why I’m counting on you, to comment on this post, and share your opinions.

Can you share any good/bad practices for build scripts? Do CI servers make your life easier, and if yes, how so, and which one do you use? How often does your build script change? Do you have any particular way of organizing your build scripts?

Waiting for you feedback,

Adam

PS. Thanks to Pete Johnson for inspiring me to write this:)

I’ve recently watched a great presentation by Venkat Subramaniam titled “Pragmatic factors for Agile Success“. It’s about one and a half hours long, but if you’re doing agile project management in your company, I believe you should definitely watch it. Venkat talks about some good and bad practices, and mostly deals with communication issues. The presentation is cleverly organized into parts, each of which having two sides of it’s own - one beginning with a little devil saying some stuff (the bad practice) and the other one with a little angel saying how the things should work.

The part of the presentation, which in my opinion got the most attention from the audience (and myself), was the part where Venkat talked about the idea to criticize ideas and not people and dealing with communication between team members in general. I believe that this part is highly relevant to lots of people, because the every day communication is possibly the hardest thing to get right in any team. I strongly support the ideas of egoless programming and making design (or for that matter any other kind) decisions based solely on the solutions merits and drawbacks. The latter is the idea that I’d like discuss a little bit more in this post.

I think that although picking some solution based solely on it’s advantages and disadvantages is the only logical thing that comes to mind, I disagree that it is possible to just “let go” of the idea’s ownership. Of course you should go ahead anyway and try not to think about the solution in terms of “my idea” or “someone else’s idea”, the problem is that no matter how hard you try there is just no possibility of really doing that. Sure, you can release the idea, no longer considering “yours”, but if you have already put a substantial amount of thought into it, it’s become a product of your knowledge, experience, your character and beliefs. Your idea is yours not because you consider it yours or not, it’s yours because you’ve made your mark on it.

Hardly ever do you encounter a problem which solutions can be objectively evaluated to pick the best one. Most of the decisions we make are made totally subjectively. Of course in software design the subjectivity level is lower than in for example making business decisions, but it’s still there. From my experience, the area in software design that raises the most controversy is user interface design. The reason for that is that in user interface design you often base your opinions on your own preference. Something that is unintuitive for one person may be perfectly intuitive for another. Other design decisions can be evaluated a little better - something either handles concurrency, or it doesn’t, it handles all the functional and non-functional requirement’s or it doesn’t. However, even if you created a list of facts about some design, which are objectively true, people still can differ about how important those features are, as advantages or disadvantages. What I’m saying is, even when everybody in the team is perfect and letting go of the idea ownership - it’s still pretty hard to arrive at a conclusive decision without someone coming and saying: “OK, I’ve heard all your arguments - let’s do it this way” and ending the discussion. It’s not anybody’s fault - it’s just human nature.

In Agile the communication between the team members is of utmost importance. It’s really hard to succeed if you can’t talk with your team, so we should all try our best and learn communication skills. But if you have a colleague, that disagrees with your opinions and criticizes your designs, don’t think that you’ll change this by just changing the way you speak to that person. You have to understand that the differences between you will still be there, and you’ll still disagree with each other, but that’s the way it should be. The key is to seek those differences, because you might learn from them, and when you still disagree at the end of the day - just learn to let go, sometimes the other persons idea is different, but just as good.

Psyho