Safe one-time bash scripts

Ok, let’s assume you have some bash commands you need to run on some servers, but you don’t want to write a bash script (file). This is when you would just copy those commands and paste them into the shell on the target servers.

Two good practices I would recommend in these situations is a) to run these commands inside a subshell and b) use “set -e” within it. With an example you should understand why:

For example I want to zip some old files and delete them after copying them on my backup server:
cd /var/log/
tar -czf ~/logs.2016.tgz *.2016.log
scp ~/logs.2016.tgz mybackup.com:archive/
rm *.2016.log

The danger here is the following: what if one of the first commands fail but the “rm” command runs successfully. Whoops – all files gone.
Now let’s put that into a sub shell with a save exit:
(
set -e
cd /var/log/
tar -czf ~/logs.2016.tgz *.2016.log
scp ~/logs.2016.tgz mybackup.com:archive/
rm *.2016.log
)

Now if the cd, tar or scp command fail, you will just see a warning and your “temporary script” stops. You are saved! :)

set log4j 2 configuration file on startup

If you’d like to configure the log4j 2 configuration file using a system/startup property of your java process (in my case this was tomcat 8), you’ll find many examples in the web with the property “-Dlog4j.configuration=file:{path-to-file}“. Attention: this does not work (any more?) with log4j 2!

According to the documentation the property is now “-Dlog4j.configurationFile={path-to-file}“. You might not see the difference at the first glance, but you should consider it if you want it to work. :)

From Developer to Software Engineer

The last few years I experienced the difference between software development and software engineering. I guess you heard about the “Definition of Done” before and I’ve seen some of those definitions, but I am still concerned, that the whole picture is rarely seen. As least I haven’t seen it a long time as a developer.

What does it mean to build a piece of software?

As a (fresh) developer you want to start coding as soon as you have a fraction of the basic idea, what the software should do. This is how we use to think, because coding is what we actually do and where we feel productive. However we are not used to questioning the requirements. Maybe there is something wrong already and we would just implement something unnecessary? Did we understand the purpose of the software, how it should be used and what its benefits are for daily work?

Often we ignore those questions and let someone else answer them. Maybe the product manager will take care of it. It’s not our business, how the sales guy will sell it and we barely have to care about how some operator will do the installation and keep it running.

Getting a software engineer means for me to start seeing that whole picture and start taking responsibility for all of those questions. It means to understand the requirements and already start searching for defects there. It means to understand the needs of the other teams and how your decisions influence their work. And finally it means to understand the users need, their environment and their business.

How can we get such a understanding? Well for me it helped to get in touch with those people – not just for a project or the development of a product, but also in daily live: have lunch with them or get out for a beer. Have an open ear for their problems without trying to solve them or doing their job. Just understand them and keep in mind, that your programming decisions might influence their work.

If you fail, at least learn from it

This morning I woke up very early on my own, around one hour before my early alarm, starting to think about a project I messed up 3 years ago and what I have could done better. Wait – What? Isn’t there anything better to think about at that time, like breakfast for example… anyways, this is how my brain works, I guess.

So I thought about the root causes why the project failed and what I could have done better or should have done completly different. Here are some learning I recognized:

1. Have the currage to find the best matching tool (aka never use the hammer for a screw)

Our software was not made for such an scenario, so I started to customize our solution to the requirements of the customer. If I would have taken a look on the requirements and thought about the impact on our software, I would have declined the project or simply used just the relevant parts from our software and combined it with software that already fulfiled the requirements.

2. Find your own decisions were ever possible

At some point I realized, it was too much data, for our software, to get the necessary performance. I needed a¬†database I could put in all the data, our software shouldn’t touch. But I haven’t used much databases at that time, so I asked the customer what kind of database I should use. I took the first one he told me to, without evaluating it or even thinking about it on a conceptual level, althoug I noticed the nescience of the customer. I was such a fool. It was horrible. I was working with a unfitting tool I never really wanted.

The better solution would have been, to find my own decision by finding experienced people and taking the time, to evaluate the necessary software in the context of the requirements. This is the essence of my next learning:

3. Take your time to make a plan – this is much more productive than producing code

4. Be honest about the shit

The most things I did were “motivated” by the preassure I got: the budget was very low, the customer was promised to get first results after two weeks and there was other work to do as well and the requirements never ever fit to the estimations.

If you are in a such a situation – don’t try to handle it. Be serious about your responsablity and tell your boss about it. I think it might even be an option to refuse doing such unnecessary work – it’s better for all. Instead strive for a solution that might work for all.

I know there is the SNAFU principle, but you must be as honest and detailed as possible.

5. If you fail, at least learn from it

The best to do so is to get some distance – for example 3 years ;)… From a distance you’ll get a proper view on the failure and are able to extract the experience from it. The best way however, is to learn from the failures of others, but this is much harder (and another topic).

mounting hfsplus rw on linux

The last weeks I struggled mounting my HFS+ formated Drobo on my Linux (Mint) PC in rw mode. Although it was mounting successfully, it always told me

mount: warning: /media/Drobo/ seems to be mounted read-only.

Of course all the necessary steps I was reading about on various sites were all fulfilled:

  • Journaling was disabled
  • I added the device into my /etc/fstab:
    • Identifying the device by UUID=…
    • “rw,force” as options

After trying the debug options:

> sudo mount UUID=... -t hfsplus -o rw,force,debug -v /media/Drobo/

and checking

> dmesg | tail

I got the answer for my problem:

hfsplus: Filesystem was not cleanly unmounted, running fsck.hfsplus is recommended.  mounting read-only.

Well, with this information it was easy. :)

> sudo fsck.hfsplus -f /dev/sdb2
...
** The volume Drobo was repaired successfully.

Afterwards mounting rw was possible without any problems again. The learning from the story: before searching the web, search the relevant logs! (RTFL!) ;) I hope I’ve learned that this time.

Checking if a String can be represented by a certain charset

I had the task to find out, if a character is part of a certain charset. What I did to achieve that is to just convert it and see if the new string is broken. It worked pretty well with some tests I made.

public static boolean isStringInCharset(final String sourceString, final String charsetName) throws UnsupportedEncodingException {

final String transcodedString = new String(sourceString.getBytes(charsetName));
return transcodedString.equals(sourceString);

}

However that was pretty easy – too easy for my taste.
Have I made a mistake?

What would you need for your perfect task planer?

For a while now I am searching for a task planer tool which fits to my needs. But I can’t find one, so I started to think about writing my own tool. I already started working on a mock-up which is representing my basic thoughts, but I fear this might get a huge project; so participate on another comparable project would also be an option for me.

The first step of course is to find out, what I actually need and what are my requirements? With answering this question there is also the possibility to find a matching tool.
However first of all I would like to know how are you organizing your tasks without loosing control and overview?
Which features would you love to see in your tool?

I am currently organizing my tasks with outlook and these are the features I would really love inside a task managing tool:

  • getting the information when each task might get finished; this should base on the estimated time I can specify for tasks, also considering my calendar, my daily working time, the need for breaks etc.
  • getting the information which task to do next; this one could be calculated on deadlines, priorities and estimated time
  • integration with my email account so new emails are automatically considered as a “has to be read task which need 5 minutes minimum”
  • tracking my work, the interruptions, breaks etc., so I can improve my workflows and get more productive

Do you know a tool that could satisfy my needs? :)