Converting an Integer to an IPv4 IP Address in Bash Using Bitwise Operations

I needed a script to automate the generation of statically assigned IP addresses, but I could not find any good example. It's a simple algorithm, so I decided I'd just write it up myself, but then I discovered that I could not find a good example of using bitwise operators in Bash. I found many pages where the operators are listed, but there is always the mention that they really aren't used in scripts all that often. Well, thanks for that, but how about one example of how to use them?

I came up with the following code, though I'm not sure if the $(( )) constructs are really necessary, but I simply could not make things work without them. Please let me know if you know a simpler syntax.

#!/bin/sh
#
# The IP address in this example is simply harcoded.
# In actual use, I read and write this from a file.
#
# 192.168.1.100
ipint=3232235876

MASKA=0xFF000000
MASKB=0x00FF0000
MASKC=0x0000FF00
MASKD=0x000000FF

a=$(( ($ipint & $MASKA) >> 24 ))
b=$(( ($ipint & $MASKB) >> 16 ))
c=$(( ($ipint & $MASKC) >> 8 ))
d=$(( $ipint & $MASKD ))

ipstr="$a.$b.$c.$d"

echo "The IP Address is ${ipaddr}."

The New Microsoft Tax

Apparently Microsoft now wants to expand the Microsoft Tax to cover everyone. That's pretty clever, now isn't it?

Simple Filter to Extract Links from a Pidgin Log

I often trade political links via the Pidgin IM client with my friend Jeremiah. Last week, he had the idea that we should coauthor a blog about these links. Towards this end I decided I harvest all of the links from my Pidgin log. This script will do that:

grep http ~/.purple/logs/aim/yourimid/friendsimid/* | grep -v -E "content-type|funpic\.hu|funnyjunk" | sed -e "s/^.*href=\"//" -e "s/\">.*//" | grep -v "font color"

The first grep finds anything that looks like a link, the second filters out any sites you don't care about. You can add more to that list by adding more "|sitename" clauses to the regex. The sed command scours off the html that Pidgin puts around anything and the last grep kills off some oddball lines that made it through the filters.

I'm sure this could all be made more efficient, but it did the job and unless you had an enormous quantity of logs to search, it's efficient enough.

Ditching the Euro?

I know it's a little premature to forecast the abandonment of the Euro and a return to national currencies, but I was surprised to find that some people are already discussing the topic.

Blogging, JavaScript and CSS

I've started posting little bits of code to my blog, and I thought it would be nice to have some sort of a code tag, similar to how blocks of code are set apart on Gentoo Forums. My first thought was that there must certainly already be something like that already for Blogger. Well, no. Then I decided to just start using <pre> tags. That works, but then you lose line wrapping. The then realized that I was going to have to write some CSS. I hate having to delve into CSS because despite the fact that I spend all day, every day coding in Java in Eclipse, I don't have much time for web programming and I find writing HTML or programming in loosely typed languages to be distasteful.

I won't go into all of the details, but even that has turned out to be more involved than I was hoping. At least to do it right. After trying several things, I've finally come up with a little bit of CSS and JavaScript that work together to do a mediocre job of what I want to accomplish. You'll see the results in this blog posting.

First, I created this CSS that I added directly to my Blogger template in Layout -> Edit HTML:

.code {
padding: .5em;
border-right: #d1d7dc 1px solid;
border-top: #d1d7dc 1px solid;
border-left: #d1d7dc 1px solid;
color: #000000;
border-bottom: #d1d7dc 1px solid;
font-family: 'Liberation Mono', Courier, 'Courier New', monospace;
font-style: bold;
background-color: #ffffcc
}

That gives me the block that I want, setting off the code from the article text. But then I noticed that any extra spaces were being lost as happens with HTML. That led to this bit of JavaScript added to the HTML/JavaScript block in Layout -> Page Elements:

<script type="text/javascript">
var divs = document.getElementsByTagName('div');
for(var i = 0; i < divs.length; i++) {
if(divs[i].className == 'code') {
str = divs[i].innerHTML;
str = str.replace(/ /g, '&nbsp;');
divs[i].innerHTML = str;
}
}
</script>

This is very sub-optimal, though, as the regex in replace() is overly simplistic. I tried several expressions, trying to get something better, but had no luck. What needs to happen is that all blocks of multiple spaces at the beginning of a code line need to be replaced with &nbsp;. I simply could not figure out a single expression that would replace a variable number of spaces with a corresponding quantity of &nbsp;, and only do it on the beginning of a line. If you can think of something that might work, let me know. Otherwise, I may just write some code to inspect the line and figure out what spaces need to be replaced.

I'll come back around to this when I have more time. Or, I might just use Alex Gorbatchev's Syntax Highlighter, but that was way more involved than I wanted to deal with today.

Awesome Steaks

My wife has some friends who are chefs who told me the way to cook a steak in an oven and have it come out excellent. Here is the procedure:

• Ahead of time, whip butter with salt and pepper, fresh chopped thyme, a little fresh lemon juice, fold in roquefort cheese. Roll in wax paper into a cylinder shape and chill. The wax paper should be rolled around the butter like a tube, not rolled up with the butter like a newspaper.
  
• Steaks should be at least 1" thick. I used 1.5".
• Preheat oven to 375° to 400° F.
  
• Pat steaks dry. Season to taste.
• Heat a little oil in a pan over medium heat. When pepper dropped into the oil sizzles, add steaks. Sear on each side for 2 to 3 minutes. I went a little short because the steaks started looking done VERY fast. Do not be alarmed. Next time I'll do the entire 3 minutes on my thick steaks.
• Place whole pan in oven for 3-5 minutes until the steaks are done to taste.
• Cut butter cylinder into slices. Place butter slices on steaks while they are hot from the oven so the butter melts over the steak.
  
• Eat.
  

Simple Filters in Perl

The other day I needed to pull some XML out of a log file. Some of the XML is in human readable format, spanning multiple lines. I changed my logging to spit out a tag before and after the XML to make it easy to mechanically separate the XML from the rest of the log.¹ The tags I used were "--- XML Request" or "--- XML Response" at the beginning of the line before the XML and "--- End XML" at the beginning of the line after the end of the XML. The Perl I came up with to filter these logs is:

#!/usr/bin/perl -w

$in_xml = 0;

while (<>&) {
if ($in_xml) {
print $_;
if ($_ =~ /^--- End/) {
$in_xml = 0;
}
} else {
if ($_ =~ /^--- XML/) {
print $_;
$in_xml = 1;
}
}
}

I then remembered that I didn't have to specify the =~ or $_, Perl being able to assume that for you, my revised version is:

#!/usr/bin/perl -w

$in_xml = 0;

while (<>) {
if ($in_xml) {
print $_;
if (/^--- End/) {
$in_xml = 0;
}
} else {
if (/^--- XML/) {
print $_;
$in_xml = 1;
}
}
}

¹ This is entirely unlike mechanically separated chicken.

Out of Scope

Why is it that every time I opine aloud that Java would benefit from a destructor, something that gets called the moment an object goes out of scope, people start whining about memory leaks and saying memory management is hard? I didn't say ditch the garbage collector. Since you have brought up the subject, let me say again that memory management is not hard.

Java Needs Destructors

I've been a Java programmer for seven years now. Java is no C++, but it's not a bad language, especially in it's Java 6 incarnation. There are just a few things about it, though, that continue to really get under my skin. This biggest is the lack of destructors in Java. In C++, you'll often see code like this:

public void foo() {
    ResourceIntensiveClass ric = new ResourceIntensiveClass("have bugs");
    ric.initiateResourceLockingStuff();
    ric.exceptionRiddledFunction();
}

And you can be comfortable that, if written correctly, ResourceIntesiveClass' destructor will be called and all resources freed when ric goes out of scope, regardless of whether it was under control or because of an exception thrown by ResourceIntensiveClass::exceptionRiddledFunction().

Unfortunately, when Java was being designed, the designers seem to want to get rid of every aspect of C++ that gave people trouble. Many people have trouble with destructors, so they are completely non-existent in Java. The only possible replacement, the finally clause to the try/catch block, requires the programmers who use a class to remember to call resource freeing methods of a class in their finally blocks. Every time. Without fail. Good luck. Java code is needlessly much more verbose for the same task than many other languages. Consider this Java equivalent to the code above:

public void foo() throws Exception {
    ResourceIntensiveClass ric = null;
    try {
        ric = new ResourceIntensiveClass("have bugs");
        ric.initiateResourceLockingStuff();
        ric.exceptionRiddledFunction();
    } finally {
        if (ric != null) {
            ric.manuallyCalledCleanupMethod();
        }
    }
}

Twelve lines to the first example's five lines. More typing and more opportunities for programmer induced bugs.

When is Java going to grow up and get destructors?

Ungovernable by Design

I often find myself saying that one form of tyranny is the tyranny of the majority electorate over the minority electorate. This is generally during a conversation discussing the Constitution and how the United States of America is a republic, not a democracy. There seem to be those with grand ideas about how the USA could be a "better" country who seem to find the Constitution and the, at least intended, weakness of the federal government to be a hindrance to their plans. Apparently one such person has even bemoaned that America has become ungovernable. Adam Graham has written an excellent article about how and why this is so. The point is, be glad that it is.

Java Timer for Running Recurrent Housekeeping Tasks

My company has several products which use long running Java programs. Occasionally, these programs need to do some sort of recurrent houskeeping tasks, such as deleting old files or some such task. I've found that the Java Timer class is very helpful for this. First, I create a class that I call a housekeeper to keep the Timer instance and load it with the houskeeping classes needed by the program. The housekeeper looks like this:

package net.tadland.examples.timer;

import java.util.Timer;

public class Housekeeper {

// The task delay is how many milliseconds transpire between the time
// that the task is scheduled and its first run. The task period is
// the time interval between runs.

private static final long TASK1DELAY = 10000;
private static final long TASK1PERIOD = 71 * 60 * 1000; // Run every 71 minutes;
private static final long TASK2DELAY = 60000;
private static final long TASK2PERIOD = 45 * 60 * 1000; // Run every 45 minutes;

private static Housekeeper me = new Housekeeper();

private Timer housekeeperTimer;

private Housekeeper() {
super();

// The boolean parameter here indicates that the timer is
// to be run as a daemon. It will continue running, launching
// each TimerTask specified below, over and over, running
// each TimerTask every period milliseconds.

housekeeperTimer = new Timer("Housekeeper", true);
housekeeperTimer.schedule(new HousekeepingTask1(), TASK1DELAY, TASK1PERIOD);
housekeeperTimer.schedule(new HousekeepingTask2(), TASK2DELAY, TASK2PERIOD);
}

public static synchronized void stop() {
if (me != null) {
me.housekeeperTimer.cancel();
me = null;
}
}
}

Then you will need one or more housekeeping task classes. I create one class per discrete task. A trivial example is:

package net.tadland.examples.timer;

import java.util.TimerTask;

public class HousekeepingTask1 extends TimerTask {

/*
* Since an instance of this class is created once when the daemon
* Housekeeper is created, don't store any transient data in class variables
* in this worker class. Data relative to a given run of this housekeeper
* should be in variables local to the run() method of this class.
*/

@Override
public void run() {
// Do your housekeeping tasks here.
}
}

There are other ways to use the Timer and TimerTasks, including scheduling tasks to run once at a specific time. The way I've illustrated Timers here is how I've successfully used Timers and TimerTasks to perform recurring houskeeping chores in an unattended system which is deployed at many locations and which runs for months at a time.

Our Phony Economy

Harper's magazine has an interesting article about what they call our "Phony Economy". Maybe all growth isn't good, nor is lack of it bad.

Is Your Password Secure?

Weak passwords continue to be a major problem in IT. Even your Facebook account should not have a weak password.

You Cannot be a Programmer Without Understanding Computers

In The Perils of JavaSchools, Joel Spolsky pretty accurately describes the problems with a lot of programmers who are coming out of colleges today. Back in the mid to late 1980s when I was in college, C++ was brand new and Java didn't yet exist. Most classes still used Pascal, though C was becoming more and more popular. The thing is, our classes back then mostly taught concepts, not languages. I even had to take a hardware digital logic class. (Much fun!) The only classes I really remember are Assembler (IBM 370 assembler at that!), Data Structures (taught by Jeff Harris, who last I heard, went off to a very well paying job at Motorola after being let go (!) by a university that obviously didn't appreciate the tremendous value of his teaching), Systems Design, where we wrote a software coputer and then later an assembler and linker for it. Everything else was fun, but just entertainment. Those three classes are the ones where I learned stuff that became the foundation for everything I've done since then. The sad thing is now, none of those classes are even offered, much less required.

This is Certainly a First

A friend sent me a link Well, That Certainly Didn’t Take Long and told me, "read this article...and pretend like Limbaugh or Noonan wrote it. I'm in somewhat of a state of shock." I generally don't read columns from the likes of Maureen Dowd. As a matter of fact, I had to ask my friend who she is. His response was, "ultra-ultra-ultra lefty; Huffington crowd".

Well, that was certainly interesting. Now I'm in a state of shock. Happy, but still shocked. Some leftist acknowledgment that there's a lot of liberal agenda in the stimulus bill as is and that even some Democrats are getting disillusioned is very refreshing.

Libertarian Frustration

A friend pointed to me to an excellent article by John Hasnas, Associate Professor in the McDonough School of Business at Georgetown University. In this short article, titled What It Feels Like To Be A Libertarian, Mr. Hasnas accuratly sums up the frustration felt by Libertarians, as they watch markets be manipulated by politicians who later declare that free markets don't work. The frustration that comes from watching "fixes" that make the problems worse.

Righteous Politicians

I'm already tired of the righteousness I'm perceiving from the Democrats in Congress. I'd like to remind them that they rode into office on the coattails of a man for whom people voted for not primarily for his policies. The reasons I believe many people voted for Barack Obama are, in order:

1. He's not George W. Bush.
2. Sarah Palin was perceived to not be qualified for a job she was not even running for. I'm still scratching my head over this one.
3. His race. I'm sorry to say this, but I really do think that for some voters at least, the color of his skin was more important than the content of his character. Hopefully someday Dr. King's dream will be achieved, but I don't believe it was this time.
4. His policies.

With Democratic policies in fourth place for a lot of people, I feel like the Democrats should be very careful. They need to stick to their bipartisan¹ promises. They need to do what is right for the country, not what rewards their pet issues.

I am still hopeful that Mr. Obama can guide his party to a higher standard, but I have my doubts, considering what he has to work with. Despite all of the Democratic complaints during the past eight years of the Bush administration, they seem to be willing to act in the same ways they complained about now that they are in power. I'm hoping for better, but not expecting it.

1) I hate the word bipartisan as it implies their is something inherently "right" about a bipolar system. I believe that our cyclical tit-for-tat approach to politics does far more harm than good.

New in Java 7

Java 5 was the first version of Java that I found usable after years of developing in C and C++. The inclusion of generics and other syntax enhancements combined with the appearance of Eclipse made developing in Java finally much less tedious and more productive. Java 6 seemed like more of a maintenance release, though it did provide some nice enhancements such as a better JAXB. I am particularly excited about some of the things promised in Java 7, including closures, BigDecimal operator support, type inference, improved catch clauses, and other fixes that will make working with collections more natural. I still wish they would just give us real operator overloading. The one that has me scratching my head, though, is automatic resource block management. In an attempt to rid Java code of most finally code, they have come up with:

    do (BufferedInputStream bis = new BufferedInputStream(is); BufferedOutputStream bos = BufferedOutputStream(os)) {
  // do stuff with bis and bos
}

Which is certainly an improvement, but I just don't understand why we can't get a real destructor in Java. My nearly twenty years of OO programming have convinced me that if a language has a constructor, it should have a destructor. Exceptions are academic languages and pseudo code, both of which are free of resource concerns. Real world programming is almost always working with some sort of a finite resource, such as database connections. It is not just good form to clean these up the moment you are done with them, but often simply necessary to make things work in a busy production system.

These new blocks are an improvement I suppose, but they still suffer the fatal flaw of resource management in Java which is that it is dependent upon the programmer to always remember to free up resources. My real world experience has also taught me that the less that you have to have programmers do manually, the less problems you'll have. I'll gladly accept the new features of Java 7, but I'm still waiting for a truly automatic solution to resource management in Java.

Don't Just Stand There, DO Something!

Finally, an article that quantifies what I have often felt but didn't know how to explain concerning the inefficiencies of big corporations: They become inefficient because they have too much process, or, as Paul Graham explains, because they are too careful.

On a smaller scale, I've realized that it is often better to simply DO something rather than spend too much time thinking, or really, agonizing, about how it should be done. If you're not sure how to proceed, then simply exclude any stupid directions and pick randomly from what is left. If you happen to have chosen the right path, then you are done. If you chose wrong, then you have effectively reduced the number of choices and can perhaps even use the experienced gained to choose better between those that are left. Often you can go through several cycles of trial and error before you could have made a decision simply by thinking about it.

You may say there are some endeavors where this does not work such as spaceflight or safety systems, but I suggest you can use this same principle there. Not that you won't have failure, but if you are working in engineering, you should be working with test harnesses and other simulators. In software, this has led to "agile development" or "test driven development". If you're not doing this, start.

No Way Can We Be Alone

Consider this image taken by the European Southern Observatory deep into the universe. Those aren't stars. They're galaxies. Tens of thousands of them. Each containing billions of stars. Even if earth-like worlds are one in a billion, there must be tens of thousands of earths in this photo. There is no way that there isn't intelligent life somewhere out there.

To those who say intelligent life elsewhere is too unlikely, I say you're just not capable of grasping just how huge the universe is.