Wednesday, September 19, 2007

Quote of the day

This blog started up as a place to post interesting things I do. I haven't done much (that I would blog about), so here is a quote to keep the site live.


It may be a bit unusual, but my way of thinking of "distributed systems" was the 30+ year (and still continuing) effort to make many systems look like one. Distributed transactions, quorum algorithms, RPC, synchronous request-response, tightly-coupled schema, and similar efforts all try to mask the existence of independence from the application developer and from the user. In other words, make it look to the application like many systems are one system. While I have invested a significant portion of my career working in this effort, I have repented and believe that we are evolving away from this approach.
-- Pat Helland

Saturday, September 1, 2007

Phrase of the day

"There is surely nothing quite so useless as doing with great efficiency what should not be done at all" Peter Drucker, 1963

Thursday, July 5, 2007

Horror movies

I wish someone would do an horror movie showing the point of view of the killer, showing how he thinks, how he hides, how he picks the next victim, if he ever goes to the restroom, and so on.

Monday, June 4, 2007

Phrase of the day

"The pace of scientific progress may be hindered by the tendency of our academic institutions to reward being popular, rather than being right … Academia is still largely a medieval guild, with a few powerful elites, many slave-like apprentices, and members who hold a monopoly on the research patronage of princes and the teaching of their sons …" Robin Hanson

Thursday, May 31, 2007

Phrase of the day

"If I occasionally neglect to cite a theorist, it’s not because I’ve forgotten, it’s probably because I hate him.” Leon Lederman

Monday, May 28, 2007

Netbooting Ubuntu 7.04

When doing experiments in our cluster, we often have to reinstall all machines from scratch to build a new configuration. Instead of doing the installation by hand on each machine, we use netboot to install Ubuntu, and build our preseed files to install the software as we need.

The first time some friends and I did it, I had never heard of preseed files and, although I knew netboot was possible, I had never done it myself. Naturally, we followed a step by step tutorial.

In the end the tutorial didn't give us exactly what we wanted, so we did some adaptations. Anyway, it was quite good. Have a look here, and come back for the php file that you won't be able to retrieve from there. I am also posting a preseed file (preseed) that executes two post install scripts, one right after the installation (after-preseed) and one after the first boot (after-baseconf), when the NIS client daemon is already running. This is important for packages which create new users in the system, like mysql. The last script also installs Sun's jdk without asking for confirmation.

Don't forget to check the original howto and the links there.

PRESEED
#### Shell commands.

# This first command is run as early as possible, just after
# preseeding is read.
#d-i preseed/early_command string wget http://url/to/my.udeb -O /tmp/my.udeb ; udpkg -i /tmp/my.udeb

# This command is run just before the install finishes, but when there is
# still a usable /target directory.
d-i preseed/late_command string wget http://192.168.3.1/after-preseed; mv after-preseed /target/; chmod ugo+x /target/after-preseed; in-target /after-preseed; rm /target/after-preseed; wget http://192.168.3.1/after-baseconf; mv after-baseconf /target/; chmod ugo+x /target/after-baseconf; sed -i 's_exit 0_/bin/bash /after-baseconf_' /target/etc/rc.local;



# This command is run just as base-config is starting up.
#base-config base-config/early_command string echo hi mom

d-i netcfg/choose_interface select auto


# Note that any hostname and domain names assigned from dhcp take
# precedence over values set here. However, setting the values still
# prevents the questions from being shown even if values come from dhcp.
d-i netcfg/get_hostname string unassigned-hostname
d-i netcfg/get_domain string unassigned-domain

# Disable that annoying WEP key dialog.
d-i netcfg/wireless_wep string

#### Mirror settings.

d-i mirror/country string enter information manually
d-i mirror/http/hostname string 192.168.3.1
d-i mirror/http/directory string /ubuntu
d-i mirror/suite string feisty
d-i mirror/http/proxy string http://192.168.3.1:3128
base-config mirror/http/proxy string http://192.168.3.1:3128

### Partitioning.

d-i partman-auto/init_automatically_partition select Guided - use entire disk
d-i partman-auto/method string regular

# Alternatively, you can specify a disk to partition. The device name can
# be given in either devfs or traditional non-devfs format.
# For example, to use the first disk devfs knows of:
#d-i partman-auto/disk string /dev/discs/disc0/disc

# You can choose from any of the predefined partitioning recipes:
#d-i partman-auto/choose_recipe select All files in one partition (recommended for new users)
#d-i partman-auto/choose_recipe select Desktop machine
#d-i partman-auto/choose_recipe select Multi-user workstation
# Or provide a recipe of your own...
# The recipe format is documented in the file devel/partman-auto-recipe.txt.
# If you have a way to get a recipe file into the d-i environment, you can
# just point at it.
#d-i partman-auto/expert_recipe_file string /hd-media/recipe
# If not, you can put an entire recipe in one line. This example creates
# a small /boot partition, suitable swap, and uses the rest of the space
# for the root partition:
#d-i partman-auto/expert_recipe string boot-root :: 20 50 100 ext3 $primary{ } $bootable{ } method{ format } format{ } use_filesystem{ } filesystem{ ext3 } mountpoint{ /boot } . 500 10000 1000000000 ext3 method{ format } format{ } use_filesystem{ } filesystem{ ext3 } mountpoint{ / } . 64 512 300% linux-swap method{ swap } format{ } .
# For reference, here is that same recipe in a more readable form:
# boot-root ::
# 40 50 100 ext3
# $primary{ } $bootable{ }
# method{ format } format{ }
# use_filesystem{ } filesystem{ ext3 }
# mountpoint{ /boot }
# .
# 500 10000 1000000000 ext3
# method{ format } format{ }
# use_filesystem{ } filesystem{ ext3 }
# mountpoint{ / }
# .
# 64 512 300% linux-swap
# method{ swap } format{ }
# .
d-i partman-auto/expert_recipe string boot-root :: 40 50 100 ext3 $primary{ } $bootable{ } method{ format } format{ } use_filesystem{ } filesystem{ ext2 } mountpoint{ /boot } . 2048 2048 2048 linux-swap method{ swap } format{ } . 500 10000 1000000000 ext3 method{ format } format{ } use_filesystem{ } filesystem{ ext3 } mountpoint{ / } .
# boot-root ::
# 40 50 100 ext3
# $primary{ } $bootable{ }
# method{ format } format{ }
# use_filesystem{ } filesystem{ ext2 }
# mountpoint{ /boot }
# .
# 2048 2048 2048 linux-swap
# method{ swap } format{ }
# .
# 500 10000 1000000000 ext3
# method{ format } format{ }
# use_filesystem{ } filesystem{ ext3 }
# mountpoint{ / }
# .

# This makes partman automatically partition without confirmation.
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select Finish partitioning and write changes to disk
d-i partman/confirm boolean true

#### Boot loader installation.

# Grub is the default boot loader (for x86). If you want lilo installed
# instead, uncomment this:
#d-i grub-installer/skip boolean true

# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true
# This one makes grub-installer install to the MBR if if finds some other OS
# too, which is less safe as it might not be able to boot that other OS.
d-i grub-installer/with_other_os boolean true
# Alternatively, if you want to install to a location other than the mbr,
# uncomment and edit these lines:
#d-i grub-installer/bootdev string (hd0,0)
#d-i grub-installer/only-debian boolean false
#d-i grub-installer/with_other_os boolean false

##### Finishing up the first stage install.

# Avoid that last message about the install being complete.
d-i prebaseconfig/reboot_in_progress note

d-i mysql-server-5.0/nis_warning note
d-i sun-java5-bin shared/accepted-sun-dlj-v1-1 boolean true
d-i sun-java5-jdk shared/accepted-sun-dlj-v1-1 boolean true
d-i sun-java5-jre shared/accepted-sun-dlj-v1-1 boolean true
d-i sun-java5-jre sun-java5-jre/jcepolicy note
d-i sun-java5-bin shared/present-sun-dlj-v1-1 note
d-i sun-java5-jdk shared/present-sun-dlj-v1-1 note
d-i sun-java5-jre shared/present-sun-dlj-v1-1 note

##### Preseeding base-config.
# XXX: Note that most of this will not work right until base-config 2.40.4
# is available.

# Avoid the introductory message.
base-config base-config/intro note

# Avoid the final message.
base-config base-config/login note

# If you installed a display manager, but don't want to start it immediately
# after base-config finishes.
#base-config base-config/start-display-manager boolean false

###### Time zone setup.

# Controls whether or not the hardware clock is set to GMT.
#base-config tzconfig/gmt boolean true

# If you told the installer that you're in the United States, then you
# can set the time zone using this variable.
# (Choices are: Eastern, Central, Mountain, Pacific, Alaska, Hawaii,
# Aleutian, Arizona East-Indiana, Indiana-Starke, Michigan, Samoa, other)
#base-config tzconfig/choose_country_zone/US select Eastern
# If you told it you're in Canada.
# (Choices are: Newfoundland, Atlantic, Eastern, Central,
# East-Saskatchewan, Saskatchewan, Mountain, Pacific, Yukon, other)
#base-config tzconfig/choose_country_zone/CA select Eastern
# If you told it you're in Brazil. (Choices are: East, West, Acre,
# DeNoronha, other)
#base-config tzconfig/choose_country_zone/BR select East
# Many countries have only one time zone. If you told the installer you're
# in one of those countries, you can choose its standard time zone via this
# question.
#base-config tzconfig/choose_country_zone_single boolean true
# This question is asked as a fallback for countries other than those
# listed above, which have more than one time zone. You can preseed one of
# the time zones, or "other".
# base-config tzconfig/choose_country_zone_multiple select Europe/Amsterdam

d-i clock-setup/utc boolean true
d-i time/zone string Europe/Zurich

###### Account setup.

# To preseed the root password, you have to put it in the clear in this
# file. That is not a very good idea, use caution!
#passwd passwd/root-password password r00tme
#passwd passwd/root-password-again password r00tme

# If you want to skip creation of a normal user account.
#passwd passwd/make-user boolean false
# Alternatively, you can preseed the user's name and login.
passwd passwd/user-fullname string not root
passwd passwd/username string notroot
# And their password, but use caution!
passwd passwd/user-password password notroot
passwd passwd/user-password-again password notroot

###### Apt setup.

# This question controls what source the second stage installation uses
# for packages. Choices are cdrom, http, ftp, filesystem, edit sources list
# by hand
base-config apt-setup/uri_type select http

# If you choose ftp or http, you'll be asked for a country and a mirror.
base-config apt-setup/country select enter information manually
base-config apt-setup/hostname string archive.ubuntu.ch
base-config apt-setup/directory string /ubuntu
# Stop after choosing one mirror.
base-config apt-setup/another boolean false

# You can choose to install non-free and contrib software.
#base-config apt-setup/non-free boolean true
#base-config apt-setup/contrib boolean true
base-config apt-setup/universe boolean true

# Do enable security updates.
base-config apt-setup/security-updates boolean false


# Package selection
base-config base-config/package-selection string (~n^ubuntu-base$|~n^openssh-server$)|(~n^build-essential$|~n^slocate$)
base-config base-config/install-language-support boolean false
d-i base-config/install-language-support boolean false
###### Package selection.

# You can choose to install any combination of tasks that are available.
# Available tasks as of this writing include: Desktop environment,
# Web server, Print server, DNS server, File server, Mail server,
# SQL database, manual package selection. The last of those will run
# aptitude. You can also choose to install no tasks, and force the
# installation of a set of packages in some other way.
# XXX: this will not work until tasksel 2.12 is available
##tasksel tasksel/first multiselect Desktop environment
tasksel tasksel/first multiselect ubuntu-standard, ubuntu-desktop, lamp-server


###### Mailer configuration.

# During a normal install, exim asks only two questions. Here's how to
# avoid even those. More complicated preseeding is possible.
exim4-config exim4/dc_eximconfig_configtype select no configuration at this time
# It's a good idea to set this to whatever user account you choose to
# create. Leaving the value blank results in postmaster mail going to
# /var/mail/mail.
exim4-config exim4/dc_postmaster string

###### X Configuration.

# Preseeding Ubuntu's X config is possible, but you probably need to know
# some details about the video hardware of the machine, since Ubuntu's X
# configurator does not do fully automatic configuration of everything.

# X can detect the right driver for some cards, but if you're preseeding,
# you override whatever it chooses. Still, vesa will work most places.
#xserver-xorg xserver-xorg/config/device/driver select vesa

# A caveat with mouse autodetection is that if it fails, X will retry it
# over and over. So if it's preseeded to be done, there is a possibility of
# an infinite loop if the mouse is not autodetected.
#xserver-xorg xserver-xorg/autodetect_mouse boolean true

# Monitor autodetection is recommended.
xserver-xorg xserver-xorg/autodetect_monitor boolean true
# Uncomment if you have a LCD display.
#xserver-xorg xserver-xorg/config/monitor/lcd boolean true
# X has three configuration paths for the monitor. Here's how to preseed
# the "medium" path, which is always available. The "simple" path may not
# be available, and the "advanced" path asks too many questions.
xserver-xorg xserver-xorg/config/monitor/selection-method select medium
xserver-xorg xserver-xorg/config/monitor/mode-list select 1024x768 @ 60 Hz

###### Everything else.

# Depending on what software you choose to install, or if things go wrong
# during the installation process, it's possible that other questions may
# be asked. You can preseed those too, of course. To get a list of every
# possible question that could be asked during an install, do an
# installation, and then run these commands:
#debconf-get-selections --installer > /target/log
#debconf-get-selections >> /target/log

# If you like, you can include other preseed files into this one.
# Any settings in those files will override pre-existing settings from this
# file. More that one file can be listed, separated by spaces; all will be
# loaded. The included files can have preseed/include directives of their
# own as well. Note that if the filenames are relative, they are taken from
# the same directory as the preseed file that includes them.
#d-i preseed/include string x.cfg
# More flexably, this runs a shell command and if it outputs the names of
# preseed files, includes those files. For example, to switch configs based
# on a particular usb storage device (in this case, a built-in card reader):
#d-i preseed/include_command string if $(grep -q "GUID: 0aec3050aec305000001a003" /proc/scsi/usb-storage-*/*); then echo kraken.cfg; else echo otherusb.cfg; fi

d-i finish-install/reboot_in_progress note



----------------------------
AFTER-PRESEED
#!/bin/bash

## configure apt
cat /etc/apt/apt.conf > /tmp/log
touch /etc/apt/apt.conf
echo 'Acquire::http::Proxy "http://192.168.3.1:3128";' >> /etc/apt/apt.conf

##install usefull packages
aptitude -y install openssh-server acpi-support acpid wget elinks screen unzip >> /tmp/log
aptitude -y install gcc make >> /tmp/log






mkdir -p /root/.ssh
wget http://192.168.3.1/authorized_keys -O - > /root/.ssh/authorized_keys
ls /root/.ssh/ >> /tmp/log

# configure SUDOERS
#echo '%netadmin ALL=(ALL) ALL' >> /etc/sudoers
echo '%netadmin ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers

# configure NFS
aptitude -y install nfs-client
echo '192.168.3.1:/home /home nfs rw,soft,bg,rsize=8192,wsize=8192,intr 0 0' >> /etc/fstab
wget http://192.168.3.1/hosts -O - > /etc/hosts
cat /etc/hosts >> /tmp/log

# configure NIS
aptitude -y install portmap nis
echo 'portmap : dell01 192.168.3.1' >> /etc/hosts.allow
echo 'domain mydomain server 192.168.3.1' >> /etc/yp.conf
echo 'mydomain' > /etc/defaultdomain
echo '+::::::' >> /etc/passwd
echo '+:::' >> /etc/group
echo '+::::::::' >> /etc/shadow
echo 'order hosts,nis' > /etc/host.conf
echo 'multi on' >> /etc/host.conf


##install specific stuff
# mysql
#groupadd -g 1010 mysql
#useradd -g mysql -u 1010 mysql -d /var/lib/mysql
#aptitude -y install mysql-server-5.0 >> /tmp/log

# vmware
#aptitude -y install vmware-server-kernel-modules vmware-tools-kernel-modules linux-headers xinetd build-essential




# update apt
wget http://192.168.3.1/sources.list -O - > /etc/apt/sources.list
aptitude update
aptitude -y upgrade

# register node
wget http://192.168.3.1/register.php?mac=`ifconfig eth0 |grep HWaddr |cut -d' ' -f 11` -O /etc/pxe.id
echo register >> /tmp/log



----------------------------
AFTER-BASECONF
#!/bin/bash

##install specific stuff
#java

echo "sun-java5-jre sun-java5-jre/stopthread boolean true" | debconf-set-selections
echo "sun-java5-bin shared/accepted-sun-dlj-v1-1 boolean true" | debconf-set-selections
echo "sun-java5-jdk shared/accepted-sun-dlj-v1-1 boolean true" | debconf-set-selections
echo "sun-java5-jre shared/accepted-sun-dlj-v1-1 boolean true" | debconf-set-selections
echo "sun-java5-jre sun-java5-jre/jcepolicy note" | debconf-set-selections
echo "sun-java5-bin shared/present-sun-dlj-v1-1 note" | debconf-set-selections
echo "sun-java5-jdk shared/present-sun-dlj-v1-1 note" | debconf-set-selections
echo "sun-java5-jre shared/present-sun-dlj-v1-1 note" | debconf-set-selections

aptitude install -y sun-java5-jdk

# mysql
echo "mysql-seerver-5.0 mysql-server-5.0/nis_warning note" | debconf-set-selections
echo "postfix postfix/main_mailer_type select No configuration" | debconf-set-selections
echo "postfix postfix/root_address string" | debconf-set-selections
echo "postfix postfix/rfc1035_violation boolean false" | debconf-set-selections
echo "postfix postfix/master_upgrade_warning boolean" | debconf-set-selections
echo "postfix postfix/mydomain_warning boolean" | debconf-set-selections
echo "postfix postfix/mynetworks string 127.0.0.0/8" | debconf-set-selections
echo "postfix postfix/db_upgrade_warning boolean true" | debconf-set-selections
echo "postfix postfix/mailname string /etc/mailname" | debconf-set-selections
echo "postfix postfix/tlsmgr_upgrade_warning boolean" | debconf-set-selections
echo "postfix postfix/dynamicmaps_upgrade_warning boolean" | debconf-set-selections
echo "postfix postfix/recipient_delim string +" | debconf-set-selections
echo "postfix postfix/main_mailer_type select No configuration" | debconf-set-selections
echo "postfix postfix/destinations string" | debconf-set-selections
echo "postfix postfix/transport_map_warning note" | debconf-set-selections
echo "postfix postfix/nqmgr_upgrade_warning boolean" | debconf-set-selections
echo "postfix postfix/not_configured note" | debconf-set-selections
echo "postfix postfix/mailbox_limit string 0" | debconf-set-selections
echo "postfix postfix/relayhost string" | debconf-set-selections
echo "postfix postfix/procmail boolean" | debconf-set-selections
echo "postfix postfix/bad_recipient_delimiter note" | debconf-set-selections
echo "postfix postfix/protocols select" | debconf-set-selections
echo "postfix postfix/chattr boolean false" | debconf-set-selections
aptitude -y install mysql-server

Thursday, May 17, 2007

SystemImager on Ubuntu 7.04

To install SystemImager on Ubuntu Feisty Fawn
  1. Download the newest sources (I used the unstable 3.9 version). Download as root or give root the ownership of all the files.
  2. Edit the coreutils.rul
    1. Change the version in this file to 6.9 (mine was set to 5.93).
  3. Follow the instructions here: Linux @ BHSN: HOWTO Install SystemImager from source on Ubuntu Edgy