HPC Community - High Performance Computing (HPC) Community - Blogs - George Goh

How to have sles run your own program on installation

George Goh — Wed, 01 Oct 2008 16:13:00 GMT

1. Take the root image from a pristine SLES 10.2 repository, and copy
the contents of the root image to a working location.

mkdir -p $WORKING_DIR/mnt $WORKING_DIR/rootimg
mount -o loop $REPO_LOCATION/boot/i386/root $WORKING_DIR/mnt
cp -avr $WORKING_DIR/mnt/* $WORKING_DIR/rootimg
umount $WORKING_DIR/mnt

2. Replace the original �yast� script with my own script.

mv $WORKING_DIR/rootimg/sbin/yast $WORKING_DIR/rootimg/sbin/yast.real
cp $WORKSPACE/yast.mine $WORKING_DIR/rootimg/sbin/yast

3. When desired changes are made(add dependencies in step a), pack up the modified directory tree
into a cramfs image:

mkfs.cramfs $WORKING_DIR/rootimg $REPO_LOCATION/boot/i386/root

A. For Kusu dependencies, we need the following packages:

Available on SLES installation disks:
python-2.4.2-18.13.i586.rpm
slang-2.0.5-14.2.i586.rpm

Available from other sources:
http://mondorescue.linjection.org/ft...2-1.2.i586.rpm

To install an rpm in a rootimg, python, for example,
pushd $WORKING_DIR/rootimg
rpm2cpio $RPMDIR/python-2.4.2-18.13.i586.rpm | cpio -idv
popd

Partitioning in Kusu

George Goh — Tue, 10 Jun 2008 16:18:48 GMT

Partitioning is important in Kusu for the following reasons:

On the master node, the disks need to be correctly partitioned, so that repositories can be created on the disk before handing over to the distro's auto-installation mechanism.

On the compute node, partitioning needs to take place unattended, according to a predefined schema in the database, with options to preserve certain types of partitions as well.

Over the next few blog entries, I will describe the approach taken with the partitioning code in Kusu. To kick off the series, I will start with a high-level overview.

Disk States and manipulation
Each machine (whether master or compute) has a 'DiskProfile' object that represents the current/desired state of the system's disks. Generally, there are 3 stages in the disk partitioning process - query, modification, and commission.

QUERY
Usually, the machine will start off by querying the current state of the disks - how many disks? partitions on each disk? LVM partitions? Volume groups? and so on... .

In this stage, tools such as `fdisk`, `lvm vgchange -ay`, and `lvm pvdisplay` are used by the DiskProfile. The benefit of using these tools is the inherent error checking they perform when querying the disks.

At the end of this query stage, Kusu is left with a precise knowledge of the disks in the system, which is presented to the user(in the case of the master), or the nodeinstaller(compute).

MODIFICATION
Next is the modification stage. User(master) or nodeinstaller(compute) manipulates the disk profile to the desired layout. All changes are in memory, but still, all standard rules of partitioning and LVM must be checked in this stage to prevent fatal errors midway through the commission process.

COMMISSION
All changes made in the modification process are committed from memory to disk, the relevant volumes are formatted, and repositories are made on these.

The state of the disks is then passed to anaconda, which installs grub and copies the rest of the operating system to the disk.

* Important here to note: where does grub install?

On some systems '/dev/sda' may not be the first disk seen by the BIOS, and thus may not be booted into. See [1] for more details.

grub must be installed into a place where it can be seen, either by the BIOS, or by a chain loader, which can load grub.

[1] - Dell - Dell Linux - Installer Magic

Kusu Installation

George Goh — Tue, 08 Apr 2008 09:02:30 GMT

Overview

The Kusu installer is the first program that is run when the user boots from a Kusu ISO. It is a wizard-style program responsible for collecting the initial information that will describe and set up a Kusu cluster, including the timezone, networks, root password, disk partitioning, and kits available.

The Kusu installer is made up of 2 parts: the screens (context-specific info and prompts), and the navigation framework. In this document, I will describe the screens.

Note: The installer relies heavily on the python snack module to display its screen elements, so some knowledge of that toolkit is required.

Creating the screens

The standard Kusu installer has a total of 11 screens that are stored in a ScreenFactoryImpl class' attribute called ScreenFactory.screens. Inserting a screen in this list will cause the installer to display it with the other screens in the order defined.

A screen is basically described by a subclass of the InstallerScreen. A simple example is the RootPasswordScreen class:

Most of the elements in this screen are auto-generated by the Kusu Framework, so we concentrate on the specific needs of this screen:

Instruction for the user

Password entry and verification input fields

'Clear All' button for the input fields

Buttons are defined differently from other elements on the screen, so we define the 'Clear All' button on this screen first.

User-defined buttons are defined in the class variable 'buttons', and the callbacks on the buttons are defined in the setCallbacks method:

RootPasswordScreen::setCallbacks
Code:
buttons = [_('Clear All')]
 
def setCallbacks(self):
"""
 
Implementation of the setCallbacks interface defined in parent class
screenfactory.BaseScreen. Initialise button callbacks here.
 
"""
self.buttonsDict[_('Clear All')].setCallback_(self.clearAllFields)
Next are the instruction and entry fields. These are defined and laid out on-screen in the drawImpl method (where self.msg='Please enter a secure root password:'):

RootPasswordScreen::drawImpl
Code:
def drawImpl(self):self.screenGrid = snack.Grid(1, 3)
entryWidth = 20
self.password0 = kusuwidgets.LabelledEntry(labelTxt=_('Enter root password ').rjust(26),
width=entryWidth, password=1
)
 
self.password1 = kusuwidgets.LabelledEntry(labelTxt=_('Verify root password ').rjust(26),
width=entryWidth, password=1
)
 
self.screenGrid.setField(snack.TextboxReflowed(text=self.msg,
width=self.gridWidth),
col=0, row=0, anchorLeft=1
)
self.screenGrid.setField(self.password0, col=0, row=1, anchorLeft=1,
padding=(0,1,0,0)
)
self.screenGrid.setField(self.password1, col=0, row=2, anchorLeft=1,
padding=(0,0,0,1)
)
Inserting the screen into the display order

As mentioned earlier, a screen is inserted into the ScreenFactory.screens attribute of the ScreenFactoryImpl class in the order that it should be displayed.

In the following code segment, we see that the root password screen is placed as the 8th screen in the Kusu installation sequence:

ScreenFactoryImpl
Code:
class ScreenFactoryImpl(ScreenFactory):
"""The ScreenFactory is defined by the programmer, and passed on to theKusuInstaller class.
"""
 
...
 
ScreenFactory.screens = \[WelcomeScreen(kiprofile=kiprofile),
LanguageSelectionScreen(kiprofile=kiprofile),
KeyboardSelectionScreen(kiprofile=kiprofile),
NetworkScreen(kiprofile=kiprofile),
GatewayDNSSetupScreen(kiprofile=kiprofile),
FQHNScreen(kiprofile=kiprofile),
TZSelectionScreen(kiprofile=kiprofile),
RootPasswordScreen(kiprofile=kiprofile),
PartitionScreen(kiprofile=kiprofile),
ConfirmScreen(kiprofile=kiprofile),
KitsScreen(kiprofile=kiprofile)
]
 
...
Hooks

Through examination of the InstallerScreen class, you will find that it is inherited from the BaseScreen class.

The BaseScreen class contains the following hooks:

[table="head;width=70%"]Function name|Documentation
def drawImpl(self) | Children implement this method to draw their required widgets onscreen. Don't draw the buttons as well, this is already done for you. |
def validate(self) | Validation checks(if any) for user inputs should be here. validate() is called by the framework after the 'Next/Finish' button is pressed on the UI.
def rollback(self) | Rollback method actions called to rollback changes made in this screen. rollback() is called by the framework after the 'Prev' button is pressed on the UI.
def formAction(self) | This method is called by the framework when the 'Next' or 'Finish' button is pressed on this screen.
def executeCallback(self, obj) | Callbacks for objects other than buttons. This is an advanced operation for screens that have widgets that need to handle callbacks.
[/table]

The most commonly used methods from this set are: drawImpl, validate, and formAction. drawImpl has already been discussed, so let's move on to validate and formAction.

validate method

The validate method contains screen-specific logic to validate the inputs given by the user. The Kusu installer framework expects a tuple to be returned from this method in the form of (boolean, string). The first member of the tuple denotes the success of the validate method, and the second is the error message that will be displayed to the user on validation failure.

formAction method

The formAction method is called by the Kusu installer framework when the user clicks on the Next/Finish button of that screen. Typically, this is used to store the values entered by the user to a database.

New machines for OSGDC :-)

George Goh — Thu, 13 Mar 2008 10:09:00 GMT

Liming and I took delivery for some shiny new hardware at our data centre today. These are a Dell 1950, and 2950 with dual Quad-core Xeon CPUs and 8GB of RAM each. The 1950 has two 15k, 300G SAS drives, while the 2950 has three 10k, 400G SAS drives. Very nice!

The 2950: Don�t carry this alone!

It was a blur unpacking the machines. We were anxious to get stuff up and running.

Liming got close and intimate with this 2950.

Sexy beast.

Yet another intimate shot.

Major milestone reached

George Goh — Tue, 15 May 2007 09:48:00 GMT

Greetings true believers!

I�m George Goh, one of your friendly neighbourhood cluster monkeys. Najib, our lead ape, has granted me access to blog here about general things happening in the OSGDC, and more specifically, Project Kusu and the components that I maintain, so let�s get things rolling� .

Here at OSGDC, we adopt the SCRUM methodology(http://en.wikipedia.org/wiki/Scrum_%28development%29) to keep things tight and moving fast. We have recently ended our third full sprint, and we have achieved quite a bit:

Automated build system to build and create ISO images.

Lightweight Textual User Interface(TUI) framework based on Python�s snack module.

Disk/partition/LVM detection, CRUD operations.

Network adapter detection/configuration + DHCP.

Migration of settings to native install script(FC6 kickstart).

More features are coming up, so watch this space. But in the meantime, here�s a screencast as a taste of things to come:

http://www.osgdc.org/pub/html/kusu/s...st/2007-05-11/

See ya.swf