User guide at SBC

You need to make sure your ~/.bashrc looks like this

. /etc/profile

and that your ~/.bash_profile looks like this

if [ -f ~/.bashrc ]; then
        . ~/.bashrc
fi

If these two files don't exit, you need to create them.

To be able to log in to the Ferlin cluster you need to make sure your ~/.ssh/config looks like this

# Hosts we want to authenticate to with Kerberos
Host *.kth.se *.kth.se.
# User authentication based on GSSAPI is allowed
GSSAPIAuthentication yes
# Key exchange based on GSSAPI may be used for server authentication
GSSAPIKeyExchange yes

# Hosts to which we want to delegate credentials. Try to limit this to
# hosts you trust, and were you really have use for forwarded tickets.
Host *.csc.kth.se *.csc.kth.se. *.nada.kth.se *.nada.kth.se. *.pdc.kth.se *.pdc.kth.se.
# Forward (delegate) credentials (tickets) to the server.
GSSAPIDelegateCredentials yes

# All other hosts
Host *

If the file doesn't exit, you need to create it. See also Section 34.2, “Log in to the Ferlin cluster”

You need to set your preferred mail client to be thunderbird in the menu System → Preferences → Preferred applications so that thunderbird is being used.

If you in the menu choose System → Preferences → Appearance you will start the configuration program gnome-appearance-properties. In the tab "Visual Effects" you need to choose "None". By doing this you avoid some X server bugs that can hang your desktop from time to time.

As you now are starting to use the SBC computers, you now need to be aware of how to ask support. Read more about it in Section 44.1, “How to request support”. You must not forget to write down all your SBC passwords on a paper and keep the paper in safe place at home ( Section 10.1.3, “You are required to write down all your passwords at SBC on a paper and store it at home” ).

This section is mostly intended for those of you who consume a lot of AFS usage. If you just can free up some 100 Mb of disk space it may not be worth the effort, but if you can save many Gb, please spend some time looking into this.

You need to take a look in your home directory and see if you somehow can use less file system space. A good idea is to focus on the files taking the most space. First identify big volumes ( see Example 5, “The amount of used AFS space per volume” ).

We saw that the volume "prj.sbc.esjolund00" is big.

[esjolund@otto ~]$ fs lsmount vol01
'vol00' is a mount point for volume '#prj.sbc.esjolund00'
[esjolund@otto ~]$ du -sm vol00/*
3876 vol00/research_dir
90   vol00/smalldir

Here we identified vol00/bigdir as the directory that consumes more disk space. But we didn't check directories and files with names starting with a dot.

[esjolund@otto ~]$ du -sm vol00/.[^.]*
76 vol00/.dot-dir
5  vol00/.some-other-dot-dir
10 vol00/.some-dot-file

[esjolund@otto ~]$ cd vol00
[esjolund@otto vol00]$ find -noleaf research_dir
research_dir
research_dir/file1
research_dir/file3
research_dir/file2

Let us compress the directory ~/vol00/research_dir

[esjolund@otto vol00]$ tar czf research_dir.tar.gz research_dir/

Take a quick look into the generated tar.gz file

[esjolund@otto vol00]$ tar tzf research_dir.tar.gz 
research_dir/
research_dir/file1
research_dir/file3
research_dir/file2

After you are sure it worked, don't forget to remove the original files.

Instead of compressing whole directories you might want to compress the files separately ( see Example 7, “Freeing up disk space by compressing files” )

Having hundreds of thousands of files will cause difficulties for the PDC backup system. It will then take a long time for the PDC backup system to traverse the AFS file tree. If possible try to make tar archives dir.tar.gz or dir.tar.bz2 if you have a lot of files that don't need to be directly accessible.

Talk to your supervisor about what file permissions are suitable for your research directories. It is advisable to at least give read permissions to your supervisor, maybe also write permissions. Maybe the whole research group should be given the same permissions. ( See Example 2, “Recursively changing AFS access permissions” )

If you are leaving SBC you must make a personal copy of all the files in your AFS home directory that you might need in the future. Copy the files to your own personal computer. The normal scenario for people leaving is that PDC is soon disabling access to their AFS home directories.

The address for letters:

The address for business parcels ( företagspaket ) and courier parcels ( DHL, UPS ... ):

There is no internal mail service (in Swedish: internpost) at SciLifeLab, so we use external mail (Posten AB) (see Section 6.2.2, “External mail”) for outgoing post.

Envelopes and bar code labels are found near the entrance at floor 2.

Envelopes and bar code labels

Put a bar code label on your envelope because they function as stamps.

example letter

You then need to put the envelope in the right most and lowest mail slot (seen in Figure 2, “Mail slot at floor 2 at SciLifeLab”) that is used for outgoing mail.

Where to put outgoing post

The mail slot for outgoing mail is emptyed 15.00 every working day.

If you are a Ph D Student, Postdoc or employee you will get a mail slot at SciLifeLab.

Figure 2. Mail slot at floor 2 at SciLifeLab

Normally the post for Albanova arrives to the building at 9.30-10.00. Normally between 10.00 and 10.30 the post has been delivered into the post boxes. There is often a small second delivery of postage to the building in the afternoon. Around 15.30 those mails have been delivered to the post boxes. On the wall the time is written for the last delivery.

We see that the last delivery occured at 10.15

The Intel graphics driver supports rotating of the screen. To rotate

[esjolund@otto ~]$ xrandr -o left

To rotate back

[esjolund@otto ~]$ xrandr -o normal

You should choose a password consisting of digits, lower and upper case letters. If you also use other characters, e.g. ( !/%,#=- ) your password will be even more secure.

You use your Section 10.2.4, “KTH kerberos password” to log in to the desktop computers.

You have one password for all mail services. The same password is also used for logging in to mail.sbc.su.se. If you want to change that password log in to mail.sbc.su.se with ssh and type passwd.

esjolund@otto:~$ ssh mail.sbc.su.se
esjolund@mail.sbc.su.se's password: 
Last login: Tue May 20 10:12:56 2008 from otto.sbc.su.se
[esjolund@boron ~]$ passwd
Changing password for user esjolund.
Changing password for esjolund
(current) UNIX password: 
New UNIX password: 
Retype new UNIX password: 

See also Section 44.2.12.4, “Forgetting the Mail password” and Section 10.1.3, “You are required to write down all your passwords at SBC on a paper and store it at home”

You are not automatically added as a user to the computer www.sbc.su.se. If you need to publish things on www.sbc.su.se, please send a support request to sbc-sys@sbc.su.se and say that you need an account. If you want to change your password on www.sbc.su.se, first log in to www.sbc.su.se with ssh and type passwd.

esjolund@otto:~$ ssh -p 50011 www.sbc.su.se
esjolund@www.sbc.su.se's password: 
Last login: Tue May 20 12:26:56 2008 from sly.sbc.su.se
[esjolund@zinc ~]$ passwd
Changing password for user esjolund.
Changing password for esjolund
(current) UNIX password: 
New UNIX password: 
Retype new UNIX password: 

See also Section 44.2.12.5, “Forgetting the Web password” and Section 10.1.3, “You are required to write down all your passwords at SBC on a paper and store it at home”

When you start working at SBC you will be given an envelope containing your KTH Kerberos password. That password is used to get valid Kerberos tickets needed to access your AFS home directory and to log in to the cluster ( see Section 14.2.1, “Getting tickets” ). You use the Kerberos password to log in to the SBC desktop computers. At the same time you will get valid Kerberos tickets. To change the Kerberos password type kpasswd on the command line on your desktop computer. See also Section 44.2.12.1, “Forgetting the KTH kerberos password” and Section 10.1.3, “You are required to write down all your passwords at SBC on a paper and store it at home”

With a kth.se password you can get eduroam wireless access (see Section 29, “Eduroam”) or a VPN connection to KTH. Note, the kth.se password is not the same as your KTH kerberos password. ( see Section 10.2.4, “KTH kerberos password” ).

Information about kth.se accounts (written in Swedish)

See also Section 44.2.12.2, “Forgetting the kth.se password” and Section 10.1.3, “You are required to write down all your passwords at SBC on a paper and store it at home”

When you start working at SBC you are normally given your subversion password. Unfortunately it is not possible to change this password. See also Section 16.3, “Repositories and access rights”, Section 44.2.12.3, “Forgetting the Subversion password” and Section 10.1.3, “You are required to write down all your passwords at SBC on a paper and store it at home”

This password will be given to you by Petra Lauritsch. It is used for the SciLifeLab email address. There is a web interface for the SciLifeLab emails (see Section 11.3.2, “Web interface for your SciLifeLab email address”). There you can change this password. Note, after changing this password, it will no longer match the password for homer.scilifelab.se ( Section 10.2.8, “Password for homer.scilifelab.se” ).

This password is the same as the original password for the SciLifeLab email (see Section 10.2.7, “Password for SciLifeLab email”). The password for homer.scilifelab.se can not be changed. So if you have altered the SciLifeLab email password, the two passwords will no longer match.

The password for homer.scilifelab.se is used for the homer file server (see Section 14.1, “file server homer.scilifelab.se”, room booking and for the VPN access (this might change in the future).

Contact Petra Lauritsch to get a SciLifeLab email account.

The SciLifeLab email is run by Google and has a web interface. In the field "Username:" specify your new email address "firstname.lastname" and in the field "Password:" specify the email passsword you were given by Petra Lauritsch (see Section 10.2.7, “Password for SciLifeLab email”). If you want to use thunderbird or some of the other supported IMAP clients, you should configure your IMAP client to use imap.gmail.com (port 993).

If you want to use thunderbird or some of the other supported IMAP clients, you should configure your email client like this

The same password and username is used for the POP server, IMAP server, webmail ( see Section 11.5, “Webmail” ), and for logging in to mail.sbc.su.se with ssh .

You can use either POP over SSL or IMAP over SSL to receive mail to your mail client. "Over SSL" means that the connection is encrypted. Your mail client may have a check box named "Use secure connection" or something similar. Use that.

If you choose to use IMAP over SSL instead of POP over SSL, you leave the mails on the mail server. This has the advantage that you will see the same mail folders independently of if you use an IMAP mail client from home or one from inside of SBC or if you access your email by webmail ( see Section 11.5, “Webmail” ).

As smtp server specify smtp.su.se in the settings of the mail client. You will only be able to send mail through this smtp server if your computer is located inside of SBC ( or of Stockholm university ).

If you have a SUKAT account ( see Section 25, “Sukat account” ) you can follow the instructions in https://support.it.secure.su.se/kb/entry!default.jspa?categoryID=192&externalID=2963&fromSearchPage=true to configure your laptop to be able to send email through smtp.su.se both from inside and outside the Stockholm University network.

All new users at SBC should automatically get subscribed to the mailing lists sbc@sbc.su.se, sbc-sys@sbc.su.se and seminars@sbc.su.se when they start here. If you notice that you are not a member of these lists, you could sign up at the links below:

After SBC moved to SciLifeLab we also use the mailing lists that are run at SciLifeLab. You are recommended to subscribe to at least the mailing list:

You subscribe by filling out a support request at the SciLifeLab helpdesk web form.

If you want to create a new mailing list, please write a support request email to <sbc-sys@sbc.su.se> and provide these details:

Before you can access your afs directory you need valid tickets. To get tickets that expires in one month type kinit -f -l1month. You will now be asked for your Kerberos password ( see Section 10.2.4, “KTH kerberos password” ). See also Section 42.6.2, “Kerberos principal expiration date” and Section 42.6.3, “Kerberos password expiration date”

To list your valid tickets use the command klist -v.

[esjolund@oxygen esjolund]$ klist -v
Credentials cache: FILE:/tmp/krb5cc_4495
        Principal: esjolund@NADA.KTH.SE
    Cache version: 4

Server: krbtgt/NADA.KTH.SE@NADA.KTH.SE
Ticket etype: des3-cbc-sha1, kvno 4
Auth time:  Jun  9 10:16:33 2005
End time:   Jul  9 10:16:33 2005
Renew till: Jul  9 10:16:33 2005
Ticket flags: forwardable, renewable, initial
Addresses: IPv4:130.237.93.68

Server: afs/pdc.kth.se@NADA.KTH.SE
Ticket etype: des-cbc-crc, kvno 2
Auth time:  Jun  9 10:16:33 2005
End time:   Jul  9 10:16:33 2005
Ticket flags: transited-policy-checked
Addresses: IPv4:130.237.93.68

Server: afs@NADA.KTH.SE
Ticket etype: des-cbc-crc, kvno 12
Auth time:  Jun  9 10:16:33 2005
End time:   Jul  9 10:16:33 2005
Ticket flags: transited-policy-checked
Addresses: IPv4:130.237.93.68

We see that the validity of the tickets end at July 9 2005. Because the -f flag was used in the kinit command, the ticket got marked as forwardable. This is important when you submit jobs to the cluster. The submitting computer needs to have forwardable tickets.

[esjolund@oxygen esjolund]$ klist -v | grep forwardable 
Ticket flags: forwardable, renewable, initial

See also Section 34.3.1, “esubmit”

Your AFS home directory is your home directory at the desktop machine and at the PDC compute cluster machines.

let us look into the directory $HOME

[esjolund@oxygen esjolund]$ ls -l $HOME
total 8
lrwxr-xr-x    1 esjolund nfsnobody        9 Aug  6  2003 OldFiles -> .OldFiles
drwx------    7 esjolund nfsnobody     4096 Feb  9 17:08 Private
drwxr-xr-x    2 esjolund nfsnobody     2048 Aug  7  2003 Public

AFS is using a more fine-grained type of permissions than the normal unix file system permissions. Let us look at them:

[esjolund@oxygen esjolund]$ fs listacl /afs/pdc.kth.se/home/e/esjolund/Private/
Access list for /afs/pdc.kth.se/home/e/esjolund/Private/ is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  esjolund rlidwka
[esjolund@oxygen esjolund]$ fs listacl /afs/pdc.kth.se/home/e/esjolund/Public/
Access list for /afs/pdc.kth.se/home/e/esjolund/Public/ is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  system:anyuser rl
  esjolund rlidwka

This means, files that you want to be public and readable from the whole Internet should be placed in the Public directory. Private files that you want to keep for yourself should be placed in the Private directory.

A newly created directory inherits the access permissions from its parent directory.

[esjolund@otto ~]$ mkdir ~/Public/testdir
[esjolund@otto ~]$ fs la ~/Public/testdir
Access list for /afs/pdc.kth.se/home/e/esjolund/Public/testdir is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  system:anyuser rl
  esjolund rlidwka

To change the permissions for a directory, use the command fs setacl ( abbreviated fs sa ).

[esjolund@otto ~]$ fs sa ~/Public/testdir system:anyuser l
[esjolund@otto ~]$ fs la ~/Public/testdir
Access list for /afs/pdc.kth.se/home/e/esjolund/Public/testdir is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  system:anyuser l
  esjolund rlidwka

Now the anonymous Internet user ( system:anyuser ) can't read the files in ~/Public/testdir anymore. But he can list (l) the file names in the directory.

[esjolund@otto ~]$ fs la ~/Public/testdir
Access list for /afs/pdc.kth.se/home/e/esjolund/Public/testdir is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  esjolund rlidwka

Now he can't even list filenames anymore.

[esjolund@otto ~]$ mkdir research research/dir
[esjolund@otto ~]$ fs la ~/research
Access list for /afs/pdc.kth.se/home/e/esjolund/research is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  system:anyuser l
  esjolund rlidwka
[esjolund@otto ~]$ fs la ~/research/dir
Access list for /afs/pdc.kth.se/home/e/esjolund/research/dir is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  system:anyuser l
  esjolund rlidwka

[esjolund@otto ~]$  find ~/research/ -noleaf -type d -exec fs sa {} maccallr  read \;

[esjolund@otto ~]$ fs la ~/research/
Access list for /afs/pdc.kth.se/home/e/esjolund/research/ is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  system:anyuser l
  maccallr rl
  esjolund rlidwka
[esjolund@otto ~]$ fs la ~/research/dir/
Access list for /afs/pdc.kth.se/home/e/esjolund/research/dir/ is
Normal rights:
  esjolund:remote-users rlidwka
  system:administrators rlidwka
  system:anyuser l
  maccallr rl
  esjolund rlidwka

Read more about AFS file permission ( see Section 14.2.8, “More AFS documentation” )

You can not store more data in a AFS directory than what the quota is set to for the AFS volume it resides in. The command to see the quota is fs listquota.

[esjolund@oxygen user-guide]$ fs listquota /afs/pdc.kth.se/home/e/esjolund/
Volume Name                   Quota      Used %Used   Partition
H.esjolund                   500000     84102   17%         64%

We found out that the quota was set to 500 Mb. That is given to you by default for the AFS volume mounted as your AFS home directory. If you need an additional AFS volumem, please fill in this web form at PDC. In the field "Project information", don't forget to mention that you belong to SBC.

Instead of requesting an additional AFS volume you could also try to make free space available by compressing files ( see Example 7, “Freeing up disk space by compressing files” ).

[esjolund@oxygen esjolund]$ /usr/bin/vos listvldb | egrep "(^|\.)$USER[0-9]*[[:blank:]\.]"
H.esjolund
prj.sbc.esjolund00

[esjolund@oxygen esjolund]$ cd /afs/pdc.kth.se/home/e/esjolund/Private
[esjolund@oxygen Private]$ fs mkmount -dir vol00 -vol prj.sbc.esjolund00 -cell PDC.KTH.SE
[esjolund@oxygen Private]$ fs lq -path vol00/
Volume Name                   Quota      Used %Used   Partition
prj.sbc.esjolund00          5000000         2    0%         74%
[esjolund@oxygen Private]$ ls vol00
[esjolund@oxygen Private]$ fs lsmount -dir vol00
'vol00' is a mount point for volume '#prj.sbc.esjolund00'

#!/bin/sh
for i in `/usr/bin/vos listvldb | egrep "(^|\.)$USER[0-9]*[[:blank:]\.]"`;
  do /usr/bin/vos examine $i | awk '{print $4; exit}';
done | awk 'BEGIN {sum=0} {sum +=$1} END {print sum}'

[esjolund@otto ~]$ emacs /tmp/a.sh
[esjolund@otto ~]$ cat /tmp/a.sh
#!/bin/sh
for i in `/usr/bin/vos listvldb | egrep "(^|\.)$USER[0-9]*[[:blank:]\.]"`;
  do /usr/bin/vos examine $i | awk '{print $4 " " $1; exit}' ;
done | sort -n
[esjolund@otto ~]$ sh /tmp/a.sh 
276138 H.esjolund
2742928 prj.sbc.esjolund01
4617796 prj.sbc.esjolund00

#!/bin/sh
for i in `/usr/bin/vos listvldb | egrep "(^|\.)$USER[0-9]*[[:blank:]\.]"`;
  do /usr/bin/vos examine $i; 
done | awk 'BEGIN {sum=0} $1 == "MaxQuota" {sum +=$2} END {print sum}'

Thanks Daniel for the nice scripts!

[esjolund@otto ~]$ ls -lh file.txt
-rw-r--r-- 1 esjolund users 369K Jun 11 14:16 file.txt
[esjolund@otto ~]$ gzip file.txt 
[esjolund@otto ~]$ ls -lh file.txt.gz 
-rw-r--r-- 1 esjolund users 40K Jun 11 14:16 file.txt.gz

PDC backs up AFS, but as there is no contract with PDC defining the service level of the backup service, no guarantees can be made here about what type and how often backups are made. Most often, though, backups are taken each night. Some AFS volumes are excluded from the list of backed up AFS volumes but the owners of those AFS volumes have been informed about this when those AFS volumes were created. The last backup of each AFS volume is available online as a read-only AFS volume. The read-only backup volume has the same name as the original but with ".backup" appended to it. For your convenience your home directory volume has its backup volume already mounted at ~/.OldFiles. Note that read-only backup volumes of your extra volumes are not available through ~/.OldFiles, you need to mount them explicitly as seen in Example 8, “Restore an AFS file from the nightly backup”. If you need to get hold of backups that are older than one day, manual operation is required from PDC to retrieve those files ( just use that service if you really need it ). Older backups are distributed more sparse in time than younger backups.

[esjolund@oxygen esjolund]$ rm /afs/pdc.kth.se/home/e/esjolund/project/data/my_file.txt

[esjolund@oxygen esjolund]$ cd /afs/pdc.kth.se/home/e/esjolund
[esjolund@oxygen esjolund]$ cp OldFiles/project/data/my_file.txt project/data/my_file.txt

[esjolund@oxygen esjolund]$ ls OldFiles/project/data/my_file.txt 
fs: File '/afs/pdc.kth.se/home/e/esjolund/OldFiles/project/data/my_file.txt ' doesn't exist

[esjolund@oxygen esjolund]$ fs lsmount OldFiles/project/data
'OldFiles/project/data' is not a mount point.

[esjolund@oxygen esjolund]$ fs lsmount OldFiles/project/
'OldFiles/project/data' is a mount point for volume '#prj.sbc.esjolund00'

[esjolund@oxygen esjolund]$ fs mkmount tmp_mount -vol prj.sbc.esjolund00.backup -cell PDC.KTH.SE
[esjolund@oxygen esjolund]$ fs lsmount tmp_mount
'tmp_mount' is a mount point for volume '#prj.sbc.esjolund00.backup'

[esjolund@oxygen esjolund]$ cp tmp_mount/data/my_file.txt project/data/my_file.txt 

[esjolund@oxygen esjolund]$ fs rmmount tmp_mount

The maximum number of files in a directory is 64435. If you try to create one more file than that, you will get the error File too large

dah@s10n42:~/afstest/dah.test.manyfiles/1/0$ touch foo
touch: creating `foo': File too large

openafs has a very slow algorithm for accessing files in a directory with many files. So it's not practical having more than a few thousands of files in a directory. Recommended is instead to group the filenames in different directories. For instances PDC has grouped together all home directories with usernames starting with the same letter. E.g. notice the directory e in the directory path /afs/pdc.kth.se/home/e/esjolund.

When a filename is looked up in AFS the path is searched for directories named "@sys". If such a directory is found, AFS will first try to replace "@sys" with the first string in the list of strings output by the command fs sysname.

esjolund@otto:~$ fs sysname
Current sysname list is 'i386_co5' 'i386_co4' 'i386_fc3' 'i386_rh9'

If the resulting path exists that file will be returned to the user. If the resulting path doesn't exist, AFS will continue trying the other strings in the string list. If no file was found AFS will try to lookup the original file name ( i.e. with the "@sys" ).

The string list from fs sysname is set by root. On our machines the string list is set here:

root@otto:~# grep /root/sbc-setsysname.sh /etc/openafs/afs.conf
AFS_POST_INIT=/root/sbc-setsysname.sh

The string list can be set arbitrarily. We use @sys mainly inside the MODULEPATH environment variable. See also Section 40.1, “Module usage”

To learn more about these file permissions and more about afs in general, read the AFS user guide. PDC also has AFS guided tour.

	Tip
	Read the flash news from PDC. Here temporary AFS problems will be listed.

There is a way to find out if an AFS file server is overloaded. First find out on what file server your directory is located.

esjolund@kantarell:~$ fs whereis ~
File /afs/pdc.kth.se/home/e/esjolund is on host pompano.pdc.kth.se 

We can now get some interesting metrics

esjolund@kantarell:~$ /usr/bin/rxdebug pompano.pdc.kth.se | head -5 | tail -2
0 calls waiting for a thread
122 threads are idle

Those values corresponds to the normal healthy condition of an AFS file server with not so high load. But if you on the other hand would see

esjolund@kantarell:~$ /usr/bin/rxdebug pompano.pdc.kth.se | head -5 | tail -2
500 calls waiting for a thread
2 threads are idle

then the AFS server is on a high load which will make everything go very slow. See also Section 42.6.1, “Very slow AFS”

	Important
	Do not store anything of value in /scratch . It may be erased anytime without notice.

The idea of /scratch is to have a working area to save unimportant temporary files and a place to save big genome databases that are downloadble from the Internet.

/scratch is available both on the desktop computers and at the cluster nodes. The cluster nodes have their /scratch erased after each job has run.

Only group members of the corresponding group have read and write access to the group repository.

esjolund@kantarell:~$ svn mkdir -m "creating the new project dir biosequencing" https://svn.sbc.su.se/repos/sonnh/biosequencing
esjolund@kantarell:~$ svn mkdir -m "" https://svn.sbc.su.se/repos/sonnh/biosequencing/trunk
esjolund@kantarell:~$ svn mkdir -m "" https://svn.sbc.su.se/repos/sonnh/biosequencing/branches
esjolund@kantarell:~$ svn mkdir -m "" https://svn.sbc.su.se/repos/sonnh/biosequencing/tags

esjolund@kantarell:~$ svn co https://svn.sbc.su.se/repos/sonnh/biosequencing

Project specific repositories are not recommended. Please use the group repositories instead. Only under rare circumstances can it be justified to have a project specific repository. If you think you have such situation and need a new project specific repository send an email to <sbc-sys@sbc.su.se> with this information: the name of the repository, the name of the users with read access, the name of the users with write access and some argument why you can't use a group repository.

lpr is the command to send something to the printers. More detailed info is available by typing man lpr. Invoked without any flags lpr uses the default printing queue.

[esjolund@oxygen esjolund]$ lpr file.ps

prints the file file.ps to the default printer. To print to another queue, type lpr -P queuename myfile.ps.

[esjolund@oxygen esjolund]$ lpr -P xe file.ps

prints the file file.ps to the xe printer queue

To print out the file file.ps in landscape orientation

[esjolund@oxygen esjolund]$ lpr -o landscape file.ps

To print out the file file.pdf one-sided

[esjolund@oxygen esjolund]$ lpr -o sides=one-sided file.pdf

To print out the first four pages of the file file.pdf

[esjolund@oxygen esjolund]$ lpr -o page-ranges=1-4 file.pdf 

To print out the file file.pdf where each printed page contains 2 pages of the original pdf. With this option you can save paper, but it will be harder to read the small text.

[esjolund@oxygen esjolund]$ lpr -o number-up=2 file.pdf

	Note
	The man page for lpr is lacking information. The available -o options are found in the CUPS Software Users Manual

The lpr command understands among others these file types

[esjolund@oxygen esjolund]$ lpr file.pdf
[esjolund@oxygen esjolund]$ lpr file.gif
[esjolund@oxygen esjolund]$ lpr file.png
[esjolund@oxygen esjolund]$ lpr file.jpeg
[esjolund@oxygen esjolund]$ lpr file.tiff
[esjolund@oxygen esjolund]$ lpr file.ps
[esjolund@oxygen esjolund]$ lpr file.txt
[esjolund@oxygen esjolund]$ lpr file.pbm
[esjolund@oxygen esjolund]$ lpr file.pnm
[esjolund@oxygen esjolund]$ lpr file.ppm

See also Section 17.3, “Printing queues”

lpstat is the command to get status information about the printing system. For instance, type lpstat -a to list available printing queues and printing instances.

[esjolund@dirac ~]$ lpstat -a
HPColorF3Quad2 accepting requests since 2011-10-12T10:49:03 CEST
HPF1 accepting requests since 2011-10-11T16:09:39 CEST
HPF1Color accepting requests since 2011-10-12T10:33:10 CEST
HPF2ColorQuad3 accepting requests since 2011-10-11T17:32:11 CEST
HPF2Quad2 accepting requests since 2011-10-04T14:47:24 CEST
HPF2Quad3 accepting requests since 2011-10-12T11:15:28 CEST
HPF3ColorQuad3 accepting requests since 2011-10-12T07:44:01 CEST
HPF3Quad2 accepting requests since 2011-10-12T13:18:24 CEST
HPF3Quad3 accepting requests since 2011-10-11T17:48:01 CEST
Ricoh-MP-c4000 accepting requests since 2011-06-22T18:28:15 CEST
RicohF2 accepting requests since 2011-09-28T17:30:06 CEST

See the user guide Printers at SBC

If you need that other people should be able to upload data files to you, you could use WebDAV. First make sure you have a valid password for www.sbc.su.se ( see Section 10.2.3, “Password for www.sbc.su.se” ). Then send a request to <sbc-sys@sbc.su.se> that you want to have WebDAV activated for your account on www.sbc.su.se.

[esjolund@oxygen esjolund]$ ssh -p 50011 www.sbc.su.se
esjolund@www.sbc.su.se's password:
[esjolund@tyrone esjolund]$ mkdir -p ~/public_html/webdav
[esjolund@tyrone esjolund]$ echo testing > ~/public_html/webdav/test.txt

You now need to create a htpasswd file with user account names and passwords. The path is given to you in an email. You create the htpasswd file with the command htpasswd. Your test file test.txt will be accessible at the URL https://www.sbc.su.se/~yourusername/public_html/webdav directory ( replace "yourusername" with your user name ). When you access the URL you will asked for a valid username and password.

You can publish web content ( both static and dynamic ) on the computer www.sbc.su.se. First make sure you have a valid password for www.sbc.su.se ( see Section 10.2.3, “Password for www.sbc.su.se” ). Then create a directory public_html in your home directory on www.sbc.su.se and make sure it is readable by everyone.

[esjolund@oxygen esjolund]$ ssh -p 50011 www.sbc.su.se
esjolund@www.sbc.su.se's password:
[esjolund@tyrone esjolund]$ ls -ld ~
drwxr-xr-x  10 esjolund others 4096 Aug 24 12:59 /home/others/esjolund
[esjolund@tyrone esjolund]$ mkdir ~/public_html
[esjolund@tyrone esjolund]$ ls -ld ~/public_html
drwxr-xr-x  4 esjolund others 4096 Jul 13 11:49 /home/others/esjolund/public_html

place your web content in the public_html directory. It will be accessible at http://www.sbc.su.se/~esjolund/ ( replace esjolund with your user name ).

The directory ~/public_html/download is configured to allow directory listings. If there is no index.html in that directory, the web server will generate a listing in html format of the filenames found in the directory. Here is an example of this: http://www.sbc.su.se/~esjolund/download/

If you need to have CGI scripts you should place them in the directory ~/public_html/cgi-bin. First you need to create the directory

[esjolund@oxygen esjolund]$ ssh -p 50011 www.sbc.su.se
esjolund@www.sbc.su.se's password:
[esjolund@tyrone esjolund]$ mkdir -p ~/public_html/cgi-bin

create a script ~/public_html/cgi-bin/script.cgi with the execution bit set

[esjolund@oxygen esjolund]$ emacs ~/public_html/cgi-bin/script.cgi
[esjolund@oxygen esjolund]$ chmod 755 ~/public_html/cgi-bin/script.cgi

It will be available under the URL http://www.sbc.su.se/~esjolund/cgi-bin/script.cgi

	Caution
	CGI-scripts are potential security risks if they contain security holes, so please be careful when writing them. If you download them from the Internet, please stay up to date with security patches for bugs found in those CGI-scripts.

You have the possibility to password protect your web content on www.sbc.su.se.

[esjolund@tyrone esjolund]$ htpasswd -c ~/htpasswords chaplin
New password:
Re-type new password:
Adding password for user chaplin
[esjolund@tyrone esjolund]$ htpasswd ~/htpasswords bogart
New password:
Re-type new password:
Adding password for user bogart

[esjolund@tyrone esjolund]$ emacs ~/public_html/secret_directory/.htaccess
[esjolund@tyrone esjolund]$ cat ~/public_html/secret_directory/.htaccess

 AuthName "restricted stuff"
 AuthType Basic
 AuthUserFile /home/others/esjolund/htpasswords
 require valid-user
[esjolund@tyrone esjolund]$ chmod 644 /home/others/esjolund/public_html/secret_directory/.htaccess

In the ceiling in room RB35 we have a video projector of model BENQ SP831™. User manual (pdf-format).

Figure 3. Video projector

We have a video projector of model PLUS U3-1100Z™. User's manual (pdf-format). There should also be a laser pointer kept with the video projector. The projector is most easily attached via the video cable, you may have to use the Fn (function) key on your laptop to turn on the external monitor function.

You may borrow the video projector for holding presentations. It is located in room 162:002, but you need to book it first on the calendar outside of room 162:002

This instruction is for Ubuntu Linux 10.10 but it may work for other Ubuntu versions too. To get access to Eduroam you first need to get your KTH network secret Section 28, “KTH network secret”.

First download the CA certificate to your home directory

esjolund@ubuntu:~$ cd
esjolund@ubuntu:~$ wget \-\-quiet http://www.lan.kth.se/eduroam/AddTrust_External_CA_Root.pem
esjolund@ubuntu:~$ ls -l AddTrust_External_CA_Root.pem 
-rw-r\-\-r\-\- 1 esjolund users 1521 2010-07-05 16:00 AddTrust_External_CA_Root.pem

In the right upper corner click left mouse button

you will see there is a network available called "eduroam". Now click left mouse button on that entry.

After a while you will see this window

Fill in the values according to the following table

Click Connect

Follow the instructions at http://www.lan.kth.se/eduroam/

Follow the instructions at http://www.lan.kth.se/eduroam/

At SciLifeLab, you can use EduRoam (see Section 29, “Eduroam” ).

To be able to use the wired ethernet at SciLifeLab you need to provide your laptop's MAC address (see Section 30.5, “Finding your MAC-addresses”) in a support request to the SciLifeLab IT helpdesk.

All modern Linux distributions ( from year 2003 or newer ) use cups by default so you just have to add the line

ServerName cups.sbc.su.se

to the file /etc/cups/client.conf. Another option is to set the environment variable CUPS_SERVER

[esjolund@eriks_laptop esjolund]$ export CUPS_SERVER=cups.sbc.su.se

The setup for MacOS should actually be the same as for Linux.

On Windows XP. To add the printer "bw".Press "Add printer". Choose "Local printer attached to this computer" but leave the box "Automatically detect and install my Plug and Play printer" unchecked. Press "next". Choose "Create new port". Specify "Type of port: Standard TCP/IP Port". Specify ip number "130.237.93.9". Choose driver " HP Laserjet 4250 Series PCL".

For the "xe" printer you can borrow a install CD ROM from Erik Sjölund.

If you are a senior scientist, you could apply for a Large scale application at SNIC.

If you are a senior scientist in Swedish academia, at least at the level of assistant professor (forskarassistent), you could apply for a Medium scale application at SNIC.

If you are at least a PhD you could apply for a "Small local allocation". At least PDC would prefer that master students have their supervisor applying. The supervisor could then share her/his cluster time with the master student.

Small local allocations have these characteristics:

Here follows a list with links to different SNIC centers:

The command esubmit is used to submit jobs to the queue. To be able to use esubmit you first need to follow the instructions in Section 34.2, “Log in to the Ferlin cluster”. You need to have forwardable tickets on the computer where you execute the esubmit. Also make sure the validity time of the ticket spans over the time the job is in the waiting queue and over the time the job is running ( see Section 14.2.2, “Verify validity of tickets” ).

The availabe options are listed by typing esubmit -h.

s07n01$ esubmit -h
Usage: esubmit [-c CAC][-C jid][-T time][-F jid][-[-h][-m][-n nodes][-p cpus][-v][-t minutes] [program [args]].
  -c CAC        : choose CAC, charge account category.
  -h            : help, use -v to get more.
  -m            : do not send mail.
  -n nodes      : number of nodes.
  -p cpus       : choose number of cpus, reserved for future use.
  -t minutes    : for amount of minutes.
  -T TIME       : start some time after TIME.
  -C JID        : do not start before JID started.
  -F JID        : start some time after JID finished.
                  use only one out of T C or F.

  program       : executable.
                  omitting executable reserves nodes only.
  args          : arguments for executable.

If you start a job requiring multiple nodes ( option -n with a value greater than 1 ), the job will be started on one of them. That process will then take look in the file whose filename is given in the environment variable SP_HOSTFILE. There it finds a list of the allocated nodes. Alternatively the process can find the same hostnames as a list in the environment variable SP_NODES.

Jobs requiring multiple nodes are not that common at SBC. They are needed when you use a parallel programing libraries such as, MPI or PVM. Usually the cluster users at SBC executes esubmit many times instead.

s07n01$ esubmit -n 1 -t 1 echo Hi there!

s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
 12 082414020908  esjolund wait     sbc.esjolund       2S 0h01
s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
 12 082414020908  esjolund wait     sbc.esjolund       2S 0h01
s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
  8 082414020908  esjolund wait     sbc.esjolund       2S 0h01
s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
  7 082414020908  esjolund wait     sbc.esjolund       2S 0h01
s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
  6 082414020908  esjolund wait     sbc.esjolund       2S 0h01
s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
  - 082414020908  esjolund run      -                  2U 2004-08-24 16:09:00
s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
  - 082414020908  esjolund run      -                  2U 2004-08-24 16:09:00

From: easy <easy@pdc.kth.se>
To: esjolund@pdc.kth.se
Subject: 082414020908 node allocation
Date: Tue, 24 Aug 2004 16:08:45 +0200 (CEST)
It is 2004-08-24 16:04:04 and your job 082414020908
is running on the resources you requested.
You now have access to the following node:

s07n31.pdc.kth.se

From: Erik Sjölund <esjolund@pdc.kth.se>
To: esjolund@pdc.kth.se
Subject: 082414020908 job output
Date: Tue, 24 Aug 2004 16:08:46 +0200 (CEST)
INFO: /usr/heimdal/bin/klist -Tf for job 082414020908 gives:

Credentials cache: FILE:/tmp/krb5cc_4495
        Principal: esjolund@NADA.KTH.SE

  Issued           Expires        Flags      Principal
Aug 24 16:08:45  Sep 23 15:20:32  Ff     krbtgt/NADA.KTH.SE@NADA.KTH.SE
Aug 24 16:08:45  Sep 23 15:20:32  f      afs/pdc.kth.se@NADA.KTH.SE
Aug 24 16:08:45  Sep 23 15:20:32  f      afs@NADA.KTH.SE

   V4-ticket file: /tmp/tkt4495

Aug 24 16:08:45  Sep 23 15:20:32  User's (AFS ID 4495) tokens for nada.kth.se
Aug 24 16:08:45  Sep 23 15:20:32  User's (AFS ID 4495) tokens for pdc.kth.se

STDOUT and STDERR for job 082414020908 of user esjolund is:
Hi there!

s07n01$ cat script.sh
#!/bin/sh
touch $HOSTNAME

s07n01$ chmod 755 script.sh

s07n01$ ls -l
total 1
-rwxr-xr-x  1 esjolund 995 23 Oct 19 16:37 script.sh

s07n01$ cat submit_script.sh
#!/bin/sh
for i in `seq 1 10`; do
  esubmit -m -n 1 -t 1 ./script.sh
done

s07n01$ ./submit_script.sh

s07n01$ ls -l
total 2
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:34 s05n03.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:34 s05n04.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:34 s05n09.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:34 s05n10.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:43 s05n12.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:43 s05n13.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:34 s05n14.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:34 s05n15.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:34 s05n19.pdc.kth.se
-rw-r--r--  1 esjolund gopher  0 Oct 20 09:34 s05n21.pdc.kth.se
-rwxr-xr-x  1 esjolund gopher 27 Oct 19 16:57 script.sh
-rwxr-xr-x  1 esjolund gopher 71 Oct 20 09:22 submit_script.sh

The command spq displays the scheduler job queue. The available options are listed by typing spq -h.

s07n01$ spq -h
Usage: spq [-a][-c cac][-h][-l][-L][-q][-r][-u user][-v][-w]
 -a      : show all users.
 -c cac  : show jobs of cac.
 -h      : show this help.
 -l      : show current job limits.
 -L      : show limit definitions.
 -q      : omit headers.
 -r      : show running jobs.
 -u user : show jobs of user.
 -v      : verbose, combine with help.
 -w      : wide output - show more.

If we have submitted jobs we can see their placement in the queue like this.

s07n01$ spq -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
  3 082413232738  esjolund wait     sbc.esjolund       2S 0h01

The digit in the Q column stands for the placement in the queue. To see how a job proceeds in the queue, see Example 13, “Submitting a job”. The Q-value might also be "inf", which means the job is waiting to get a queue number. This happens if you have too many jobs in the queue already.

The command sprelease removes jobs from the queue or from the machine where it is running. The available options are listed by typing sprelease -h.

s07n01$ sprelease -h
Usage: sprelease [-h][-v] [JobID]
       sprelease -S [-h][-v][-m] JobID

    -v verbose mode.
    -h prints this message.
    -m sends mail to user.
    -S use staff enforcement strategy.

       Without Job ID it releases only the node it's run from.
       With a Job ID it releases all nodes running that job,
       or removes it from the queue if it hasn't started yet.

	Note
	It may take some minutes before the job vanishes after an sprelease

s07n01$ spq -q -u $USER
 13 082507595429  esjolund wait     sbc.esjolund       1S 0h04
 14 082507595951  esjolund wait     sbc.esjolund       1S 0h04

s07n01$ spq -q -u $USER  | awk '{print $2}' | xargs -i sprelease {}
sprelease: dequeue request issued for job 082507595429 user esjolund.
sprelease: dequeue request issued for job 082507595951 user esjolund.

s07n01$ spq  -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
 13 082507595429  esjolund wait     sbc.esjolund       1S 0h04
 14 082507595951  esjolund wait     sbc.esjolund       1S 0h04
s07n01$ spq  -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
 10 082507595429  esjolund wait     sbc.esjolund       1S 0h04
 11 082507595951  esjolund wait     sbc.esjolund       1S 0h04
s07n01$ spq  -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME
  9 082507595429  esjolund wait     sbc.esjolund       1S 0h04
 10 082507595951  esjolund wait     sbc.esjolund       1S 0h04
s07n01$ spq  -u $USER
  Q          JID  USER     STATE    CAC          RESOURCE TIME

The command spusage shows who is using the nodes. The available options are listed by typing spusage -h.

s07n01$ spusage -h
usage: spusage [-h][-r]
spusage displays the current resource file.

To list your currently used nodes type spusage | grep $USER

s07n01$ spusage | grep $USER
s05n36.pdc.kth.se     U   1 esjolund 082507452729 2004-08-25 09:56:00
s05n37.pdc.kth.se     U   1 esjolund 082507453972 2004-08-25 09:56:00

spwhen gives an estimate of when a job might start. spwhen itself does a lot of computations and may take some minutes to run. The available options are listed by typing spwhen -h.

s07n01$ spwhen -h
usage: /pdc/vol/easy/1.6/bin/spwhen [-w][-h]
-h     get help.
-w     wide output.

The most common reason for AFS being slow is that many cluster nodes write to the same file at the same time. Sometimes programs write a log in a dot file in the users home directory. Please watch out for this.

	Warning
	Please watch out for the command formatdb from the bioinformatics package blast. If you use that program, make sure it doesn't write the log file to AFS. Use the command line option -l to specify another path, e.g. /scratch/$USER/formatdb.log.

It is much faster to store temporary files on the local hard drive on the cluster node than on AFS. Excessive usage of AFS tends to slow down AFS for everyone. If your job requires heavy file access, make sure your compute job first copies all your data to the local /scratch directory. Also let the compute job write its result files into the local /scratch directory. As a final step the compute job can copy the result files into AFS.

If every node needs to read the same big indata file, you might want to place a copy of that big file in /scratch on some of the untrustworthy nodes ( see Section 34.5, “Freely available untrustworthy nodes ( also called interactive nodes )” ). The reason would be to avoid using AFS. You could then copy that file to the compute nodes with scp. Don't forget to remove the file from the untrustworthy nodes afterwards.

Thanks Hossein, for sharing this tip. Note, as this tip includes using the untrustworthy nodes, you should not file bug reports about failures to PDC.

If you have the choice between submitting a single long job taking a week or doing the same by splitting it up in multiple smaller jobs taking about 4 hours, it's preferred that you choose the smaller jobs alternative. It makes the administration of the cluster easier and it makes your jobs less vulnerable to hardware failures. The cluster is optimized for 4 hours jobs ( quite some number of nodes are reserved for such jobs ).

Before a new job can be started on a cluster node, the queuing system needs to clean and prepare the node. Due to this administration overhead ( up to 5 minutes ), using a lot of short jobs will be somewhat uneconomic. The cluster nodes will then be waisting relatively seen more time in preparation mode. Jobs running some hours seem to be a good compromise.

If you submit 10.000 jobs in advance, they will, although just being in a pending state, slow down the queuing system for everyone. Try to submit more frequently instead. For instance once a day, if possible. Normally the cluster should be able to handle a submission of 1000-2000 4-hour jobs, but this all depends on how many jobs are already in the queue. To see how long the cluster queue is, run these commands

s07n01$ module add easy
s07n01$ spq | wc -l
209

Under normal conditions this number lies below 10000. If you see higher numbers, the queuing system is probably already running slow and you should try to avoid submitting large number of new jobs.

It might be hard to know in advance how long time your job will take. If the time specified in the -t option to esubmit is too small the queuing system will kill your job before it has finished. But your running process has the possibility to save an intermediate result to AFS right before the process gets killed. About 30 seconds before the queuing system definitely kills your job it sends the signal SIGTERM to your job. You can write your program to catch this signal and do neccesary actions to save an intermediate result. You only got 30 seconds time to do the savings, though.

#!/bin/sh

file=/afs/pdc.kth.se/home/e/esjolund/Private/output.txt
echo "The script starts here" > $file

trap "echo received signal SIGTERM - I should save my stuff now >> $file" SIGTERM 

while /bin/true
do
       sleep 4
       date >> $file
done

s07n01$ esubmit -n 1 -t 1 /afs/pdc.kth.se/home/e/esjolund/Public/checkpointtest.sh

The script starts here
Wed Aug 25 13:55:32 CEST 2004
Wed Aug 25 13:55:36 CEST 2004
Wed Aug 25 13:55:40 CEST 2004
Wed Aug 25 13:55:44 CEST 2004
Wed Aug 25 13:55:48 CEST 2004
Wed Aug 25 13:55:52 CEST 2004
Wed Aug 25 13:55:56 CEST 2004
Wed Aug 25 13:56:00 CEST 2004
Wed Aug 25 13:56:04 CEST 2004
Wed Aug 25 13:56:08 CEST 2004
Wed Aug 25 13:56:12 CEST 2004
Wed Aug 25 13:56:16 CEST 2004
Wed Aug 25 13:56:20 CEST 2004
Wed Aug 25 13:56:24 CEST 2004
Wed Aug 25 13:56:28 CEST 2004
Wed Aug 25 13:56:32 CEST 2004
Wed Aug 25 13:56:36 CEST 2004
Wed Aug 25 13:56:40 CEST 2004
Wed Aug 25 13:56:44 CEST 2004
Wed Aug 25 13:56:48 CEST 2004
Wed Aug 25 13:56:52 CEST 2004
Wed Aug 25 13:56:56 CEST 2004
Wed Aug 25 13:57:00 CEST 2004
Wed Aug 25 13:57:04 CEST 2004
Wed Aug 25 13:57:08 CEST 2004
Wed Aug 25 13:57:12 CEST 2004
Wed Aug 25 13:57:16 CEST 2004
Wed Aug 25 13:57:20 CEST 2004
Wed Aug 25 13:57:24 CEST 2004
Wed Aug 25 13:57:28 CEST 2004
Wed Aug 25 13:57:32 CEST 2004
Wed Aug 25 13:57:36 CEST 2004
received signal SIGTERM - I should save my stuff now
Wed Aug 25 13:57:40 CEST 2004
Wed Aug 25 13:57:44 CEST 2004
Wed Aug 25 13:57:48 CEST 2004
Wed Aug 25 13:57:52 CEST 2004
Wed Aug 25 13:58:00 CEST 2004
Wed Aug 25 13:58:04 CEST 2004

If you want to use mysql for a dynamic web page at www.sbc.su.se you could try the solution described here. But you should be aware that the CGI script won't be able to submit to the cluster.

First create the mysql configuration file ( but replace /home/esjolund with your own home directory path )

[esjolund@otto ~]$ ssh -p 50011 www
esjolund@www's password: 
[esjolund@tumba ~]$ emacs ~/.my.cnf
[esjolund@tumba ~]$ cat ~/.my.cnf
[mysqld]
datadir=/home/esjolund/mysql/datadir
socket=/home/esjolund/mysql/mysql.sock
user=esjolund
old_passwords=1
tmpdir=/home/esjolund/mysql/tmp
log=/home/esjolund/mysql/mysqld.log
skip-networking

[mysql.server]
user=esjolund
basedir=/var/lib
skip-networking

[mysqld_safe]
err-log=/home/esjolund/mysql/mysqld.log
pid-file=/home/esjolund/mysql/mysqld.pid
skip-networking

[mysql]
socket=/home/esjolund/mysql/mysql.sock

[client]
socket=/home/esjolund/mysql/mysql.sock

create some directories

[esjolund@tumba ~]$ mkdir -p  ~/mysql ~/mysql/datadir ~/mysql/tmp

initialize the database

[esjolund@tumba ~]$ mysql_install_db --defaults-file=~/.my.cnf
Installing all prepared tables
Fill help tables

To start mysqld at boot time you have to copy support-files/mysql.server
to the right place for your system

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h tumba password 'new-password'
See the manual for more instructions.

You can start the MySQL daemon with:
cd /usr ; /usr/bin/mysqld_safe &

You can test the MySQL daemon with the benchmarks in the 'sql-bench' directory:
cd sql-bench ; perl run-all-tests

Please report any problems with the /usr/bin/mysqlbug script!

The latest information about MySQL is available on the web at
http://www.mysql.com
Support MySQL by buying support/licenses at http://shop.mysql.com

[esjolund@www ~]$ mysqld_safe --defaults-file=~/.my.cnf &

Now log into the database as the root user. ( It has nothing to do with the system root user, it is the mysql root user )

[esjolund@www ~]$ mysql -u root -S ~/mysql/mysql.sock 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.1.20

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> 

Right now the root password is empty so you need to set it to something else ( see Securing the Initial MySQL Accounts ).

Probably you want to do the same with the anonymous account. After that you could create a new user account for your username and then create database tables ( see GRANT Syntax ). ( If you have some nice examples here please tell me / Erik Sj )

Sometimes the web server will be rebooted and after that your database server won't be running anymore. One possible solution is to have a crontab script running every 5 minutes that will restart your database. Sorry, about the lack of details here.

[esjolund@www ~]$ man crontab
[esjolund@www ~]$ man 5 crontab

( exit reading the man page by pressing the key "q" )

One problem with using mysql on your local desktop computer is where to place the datadir directory. Preferably it should be placed on a local hard drive but on the desktop computers all such directories could be erased anytime ( see Section 14.5, “Local hard drive” ). Placing the datadir on AFS, might or might not work. ( If you can fill in with more information here please tell sbc-sys@sbc.su.se ). But the socket file need for sure be placed on the local hard drive, it can't be placed on AFS. Just use mysql on your local desktop computer if you are aware of these caveats. You could then follow the same method as described in Section 35.1.1, “mysql at www.sbc.su.se”.

On a standard redhat installation an error beep is heard when you for instance press backspace on an empty command line. We have chosen to turn that beep off by default. You can still get the beep back if you create the file ~/.inputrc with the contents

$include /etc/inputrc
set bell-style audible

The emacs hardware beep signal is now turned on per default. To turn off beeps in emacs, put this in your ~/.emacs file.

(setq ring-bell-function '(lambda ()))

To turn off the hardware beep in xemacs add the line

(setq bell-volume 0)

to your file ~/.emacs

If you don't want any application to be allowed to produce the hardware beep signal, put this in your ~/.bashrc file.

if [ -n "$DISPLAY" ]; then
  xset b off
fi

Most open source projects written in C or C++ are released as sources together with Makefiles and configure scripts generated by the Auto tools (Autoconf, Automake, Libtool). If you see a script called "configure" in the root directory of the source package, chances are good that it was generated with Autotools. The script configure let you specify where to install the software with the --prefix option. You need to choose a directory where you have write permissions. A suggestion is to install the software in your public area of your afs directory, if you want others to be able to use the software. Installing it on afs also gives you the benefit that the software (hopefully) will be runnable on both our internal desktop machines and on the cluster nodes.

dir=/afs/pdc.kth.se/home/b/bens/Public/treeview-0.4 

mkdir $dir

mkdir $dir/install

cd $dir
tar xvfz /tmp/tv-0.4.tar.gz
tar xvfj /tmp/wxGTK-2.4.2.tar.bz2

cd $dir/wxGTK-2.4.2

less README
less INSTALL

./configure --help

./configure --prefix=$dir/install --with-gtk --enable-gtk2 && make && make install

export PATH=$dir/install/bin:$PATH
export LD_LIBRARY_PATH=$dir/install/lib:$LD_LIBRARY_PATH

cd $dir/tv-0.4
./configure --prefix=$dir/install && make && make install

[esjolund@oxygen esjolund]$ cd /afs/pdc.kth.se/home/a/arnee/modules/i686
[esjolund@oxygen i686]$ ls
acrobat   darwin   ghmm     modeller   netlab   rasmol  subversion  tmhmm    xmgr
beta      diction  grace    molscript  palign   sap     targetp     toppred
blast     dssp     hmmer    mysql      phylip   spdbv   tcoffee     whatif
clustalw  fasta    lgscore  naccess    psipred  stride  tinker      xgvis
[esjolund@oxygen i686]$ mkdir treeview
[esjolund@oxygen i686]$ cd treeview
[esjolund@oxygen treeview]$ echo '
#%Module2.5b
#
#
set installdir /afs/pdc.kth.se/home/b/bens/Public/treeview-0.4 

prepend-path PATH               $installdir/bin
prepend-path LD_LIBRARY_PATH    $installdir/lib
' > 0.4

See also Example 21, “Install missing include file with apt-get”

	Note
	The word module is used in two different contexts in this section. Module in the sense of Perl module and also in the sense of module from modules.

CPAN a large archive of software written in Perl. To install perl software from CPAN, first choose a directory where you have a lot of space

[esjolund@otto ~]$ fs lq ~/space/Private/
Volume Name                   Quota      Used %Used   Partition
prj.sbc.esjolund00          5000000   1247   1%       34%    

create directories for CPAN configuration and the installation of CPAN perl modules

[esjolund@otto ~]$ bigdir=~/space/Private
[esjolund@otto ~]$ mkdir $bigdir/cpan  $bigdir/perl-from-cpan

create symlinks

[esjolund@otto ~]$ ln -s $bigdir/cpan ~/.cpan
[esjolund@otto ~]$ ln -s $bigdir/perl-from-cpan ~/perl-from-cpan

create some directories

[esjolund@otto ~]$ mkdir ~/.cpan/CPAN  ~/.cpan/sources

download configuration files for CPAN

[esjolund@otto ~]$ wget --no-check-certificate  -O  ~/.cpan/sources/MIRRORED.BY https://www.sbc.su.se/intra/user-guide/files/MIRRORED.BY 
[esjolund@otto ~]$ wget --no-check-certificate  -O  ~/.cpan/CPAN/MyConfig.pm https://www.sbc.su.se/intra/user-guide/files/MyConfig.pm

set up CPAN

[esjolund@otto ~]$ perl -MCPAN -e shell;
Terminal does not support AddHistory.

cpan shell -- CPAN exploration and modules installation (v1.7601)
ReadLine support available (try 'install Bundle::CPAN')

cpan> 

type install Bundle::CPAN

cpan> install Bundle::CPAN
CPAN: Storable loaded ok
LWP not available
CPAN: Net::FTP loaded ok
Fetching with Net::FTP:
  ftp://ftp.funet.fi/pub/languages/perl/CPAN/authors/01mailrc.txt.gz

Try to answer the questions. Then run reload cpan

cpan> reload cpan

If you want to install a perl module, just type install modulename. For instance

cpan> install Compress::Zlib

quit the CPAN setup

cpan> quit
Lockfile removed.

Before running your Perl program you need to set the PERL5LIB environment variable

[esjolund@otto ~]$ export PERL5LIB=~/perl-from-cpan/lib/perl5/

Your perl program now has access to the perl modules installed in ~/perl-from-cpan/lib/perl5/. You might need to experiment with the colon separated list of directories that are set in the environment variable PERL5LIB. Maybe you need to have:

[esjolund@otto ~]$ export PERL5LIB=~/perl-from-cpan/lib/perl5/:~/perl-from-cpan/lib/perl5/x86_64-linux-gnu-thread-multi

Probably you want to put the export line in your ~/.bashrc. Another possibility is to create a module for your installed perl software. To do this create a subdirectory under one of the directories listed in the environment variable MODULEPATH. If you want to have your own module directory you need to append a directory name to that variable.

[esjolund@oxygen esjolund]$ export MODULEPATH=$MODULEPATH:~/my_modules
[esjolund@oxygen esjolund]$ mkdir -p ~/my_modules/my_cpanperl

create a file

esjolund@oxygen:~$ echo '
#%Module2.5b
#
#
set dir $HOME/perl-from-cpan

prepend-path PATH               $dir/bin
prepend-path PERL5LIB    $dir/lib/perl5
' > ~/my_modules/my_cpanperl/5.8.5

the module is now available

esjolund@oxygen:~$ module avail 2>&1 | grep my_cpanperl
my_cpanperl/5.8.5

to add it, type

esjolund@oxygen:~$ module add my_cpanperl

You are probably better off by using the installation method described in Section 41.2.2, “Installing perl software from cpan”. But if the perl software is not in CPAN you need to install in a fashion described here.

A lot of software written in perl are released as sources together with a Makefile.pl file. The file Makefile.pl let you specify where to install the software with the PREFIX option. You need to choose a directory where you have write permissions. A suggestion is to install the software in your public area of your afs directory, if you want others to be able to use the software. Installing it on afs also gives you the benefit that the software (hopefully) will be runnable on both our internal desktop machines and on the cluster nodes.

[esjolund@oxygen tmp]$ cd bioperl-1.4/
[esjolund@oxygen bioperl-1.4]$ ls
AUTHORS           biodesign.pod  bioscripts.pod  Changes     examples  INSTALL.WIN  Makefile.old  MANIFEST.SKIP  README
Bio               bioperl.lisp   bptutorial.pl   DEPRECATED  FAQ       LICENSE      Makefile.PL   models         scripts
biodatabases.pod  bioperl.pod    BUGS            doc         INSTALL   maintenance  MANIFEST      PLATFORMS      t
[esjolund@oxygen bioperl-1.4]$ mkdir /afs/pdc.kth.se/home/e/esjolund/Public/bioperl-1.4 
[esjolund@oxygen bioperl-1.4]$ perl Makefile.PL PREFIX=/afs/pdc.kth.se/home/e/esjolund/Public/bioperl-1.4
[esjolund@oxygen bioperl-1.4]$ make
[esjolund@oxygen bioperl-1.4]$ make install

If you have some computer problem, first try to see if you find the answer in this User guide. If you can't find it here you should try to see if the solution to the problem could be easily be found on Google. For instance, cut and paste the error message as a quoted search string to google.

If you don't find any relevant answer under the "Web" tab, click the "Groups" tab to see news group postings.

You might find the solution to your problem by searching these sites. For instance the site http://superuser.com is for general computer user questions.

If you see that your print job can be seen in the print queue lpq -P printerqueue, but it is not printed on the printer. Please check that the printer queue has been disabled.

esjolund@pluto:~$ lpstat -a -p | grep disable
printer hpcolorlaserjetcp3525 disabled since 2011-04-13T14:55:01 CEST -
printer HPF2ColorQuad3 disabled since 2011-06-15T12:01:14 CEST -
printer HPF2Quad3 disabled since 2011-06-10T13:52:18 CEST -

Now we need to run this command on the cups server as root

root@cupsserver:~# cupsenable HPF2Quad3  HPF2ColorQuad3  hpcolorlaserjetcp3525

You can get rid of this message "Password for 'login' GNOME keyring:"

lukask@kall:~$ svn list https://svn.sbc.su.se/repos/cbr/rtpredictor
Password for 'login' GNOME keyring:

by removing the file ~/.gnome2/keyrings/login.keyring

lukask@kall:~$ rm ~/.gnome2/keyrings/login.keyring

This seems to be a known bug.

You try to log in to log in to a text terminal ( Section 42.10, “how to log in to text terminal” ) but it doesn't work. Here is a list of requirements that need to be fulfilled to be able to log in to a text terminal at any of the SBC/CBR Linux desktop computers:

If you can't log in to a Linux desktop computer, first make sure you can log in to a text terminal ( see Section 42.4.1, “Can't log in to text terminal” ). In addition to the requirements for logging in to a text terminal, for ssh this requirement must also be fulfilled:

An expired password could also be the cause of the problem. Let someone else type kinit your_username and then type a random password. If you see this

[esjolund@otto ~]$ kinit andale 
andale@NADA.KTH.SE's Password: 
kinit: krb5_get_init_creds: Password has expired

you could try to run the same command again but this time type in your correct password. You will then hopefully be asked to change your password to something new. See also Section 42.6.3, “Kerberos password expiration date”.

If you can't log in to a Linux desktop computer, first make sure you can log in to a text terminal ( see Section 42.4.1, “Can't log in to text terminal” ). Now check that you haven't overused your quota.

[esjolund@otto ~]$ fs lq ~
Volume Name                   Quota      Used %Used   Partition
H.esjolund                   500000    176875   35%         28%

Please avoid being too close to 100%. It would be nice if you had at least 100-200 Mb free space available. To make space available you could move big directories to extra afs volumes and then create symlinks with ln -s if you want the old filename to still be functionable.

If it still doesn't work. Try to log in graphical mode with the session type "fail-safe". In the xterm window shown on the screen type

[esjolund@otto ~]$ strace -o gnome-session-log.txt -f gnome-session

You might see some interesting error messages written on the terminal. Also of interest is the file gnome-session-log.txt where all system calls have been logged. For instance all attempts to open files have been logged there.

If you get this error message:

esjolund@otto:~$ ssh ferlin.pdc.kth.se
Unable to negotiate a key exchange method

you need to follow the instructions in Section 2.3.2, “Adjust your ~/.ssh/config”

Sometimes when users log in a popup window is shown with the text "Could not update ICEauthority file". It seems to be related to the fact that we are using AFS for our home directories. You can ignore that error message. If you know how we can get rid of that error message please contact erik.sjolund@sbc.su.se

Sometimes when users have logged in their desktop configuration disappeared. A sign for this is also that the mail client evolution asks you to create a new mail account. To solve the problem try the solutions proposed in Section 42.4.3, “Can't log in to my Gnome desktop”.

Programs store their configuration in your home directory. Usually the configuration files and directories have names that start with a dot.

[esjolund@oxygen ~]$ ls -lda ~/.[^.]*
drwx------   2 esjolund others   4096 Dec 29  2003 /home/others/esjolund/.AbiSuite
-rw-------   1 esjolund others  10019 Feb  7 16:27 /home/others/esjolund/.bash_history
-rw-r--r--   1 esjolund others     24 Aug  4  2003 /home/others/esjolund/.bash_logout
-rw-r--r--   1 esjolund others    173 Dec  8 14:33 /home/others/esjolund/.bash_profile
-rw-r--r--   1 esjolund others    665 Dec  3 11:10 /home/others/esjolund/.bashrc
drwxr-xr-x   2 esjolund others   4096 Jan 10 16:10 /home/others/esjolund/.bluefish
drwxr-xr-x   3 esjolund others   4096 Jun 29  2004 /home/others/esjolund/.config
-rw-r--r--   1 esjolund others      0 Jan 27  2004 /home/others/esjolund/.desktop_printing_filetypes
-rw-r--r--   1 esjolund others     37 Nov 11  2003 /home/others/esjolund/.dmrc
drwxr-x---   2 esjolund others   4096 Feb  2 13:48 /home/others/esjolund/.eggcups
drwx------   2 esjolund others   4096 Feb  3 17:11 /home/others/esjolund/.elinks
-rw-r--r--   1 esjolund others    276 Sep  3  2003 /home/others/esjolund/.emacs
drwxr-xr-x   3 esjolund others   4096 Aug  6  2003 /home/others/esjolund/.emacs.d
-rw-rw-r--   1 esjolund web         5 Mar 22  2004 /home/others/esjolund/.emacs-places
-rw-------   1 esjolund others     16 Aug  4  2003 /home/others/esjolund/.esd_auth
drwxr-xr-x   7 esjolund others   4096 Feb  7 17:56 /home/others/esjolund/.evolution
-rw-r--r--   1 esjolund others  60973 Feb  8 11:54 /home/others/esjolund/.fonts.cache-1
drwxr-xr-x   6 esjolund others   4096 Nov 17 13:52 /home/others/esjolund/.fullcircle
drwxr-xr-x   2 esjolund others   4096 Nov 28  2003 /home/others/esjolund/.fvwm
drwx------   6 esjolund others   4096 Oct  8  2003 /home/others/esjolund/.galeon
drwx------   5 esjolund others   4096 Feb  3 15:49 /home/others/esjolund/.gconf
drwx------   2 esjolund others   4096 Feb  8 09:01 /home/others/esjolund/.gconfd
drwx------   3 esjolund others   4096 Aug  8  2003 /home/others/esjolund/.gconfd.old
drwx------   5 esjolund others   4096 Aug  8  2003 /home/others/esjolund/.gconf.old
drwxr-xr-x  17 esjolund others   4096 Jan 17 09:52 /home/others/esjolund/.gimp-1.2
drwx------   6 esjolund others   4096 Jan 24 15:09 /home/others/esjolund/.gnome
drwxr-xr-x   9 esjolund others   4096 Feb  3 15:49 /home/others/esjolund/.gnome2
drwx------   2 esjolund others   4096 Feb  2 14:15 /home/others/esjolund/.gnome2_private
drwxr-xr-x   2 esjolund others   4096 Jun 29  2004 /home/others/esjolund/.gnome-desktop
drwx------   2 esjolund others   4096 Aug  4  2003 /home/others/esjolund/.gnome_private
drwxr-xr-x   2 esjolund others   4096 Feb  2 13:49 /home/others/esjolund/.gstreamer-0.8
-rw-r--r--   1 esjolund others    146 Feb  2 13:48 /home/others/esjolund/.gtkrc-1.2-gnome2
-rw-r--r--   1 esjolund others   1118 Jan  7 11:01 /home/others/esjolund/.gtkrc-kde
-rw-------   1 esjolund others    376 Feb  3 15:48 /home/others/esjolund/.ICEauthority
drwxr-xr-x   2 esjolund others   4096 Aug  6  2003 /home/others/esjolund/.icons
drwxr-xr-x   3 esjolund others   4096 Oct 28  2003 /home/others/esjolund/.java
drwxr-xr-x   4 esjolund others   4096 Feb  2 14:06 /home/others/esjolund/.kde
-rw-------   1 esjolund others    265 Sep  6 14:08 /home/others/esjolund/.kderc
drwxr-xr-x  15 esjolund others   4096 Feb  2 16:47 /home/others/esjolund/.lyx
-rw-r--r--   1 esjolund others  26456 Feb  7 16:49 /home/others/esjolund/.mailcap
drwxr-xr-x   2 esjolund others   4096 Jun  9  2004 /home/others/esjolund/.mc
drwxr-xr-x   3 esjolund others   4096 Aug  6  2003 /home/others/esjolund/.mcop
-rw-------   1 esjolund others     31 Feb  3 15:48 /home/others/esjolund/.mcoprc
drwx------   3 esjolund others   4096 Aug  4  2003 /home/others/esjolund/.metacity
-rw-r--r--   1 esjolund others   2540 Feb  7 16:49 /home/others/esjolund/.mime.types
-rw-r--r--   1 esjolund others   2420 Feb  2 16:25 /home/others/esjolund/.modulesbeginenv
drwx------   5 esjolund others   4096 Sep 17 08:44 /home/others/esjolund/.mozilla
drwxr-xr-x   3 esjolund others   4096 Aug  4  2003 /home/others/esjolund/.nautilus
drwxr-xr-x   2 esjolund others   4096 Jan  7 12:38 /home/others/esjolund/.ncftp
-rw-r--r--   1 esjolund others    238 May 11  2004 /home/others/esjolund/.padminrc
drwxr-xr-x   2 esjolund others   4096 Feb  3 15:48 /home/others/esjolund/.qt
-rw-------   1 esjolund others   2725 Nov  2 16:29 /home/others/esjolund/.recently-used
drwx------   2 esjolund others   4096 Feb  2 13:48 /home/others/esjolund/.rhn-applet
drwxr-xr-x   5 esjolund others   4096 Feb  7 16:50 /home/others/esjolund/.rhopenoffice1.1
drwx------   2 esjolund others   4096 Feb  2 16:04 /home/others/esjolund/.spamassassin
drwx------   2 esjolund others   4096 Feb  7 15:18 /home/others/esjolund/.ssh
drwxr-xr-x   3 esjolund others   4096 Apr 20  2004 /home/others/esjolund/.subversion
-rwxr-xr-x   1 esjolund others     79 Feb  7 16:49 /home/others/esjolund/.sversionrc
drwxr-xr-x   2 esjolund others   4096 Aug  6  2003 /home/others/esjolund/.themes
drwx------   3 esjolund others   4096 Jun 29  2004 /home/others/esjolund/.thumbnails
drwx------   2 esjolund others   4096 Feb  3 17:09 /home/others/esjolund/.w3m
-rw-r--r--   1 esjolund others      7 Nov 21  2003 /home/others/esjolund/.wm_style
-rwxr-xr-x   1 esjolund others    189 Feb  2 13:58 /home/others/esjolund/.Xclients
-rwxr-xr-x   1 esjolund others    189 Feb  2 13:58 /home/others/esjolund/.Xclients-default
drwxr-xr-x   2 esjolund others   4096 Mar 22  2004 /home/others/esjolund/.xemacs
-rw-r--r--   1 esjolund others   9024 Apr  2  2004 /home/others/esjolund/.xscreensaver
-rw-------   1 esjolund others 684603 Feb  2 10:43 /home/others/esjolund/.xsession-errors

[esjolund@oxygen ~]$ pgrep -l -u $USER
1901 ssh-agent
12475 ssh-agent
12839 gconfd-2
12844 bonobo-activati
12848 gnome-settings-
12889 eggcups
16990 bash

[esjolund@oxygen ~]$ pkill -u $USER
[esjolund@oxygen ~]$ pgrep -l -u $USER
12929 bonobo-activati
12990 bash

[esjolund@oxygen ~]$ pkill -9 -u $USER
[esjolund@oxygen ~]$ pgrep -l -u $USER
12990 bash

[esjolund@oxygen ~]$ cd
[esjolund@oxygen ~]$ mkdir configbackup
[esjolund@oxygen ~]$ mv .gnome* configbackup
[esjolund@oxygen ~]$ mv .gconf* configbackup
[esjolund@oxygen ~]$ mv .metacity configbackup
[esjolund@oxygen ~]$ mv .nautilus configbackup
[esjolund@oxygen ~]$ mv .gtk* configbackup
[esjolund@oxygen ~]$ mv .kde* configbackup
[esjolund@oxygen ~]$ mv .config configbackup

[esjolund@oxygen ~]$ exit

When your desktop is very unresponsive and every action takes forever, then probably your desktop programs are reading and writing to AFS volume that is located on an overloaded AFS file server. This overload is most often caused by someone starting a lot of cluster jobs that all try to write to the same file. See also Section 14.2.9, “Current status of the AFS file servers” and Section 34.6.1, “don't write to the same AFS file from many nodes at the same time”

Example 26. how to detect which users access your afs file server a lot

Note, that your different afs volumes might located at different afs file servers. If you notice that a certain directory is very slow (for instance ~/some_slow_directory), you could first find out the name of that afs file server by running

esjolund@kantarell:~$ fs whereis ~/some_slow_directory
File /afs/pdc.kth.se/home/e/esjolund/some_slow_directory is on host grinner.pdc.kth.se 

	Warning
	Please use the following script with care. It is quite demanding for the AFS servers.

esjolund@kantarell:~$ /usr/bin/vos listvol grinner.pdc.kth.se -quiet -format -extended | awk '$1 == "BEGIN_OF_ENTRY" {involume = 1} involume && $1 == "name" {name
= $2} involume && $1 == "dayUse" {dayUse=$2} involume && $1 == "END_OF_ENTRY" {involume = 0; print dayUse " " name}' | sort -n
100217 prj.sbc.vgr.2
107137 prj.pdc.brandenb.11
715265 prj.sbc.shayat.4
1001473 prj.pdc.onofrios.0
1148417 prj.pdc.hmeier.0
1667713 prj.pdc.stefanc.0
28449537 prj.pdc.brandenb.10
esjolund@kantarell:~$ 

Here we see that the user brandenb access this AFS file server a lot.

See also Section 15.1, “Run chromium with the profile on the local harddrive”

Every Kerberos account has an expiration date. Unfortunately there is no command to see what date that is, so you may be surprised when you some day start getting warning messages.

s07n01 101% kinit
maccallr@NADA.KTH.SE's Password:
Your account will expire at Thu Jan 29 23:59:00 2004

After the warning period you won't be able to get any tickets.

-bash-2.05b$ kinit -l 1month -f melen
melen@NADA.KTH.SE's Password:
kinit: krb5_get_init_creds: Client (melen@NADA.KTH.SE) expired
-bash-2.05b$

When you start getting these messages, send an email to <support@pdc.kth.se> and ask them to extend the life time of your account.

For most kerberos accounts (principals) the kerberos password has an expiration date. Unfortunately there is no command to see what date that is. Here is shell session from an Apple computer:

$ kinit -f -l720hours joelgs@NADA.KTH.SE
Your password has expired, would you like to change it? (yes/no): yes
Please enter the old password for joelgs@NADA.KTH.SE: 
Please enter the new password for joelgs@NADA.KTH.SE: 
Verifying, please re-enter the new password for joelgs@NADA.KTH.SE again: 
Authentication error
Bad request

Maybe it is a bug but Joel solved it by running:

$ kpasswd joelgs@NADA.KTH.SE
Please enter the old password for joelgs@NADA.KTH.SE: 
Please enter the new password for joelgs@NADA.KTH.SE: 
Verifying, please re-enter the new password for joelgs@NADA.KTH.SE again: 

$

Sometimes you get "Permission denied" when you try to access afs files, even though klist tells you that you have valid tickets . Sometimes the problem is solved by first deleting your tickets with kdestroy and then getting new tickets with kinit.

If you want to log into a computer with ssh and start a script, you should after login open up a new PAG ( i.e. a credential cache sandbox ).

[esjolund@otto ~]$ ssh fegnar.sbc.su.se
esjolund@fegnar.sbc.su.se's password:
Last login: Thu Jan 19 10:32:23 2006 from otto.sbc.su.se
[esjolund@fegnar ~]$ pagsh
[esjolund@fegnar ~]$ kinit -f -l1week
esjolund@NADA.KTH.SE's Password:
[esjolund@fegnar ~]$ ./my_script.sh &
[1] 4510
[esjolund@fegnar ~]$ exit
exit
[esjolund@fegnar ~]$ exit
logout
Connection to fegnar closed.
[esjolund@otto ~]$ 

If you don't run pagsh and kinit your script won't have any afs tokens after you log out from the computer.

If there is a temporary AFS problem

[esjolund@otto ~]$ ls
ls: cannot open directory .: Connection timed out

You will not be able to work with your desktop as normal. If you are impatient and want to work by just using the local hard drive, there is a work around:

Log in to a text terminal by typing Ctrl+Alt+F2 . Then log in and type the commands

[esjolund@otto ~]$ mkdir /tmp/tmpdir
[esjolund@otto ~]$ export HOME=/tmp/tmpdir

Instead of /tmp/tmpdir choose some arbitrary directory name.

[esjolund@otto ~]$ startx /usr/bin/startkde -- :1

You have now started a second X server, that will be accessible by typing Ctrl+Alt+F8 or Ctrl+Alt+F9 ( I have seen both ). For web browsing the web browser konqueror is now the recommended alternative because it seems to have less problems when the home dir has been relocated to a subdirectory of /tmp.

When AFS is working as normal again, don't forget logging out of all your extra sessions. Go through Ctrl+Alt+F1 to Ctrl+Alt+F9 and log out the extra sessions. See also Section 14.2.9, “Current status of the AFS file servers” for more info about the current health status of AFS.

You can make firefox much faster by not storing cached data on AFS. Type "about:config" in the URL bar. Either you might want to entirely disable disk caching by setting "browser.cache.disk.enable" to false or you might want to add the variable "browser.cache.disk.parent_directory" and let it have the value "/scratch/yourusername/cache". Make sure you create the directory too.

[esjolund@oxygen esjolund]$ mkdir /scratch/esjolund/cache

A lot of strange problems regarding firefox can be solved by restarting firefox. You first have to make sure there are no firefox processes left around.

[esjolund@oxygen esjolund]$ pgrep -fl firefox
3981 /bin/sh /usr/lib/firefox-1.0.7/firefox -UILocale en-US
4000 /bin/sh /usr/lib/firefox-1.0.7/run-mozilla.sh /usr/lib/firefox-1.0.7/firefox-bin -UILocale en-US
4005 /usr/lib/firefox-1.0.7/firefox-bin -UILocale en-US

If the pgrep found any firefox processes you should kill them with the pkill command

[esjolund@oxygen esjolund]$ pkill -f firefox

Do another check with the pgrep command and if there still are firefox processes alive you should add the -9 flag to the pkill to be certain the firefox processes go away

[esjolund@oxygen esjolund]$ pkill -9 -f firefox

We should check that there are no lock files laying aruond.

[esjolund@otto ~]$ find ~/.mozilla/ -name lock -o -name .parentlock
/afs/pdc.kth.se/home/e/esjolund/.mozilla/firefox/lirpvjkl.default/.parentlock
/afs/pdc.kth.se/home/e/esjolund/.mozilla/firefox/lirpvjkl.default/lock

We have to remove them

[esjolund@oxygen esjolund]$ rm /afs/pdc.kth.se/home/e/esjolund/.mozilla/firefox/lirpvjkl.default/.parentlock
[esjolund@oxygen esjolund]$ rm /afs/pdc.kth.se/home/e/esjolund/.mozilla/firefox/lirpvjkl.default/lock

Now you can start firefox again. If it doesn't work, you might want to consider to follow the instructions in Section 42.9.5, “resetting firefox configuration”

When firefox already is running on another host or if firefox crashed the last time it was run, you might be surprised to see this window when you start firefox

This indicates that there is a lock file somewhere in your ~/.mozilla/ directory and firefox believes that your default profile is already in use. To fix this follow the instructions in Section 42.9.2, “killing all firefox processes”.

Right now parent lock files are not always removed when firefox is shut down. This is a bug that seems to have a fix in openafs 1.5.1 ( development branch ). We'll have to wait till the next openafs version gets released. Until then try to follow the instructions in Section 42.9.2, “killing all firefox processes”.

Ok, if you have already tried Section 42.9.2, “killing all firefox processes”, and it doesn't work, this is a bulletproof way to solve firefox problems. The solution is to reset the firefox configuration.

First make sure there are no other firefox processes running by following the instructions in Section 42.9.2, “killing all firefox processes”. To reset the firefox configuration move the firefox configuration, ~/.mozilla to a new directory name so that firefox won't find it when started.

[esjolund@oxygen esjolund]$ mv ~/.mozilla ~/.mozilla.old

( A note 2006-10-10: Here you may have to take a look at Section 42.9.6, “how to circumvent an AFS bug that stops firefox”. I hope this problem will be history soon /Erik ). Start firefox again. Firefox can now not find its configuration so it will create a new default configuration directory. Now close firefox. No firefox process should run when you continue from this point.

[esjolund@oxygen esjolund]$ cd
[esjolund@oxygen esjolund]$ find .mozilla .mozilla.old -name bookmarks.html -follow | grep -v US
.mozilla/firefox/wzvrk4sg.default/bookmarks.html
.mozilla.old/firefox/lirpvjkl.default/bookmarks.html

copy your old bookmarks file to your new bookmarks file

[esjolund@oxygen esjolund]$ cp .mozilla.old/firefox/lirpvjkl.default/bookmarks.html .mozilla/firefox/wzvrk4sg.default/bookmarks.html

Now start firefox again. Your settings and plugins were lost but your bookmarks should be there.

This section was written 2006-10-10. I really hope this is temporary problem that soon will go away. Right now firefox won't start if the home directory is in AFS and ~/.mozilla is empty. Try this to circumvent the problem:

[esjolund@oxygen esjolund]$ mkdir /scratch/my_temp
[esjolund@oxygen esjolund]$ export HOME=/scratch/my_temp
[esjolund@oxygen esjolund]$ firefox

Now quit firefox and after that move the newly created $HOME/.mozilla to your home directory.

[esjolund@oxygen esjolund]$ mv $HOME/.mozilla /afs/pdc.kth.se/home/e/esjolund

During the move you may see SELINUX error messages but don't worry.

Please, don't send support requests for private things ( e.g. about your home computers ).

Surprisingly many computer problems can be solved by rebooting your desktop computer. Please, try to see if a reboot of your desktop computer solves the problem.

Verify that you don't overuse your AFS file system quota. All kind of strange errors start to occur as soon as software can't write to the file system anymore.

The support request should be related to your work. Please, don't ask questions about how to configure your desktop to show your vacation photos on your desktop. Please, don't ask questions about how to use your desktop computer at work for listening to music. This is a feature that is not supported. If you are lucky and succeed listening to music with your desktop computer, you can of course use this feature as long as you don't disturb your colleagues sitting in the same room.

Please, accept some inconveniences as it is unrealistic to believe that we will be able to create the perfect computer environment. There are too many software/hardware components that might fail. If you think that you are forced to click the mouse buttons too many times to solve a task or you think that you need to type an unnecessary command once a week, please consider if this is really worth starting a new support request for.

Please, don't do a reply to an unrelated email message. Changing the subject line will not help because the email threading shown in email clients will then show your message in another email thread.

In other words, make sure you create a new message ( in the email client Thunderbird you find it under the menu Message → New Message ).

If you have multiple unrelated support requests, split them up in different emails.

If it is a problem with your desktop computer, please specify the name ( i.e. the hostname ) of your desktop computer. Of course if you cut and paste from your shell session ( Section 44.2.11, “Cut and paste from your shell session” ) the hostname can also be seen there.

A lot of the answers to computer support requests can be found in this user guide. Please, take the time to search the user guide before sending your computer support request. If your support question concerns a specific software, please search the user guide (with Ctrl-f in firefox) for the name of that specific software. In addition to that, please take a look at Section 42, “Troubleshooting”.

Examples with too little information in the subject line

Try to be a bit more specific than just a single substantive. If you have some sort of unique error message it is a good idea to include some of it in the subject line. Here is a list of better and more descriptive subject lines:

If your computer problem is occurring on the command line, please cut and paste as much as possible from your shell session, not just the error message but also the command and the surrounding commands.

Don't write

I get the error message "No such file or directory" when I try to add the program "gnuplot" with the "module" command

instead you should copy and paste like this

alewa@geriatrix:~$ module add gnuplot
bash: /usr/modulecmd/modulecmd.tcl: No such file or directory

As email clients often wrap long lines you might consider saving your shell session to a file, naming it to something like "shellsession.txt" and then attach it to your support email.

Normally you have your passwords written down at home ( see Section 10.1.3, “You are required to write down all your passwords at SBC on a paper and store it at home” ), but if you misplaced your paper note, take a look in the following sub sections.

If after sending a support request email to sbc-sys@sbc.su.se you somehow solve the support request by yourself, please tell sbc-sys@sbc.su.se about it, so no unnecessary work is spent on an issue that has already been resolved.