Difference between revisions of "Beef"
(→Current HD setup) |
|||
(18 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | Kör | + | Kör OpenAFS 1.6.9-2+deb8u2-debian built 2015-03-02 på Debian 8. |
+ | Server namngiven av Mho, in the food namespace. | ||
+ | |||
+ | Beef är en AFS-filserver som vi ärvde från mho. Det är en AMD Athlon II X4 600e med 8Gb RAM, 2 80GB WD som systemdisk och 6x2T Spinpoint för lagringen (en spare) på ZFS. | ||
+ | |||
+ | Debian 8 är uppsatt med nsg's standardinstall med 2 speglade rootdiskar. | ||
+ | |||
+ | NOTERA! Texten på den här sidan åker runt och ändrar på sig mycket, kommer snygga till när maskinen är klar. | ||
+ | |||
+ | == Essentials == | ||
+ | |||
+ | apt-get install ed | ||
+ | apt-get install parted | ||
+ | apt-get inslall smartmontools | ||
+ | apt-get install screen | ||
+ | apt-get install postfix # so we can email errors | ||
+ | |||
+ | == Partionera disk == | ||
+ | |||
+ | Först lista ut vilken som är den disken som skall bli zfs. Gör sedan: | ||
+ | |||
+ | for n in c d e f g h ; do parted -s /dev/sd$n mklabel gpt; parted -s /dev/sd$n mkpart primary ext2 0% 100%; parted -s /dev/sd$n print; done | ||
+ | |||
+ | Ibland måste man ta bort gammla md med mdadm --detail /dev/mdX ; mdadm -S /dev/mdX ; mdadm --zero-superblock /dev/sdXX ... | ||
+ | |||
+ | == Install zfs == | ||
+ | |||
+ | # apt-get install lsb-release | ||
+ | # wget http://archive.zfsonlinux.org/debian/pool/main/z/zfsonlinux/zfsonlinux_6_all.deb | ||
+ | # dpkg -i zfsonlinux_6_all.deb | ||
+ | # apt-get update | ||
+ | # apt-get install debian-zfs | ||
+ | |||
+ | == Hitta bra partitioner == | ||
+ | |||
+ | Alla diskar från sdc uppåt utom den första | ||
+ | |||
+ | ls -l /dev/disk/by-id/ | egrep 'wwn.*../../sd[c-z]1' | awk 'NR>1{print "/dev/disk/by-id/"$9}' | tr '\n' ' ' | ||
+ | |||
+ | Bara den som inte togs med ovan (den tar vi som spare) | ||
+ | |||
+ | ls -l /dev/disk/by-id/ | egrep 'wwn.*../../sd[c-z]1' | awk 'NR>1{print "/dev/disk/by-id/"$9}' | ||
+ | |||
+ | Kolla innan man pastar ihop kommandot | ||
+ | |||
+ | zpool create vicepa raidz2 PASTE-HERE spare PASTE-MORE-HERE | ||
+ | |||
+ | Kolla nu med zpool status och df och kanske en dd if=/dev/zero of=/vicepa/big bs=1024k count=20000 att allt lirar. | ||
+ | |||
+ | Om ja, grattis du har nu en ZFS pool. | ||
+ | |||
+ | == OpenAFS == | ||
+ | |||
+ | # apt-get install openafs-fileserver | ||
+ | # /etc/init.d/openafs-fileserver stop | ||
+ | |||
+ | Hm, vi fick med en ntp, fanns det inte redan | ||
+ | Hm, vi fick med en klient men ingen kärnamodul. Okejdå | ||
+ | |||
+ | Kopiera /etc/openafs/BosConfig | ||
+ | Kopiera /etc/opensafs/server/rxkad.keytab | ||
+ | Kopiera /etc/opensafs/server/UserList | ||
+ | Sätt statisk addr i /etc/network/interfaces | ||
+ | Sätt lång hostname (med kommandot och i /etc/hostname) | ||
+ | Fixa /etc/hosts: | ||
+ | |||
+ | 127.0.0.1 localhost | ||
+ | 130.237.234.216 beef.stacken.kth.se beef | ||
+ | |||
+ | == NTP == | ||
+ | |||
+ | Fixa ntp.conf | ||
+ | |||
+ | server ntp1.kth.se iburst maxpoll 7 | ||
+ | server ntp2.kth.se iburst maxpoll 7 | ||
+ | server ntp3.kth.se iburst maxpoll 7 | ||
+ | server ntp1.sp.se iburst maxpoll 7 | ||
+ | server ntp2.sp.se iburst maxpoll 7 | ||
+ | |||
+ | == Current HD setup == | ||
+ | |||
+ | # for n in `ls -l /dev/disk/by-id/ | egrep 'wwn.*../../sd[a-z]$' | awk '{print "/dev/disk/by-id/"$9}'` ; do echo '==' ; smartctl -i $n | egrep -i 'serial|WWN' ; done | ||
+ | == | ||
+ | Serial Number: S1XWJ1KS925135 | ||
+ | LU WWN Device Id: 5 0024e9 00225cd42 | ||
+ | == | ||
+ | Serial Number: S1XWJ1KS925137 | ||
+ | LU WWN Device Id: 5 0024e9 00225cd59 | ||
+ | == | ||
+ | Serial Number: S1XWJ1LSC08172 | ||
+ | LU WWN Device Id: 5 0024e9 002aff716 | ||
+ | == | ||
+ | Serial Number: 5XW00L4M | ||
+ | LU WWN Device Id: 5 000c50 01689b07a | ||
+ | == | ||
+ | Serial Number: S1XWJ1MSC02982 | ||
+ | LU WWN Device Id: 5 0024e9 002affe3d | ||
+ | |||
+ | == | ||
+ | WAY-OUT Serial Number: S1XWJ1LSC08170 | ||
+ | WAY-OUT WWN Device Id: 5 0024e9 002aff710 | ||
+ | == | ||
+ | HAS BEEN REPLACED Serial Number: S1XWJ1MSC02977 | ||
+ | HAS BEEN REPLACED LU WWN Device Id: 5 0024e9 002affe22 | ||
+ | == | ||
+ | |||
+ | In /dev/disk/by-id/ the WWN numers are without space. | ||
+ | |||
+ | Fact: The WD 80GB of the OS don't have any WWN and therefore are not matched in the grep above. | ||
+ | |||
+ | sda: | ||
+ | Serial Number: WD-WMAM9CUF8084 | ||
+ | sdb: | ||
+ | Serial Number: WD-WMAM9CUS4662 | ||
+ | |||
+ | == Setup AFSBACKUP == | ||
+ | |||
+ | Copy tivoli and afsbackup stuff | ||
+ | |||
+ | # ssh mount-kilimanjaro.stacken.kth.se 'cd /opt && tar czf - afsbackup tivoli.5.5.2.7' | ssh beef.stacken.kth.se 'cd /opt && tar xzvpf -' | ||
+ | beef# cd /opt/ | ||
+ | beef# ln -s tivoli.5.5.2.7 tivoli | ||
+ | |||
+ | Install stuff (tsmpipe is 64 and dsmc is 32 bit) | ||
+ | |||
+ | dpkg --add-architecture i386 | ||
+ | apt-get update | ||
+ | apt-get install libc6:i386 libc6-i686:i386 libgcc1:i386 libstdc++5:i386 # dsmc needs this | ||
+ | apt-get install libstdc++5 # tsmpipe needs this | ||
+ | apt-get install bc # needed to calculate stuff in shell script | ||
+ | |||
+ | Talk to PDC and get password | ||
+ | |||
+ | Test with dsmc and set password | ||
+ | |||
+ | export DSM_CONFIG=/opt/tivoli/tsm/client/ba/bin/dsm.opt.afsbackup | ||
+ | export DSMI_CONFIG=$DSM_CONFIG | ||
+ | export DSM_DIR=`dirname $DSM_CONFIG` | ||
+ | export DSMI_DIR=$DSM_DIR | ||
+ | LD_LIBRARY_PATH=/opt/tivoli.5.5.2.7/tsm/client/api/bin /opt/tivoli.5.5.2.7/tsm/client/ba/bin/dsmc q file | ||
+ | : This should ask for nodename and password | ||
+ | /opt/afsbackup/bin/tsmpipe -A -t -s /scratch -f /H.haba.backup.0\* | ||
+ | : This should give a list of backed volumes | ||
+ | |||
+ | Fix cron (don't forget to remove on old backup server or it will be very strange when 2 try at same tile) | ||
+ | |||
+ | root@beef:/# cat > /etc/cron.d/backupafs | ||
+ | 17 01 * * * root /opt/afsbackup/bin/dsmbackupstacken.sh -i | ||
+ | |||
+ | You might want to take old logs with you | ||
+ | |||
+ | root@mount-kilimanjaro:/var/log# tar cf - afsbackup | ssh beef 'cd /var/log && tar xpvf -' | ||
+ | |||
+ | You want to copy these config files | ||
+ | |||
+ | root@mount-kilimanjaro:/# tar cf - scratch | ssh beef 'cd / && tar xpvf -' | ||
+ | scratch/ | ||
+ | scratch/volumes-to-skip.lst | ||
+ | scratch/servers-to-skip.lst | ||
+ | |||
+ | == Repair later == | ||
[[File:Harald-reparerar-afs-server.jpg|200px|thumb|left]] | [[File:Harald-reparerar-afs-server.jpg|200px|thumb|left]] | ||
[[Category:Maskiner]] [[Category:Filserver]] | [[Category:Maskiner]] [[Category:Filserver]] |
Latest revision as of 14:12, 23 July 2015
Kör OpenAFS 1.6.9-2+deb8u2-debian built 2015-03-02 på Debian 8. Server namngiven av Mho, in the food namespace.
Beef är en AFS-filserver som vi ärvde från mho. Det är en AMD Athlon II X4 600e med 8Gb RAM, 2 80GB WD som systemdisk och 6x2T Spinpoint för lagringen (en spare) på ZFS.
Debian 8 är uppsatt med nsg's standardinstall med 2 speglade rootdiskar.
NOTERA! Texten på den här sidan åker runt och ändrar på sig mycket, kommer snygga till när maskinen är klar.
Contents
Essentials
apt-get install ed apt-get install parted apt-get inslall smartmontools apt-get install screen apt-get install postfix # so we can email errors
Partionera disk
Först lista ut vilken som är den disken som skall bli zfs. Gör sedan:
for n in c d e f g h ; do parted -s /dev/sd$n mklabel gpt; parted -s /dev/sd$n mkpart primary ext2 0% 100%; parted -s /dev/sd$n print; done
Ibland måste man ta bort gammla md med mdadm --detail /dev/mdX ; mdadm -S /dev/mdX ; mdadm --zero-superblock /dev/sdXX ...
Install zfs
# apt-get install lsb-release # wget http://archive.zfsonlinux.org/debian/pool/main/z/zfsonlinux/zfsonlinux_6_all.deb # dpkg -i zfsonlinux_6_all.deb # apt-get update # apt-get install debian-zfs
Hitta bra partitioner
Alla diskar från sdc uppåt utom den första
ls -l /dev/disk/by-id/ | egrep 'wwn.*../../sd[c-z]1' | awk 'NR>1{print "/dev/disk/by-id/"$9}' | tr '\n' ' '
Bara den som inte togs med ovan (den tar vi som spare)
ls -l /dev/disk/by-id/ | egrep 'wwn.*../../sd[c-z]1' | awk 'NR>1{print "/dev/disk/by-id/"$9}'
Kolla innan man pastar ihop kommandot
zpool create vicepa raidz2 PASTE-HERE spare PASTE-MORE-HERE
Kolla nu med zpool status och df och kanske en dd if=/dev/zero of=/vicepa/big bs=1024k count=20000 att allt lirar.
Om ja, grattis du har nu en ZFS pool.
OpenAFS
# apt-get install openafs-fileserver # /etc/init.d/openafs-fileserver stop
Hm, vi fick med en ntp, fanns det inte redan Hm, vi fick med en klient men ingen kärnamodul. Okejdå
Kopiera /etc/openafs/BosConfig Kopiera /etc/opensafs/server/rxkad.keytab Kopiera /etc/opensafs/server/UserList Sätt statisk addr i /etc/network/interfaces Sätt lång hostname (med kommandot och i /etc/hostname) Fixa /etc/hosts:
127.0.0.1 localhost 130.237.234.216 beef.stacken.kth.se beef
NTP
Fixa ntp.conf
server ntp1.kth.se iburst maxpoll 7 server ntp2.kth.se iburst maxpoll 7 server ntp3.kth.se iburst maxpoll 7 server ntp1.sp.se iburst maxpoll 7 server ntp2.sp.se iburst maxpoll 7
Current HD setup
# for n in `ls -l /dev/disk/by-id/ | egrep 'wwn.*../../sd[a-z]$' | awk '{print "/dev/disk/by-id/"$9}'` ; do echo '==' ; smartctl -i $n | egrep -i 'serial|WWN' ; done == Serial Number: S1XWJ1KS925135 LU WWN Device Id: 5 0024e9 00225cd42 == Serial Number: S1XWJ1KS925137 LU WWN Device Id: 5 0024e9 00225cd59 == Serial Number: S1XWJ1LSC08172 LU WWN Device Id: 5 0024e9 002aff716 == Serial Number: 5XW00L4M LU WWN Device Id: 5 000c50 01689b07a == Serial Number: S1XWJ1MSC02982 LU WWN Device Id: 5 0024e9 002affe3d
== WAY-OUT Serial Number: S1XWJ1LSC08170 WAY-OUT WWN Device Id: 5 0024e9 002aff710 == HAS BEEN REPLACED Serial Number: S1XWJ1MSC02977 HAS BEEN REPLACED LU WWN Device Id: 5 0024e9 002affe22 ==
In /dev/disk/by-id/ the WWN numers are without space.
Fact: The WD 80GB of the OS don't have any WWN and therefore are not matched in the grep above.
sda: Serial Number: WD-WMAM9CUF8084 sdb: Serial Number: WD-WMAM9CUS4662
Setup AFSBACKUP
Copy tivoli and afsbackup stuff
# ssh mount-kilimanjaro.stacken.kth.se 'cd /opt && tar czf - afsbackup tivoli.5.5.2.7' | ssh beef.stacken.kth.se 'cd /opt && tar xzvpf -' beef# cd /opt/ beef# ln -s tivoli.5.5.2.7 tivoli
Install stuff (tsmpipe is 64 and dsmc is 32 bit)
dpkg --add-architecture i386 apt-get update apt-get install libc6:i386 libc6-i686:i386 libgcc1:i386 libstdc++5:i386 # dsmc needs this apt-get install libstdc++5 # tsmpipe needs this apt-get install bc # needed to calculate stuff in shell script
Talk to PDC and get password
Test with dsmc and set password
export DSM_CONFIG=/opt/tivoli/tsm/client/ba/bin/dsm.opt.afsbackup export DSMI_CONFIG=$DSM_CONFIG export DSM_DIR=`dirname $DSM_CONFIG` export DSMI_DIR=$DSM_DIR LD_LIBRARY_PATH=/opt/tivoli.5.5.2.7/tsm/client/api/bin /opt/tivoli.5.5.2.7/tsm/client/ba/bin/dsmc q file : This should ask for nodename and password /opt/afsbackup/bin/tsmpipe -A -t -s /scratch -f /H.haba.backup.0\* : This should give a list of backed volumes
Fix cron (don't forget to remove on old backup server or it will be very strange when 2 try at same tile)
root@beef:/# cat > /etc/cron.d/backupafs 17 01 * * * root /opt/afsbackup/bin/dsmbackupstacken.sh -i
You might want to take old logs with you
root@mount-kilimanjaro:/var/log# tar cf - afsbackup | ssh beef 'cd /var/log && tar xpvf -'
You want to copy these config files
root@mount-kilimanjaro:/# tar cf - scratch | ssh beef 'cd / && tar xpvf -' scratch/ scratch/volumes-to-skip.lst scratch/servers-to-skip.lst