June 2009

Use rsync to sync, large volumes of data

I found myself in a situation where I wanted to know the list of installed packages on Debian Lenny 5.0 and compare this list with what was on another server. First for a complete list of packages present on a Debian server I use the excellent dpkg . dpkg-l will give a list of installed packages in alphabetical order with version and description and status, sthg like: gocarto01: ~ # dpkg-l Desired = Unknown / Install / Remove / Purge / Hold ii adduser 3.110 add and remove users and groups ii apache2 2.2.9-10+lenny2 Apache HTTP Server metapackage ii apache2-doc 2.2.9-10+lenny2 Apache HTTP Server documentation ii apache2-mpm-prefork 2.2.9-10+lenny2 Apache HTTP Server - traditional non-threaded model ii apache2-utils 2.2.9-10+lenny2 utility programs for webservers ii apache2.2-common 2.2.9-10+lenny2 Apache HTTP Server common files ii apt 0.7.20.2+lenny1 Advanced front-end for dpkg ii apt-file 2.1.5 APT package searching utility - command-line interface ii apt-utils 0.7.20.2 APT utility lenny1 + Programs ii base-files Debian base system miscellaneous 5lenny2 files ii base-passwd 3.5 .20 Debian base system master password and group files ii bash 3.2-4 The GNU Bourne Again SHell rc bind9 1:9.5.1. dfsg.P1-2 Internet Domain Name Server

with this nice list that are saved in a file by making a simple redirect dpkg-l> package-list_serveur1.txt Then on the other server is doing the same dpkg-l> package-list-serveur2.txt and recovered by the previous list example with scp (ssh tools supplied with) scp server1: package-list_serveur1.txt. And finally just a diff between 2 files and you get the list of packages that one side or the other: diff-package package-list-serveur2.txt list_serveur1.txt > ii php5-gd 5.2.6.dfsg.1-1 + GD module for php5 lenny3 264a270> ii php5-sybase 5.2.6.dfsg.1-1 + lenny3 Sybase / MS SQL Server module for php5

So in this example I can see I'm missing packages php5-gd and php5 sybase-on server2 (because there are only present in the generated list on server1.
Good luck if you try to do the same with Microsoft ... course nothing prevents you to filter this list with a grep So with a:
gocarto01: ~ # dpkg-l PEAR - PHP Extension and application based
ii php5-cli 5.2.6.dfsg.1-1 + command-line interpreter lenny3 pour la php5 script ii php5-common
5.2.6.dfsg.1-1 + lenny3 Common files for packages built from the PHP
ii php5-gd 5.2.6.dfsg.1-1 + GD module for php5 lenny3
ii php5-pgsql 5.2.6.dfsg.1-1 + PostgreSQL module for php5 lenny3 a man dpkg will show you lots of other possibilities of this great tool. So if you want to know the list of files installed by package postgresql-8.3-postgis simply launch: dpkg-L-8.3-postgis postgresql gocarto01: ~ # dpkg-L-8.3-postgis postgresql
/. / usr / usr / share
/ usr / share / doc / usr/share/doc/postgresql-8.3-postgis / usr/share/doc/postgresql-8.3-postgis/changelog.gz / usr / share / doc/postgresql-8.3-postgis/copyright / usr/share/doc/postgresql-8.3-postgis/README.Debian.gz
/ usr/share/doc/postgresql-8.3-postgis/changelog.Debian.gz
/ usr/share/postgresql-8.3-postgis / usr/share/postgresql-8.3-postgis/lwpostgis_upgrade.sql / usr/share/postgresql-8.3-postgis/lwpostgis.sql
/ usr/share/postgresql-8.3-postgis/spatial_ref_sys.sql

 / usr / lib 
 / usr / lib / postgis 
 / usr/lib/postgis/1.3.3 
 / usr/lib/postgis/1.3.3 / 
 postgres / usr/lib/postgis/1.3.3/postgres/8.3 
 / usr/lib/postgis/1.3.3/postgres/8.3/lib 
 / usr/lib/postgis/1.3.3/postgres/8.3/lib / liblwgeom.so.1.3 
 / usr / lib / postgresql 
 / usr/lib/postgresql/8.3 
 / usr/lib/postgresql/8.3/lib 
 / usr/lib/postgresql/8.3/lib/liblwgeom.so.1 
 / usr/lib/postgresql/8.3/lib/liblwgeom.so

Have a nice distro SliTaz its USB

Proxmox I installed on my server with 2 physical disks mirrored RAID1 (with a hard raid controller PERC) seen as / dev / sda with a size of 73GB for Linux.
Proxmox partitionnment installed with a default I had in the end the following partitionnment download:

calc0: / var / lib / vz # fdisk / dev / sda-l
Disk / dev / sda : 73.5 GB, 73555509248 bytes 255 heads
, 63 sectors / track, 8942 cylinders Units =

cylinders of 16065 * 512 = 8225280 bytes Device Boot Start End Blocks Id System

/ dev/sda1 * 1 66 524288 83 Linux / dev/sda2 66 8942 7130 2295 8th Linux LVM

 
 
 and at the LVM had the following info on the physical level, the volume group and logical

calc0nokvm: / var / lib / vz # pvdisplay

 
 
 --- Physical volume --- PV 
 
 
 Name / dev/sda2 VG Name 
 
 
 pve 
 
 
 PV Size 68.00 GB / not usable 0

Allocatable yes (view full)

 
 PE Size (KByte) 4096 Total 
 
 
 PE 17407 Free PE 
 
 
  0 
  
  
   Allocated PE          17407 
  
  
   PV UUID               3bzRCT-gRWz-7EGo-IMAl-rIAq-SJDz-rLqOOd 
  
  
 calc0nokvm:/var/lib/vz# vgdisplay 
  
  
   --- Volume group --- 
  
  
   VG Name               pve 
  
  
   System ID

Format lvm2 Metadata Areas 1

Metadata Sequence No 4

VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 3
Open LV 3
Max PV 0
Cur PV 1 Act PV 1

VG Size 68.00 GB
PE Size 4.00 MB

Total PE 17407 Alloc PE / Size 16385 / 64.00 GB
Free PE / Size 1022 / 3.99 GB
VG UUID qbzjFv-rYTt-pRGE-FfUP-BgXr-z54a-5QNmXh

calc0nokvm:/var/lib/vz# lvdisplay
--- Logical volume ---
LV Name /dev/pve/swap
VG Name pve
LV UUID cuS75E-ppYt-Rk1H-8CyL-Hhc9-Gs3b-v5yMUW
LV Write Access read/write
LV Status available
# open 1
LV Size 7.00 GB
Current LE 1792
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 254:0
--- Logical volume ---
LV Name /dev/pve/root
VG Name pve
LV UUID gZ4825-OGRN-vqvm-9JUF-1FvK-II91-feQDcg
LV Write Access read/write
LV Status available
# open 1
LV Size 17.00 GB
Current LE 4352
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 254:1
--- Logical volume ---

LV Name /dev/pve/data
VG Name pve
LV UUID uovK3Q-gOP8-svnT-TxHg-CA3X-Cek8-FaLZG1
LV Write Access read / write LV Status

available # open 1 LV Size
40.00 GB
Current LE 10241
Segments 1 Allocation inherit

Read ahead sector 0 Block device 254:2

What makes it not much I had to create my virtual machines for OpenVZ a simple df told me that I only had little more than 19GB free: calc0nokvm: / var / lib / vz # df-hT

Filesystem Type Size Used Avail Use% Mounted on / dev / pve / root ext3 17G 750M 16G 5% / tmpfs tmpfs 3.9G 0 3.9G 0% / lib / init / rw udev tmpfs 10M
64K 10M 1% / dev tmpfs tmpfs 3.9G 0 3.9G 0% / dev / shm / dev / mapper / pve-data ext3 40G 21G 19G
53 % / var / lib / vz / dev/sda1 ext3 35M 496m 436M 8% / boot
It turns out that I had added three brand new 400GB drives in RAID5 so that I had a / dev / sdb 800GB of wondering who to be used. I unable to format is to use a symbolic link to link / var / lib / vz / root partition to my new seuleument of the tremendous blow web management interface Proxmox would not have given correct information on the available space, so I ' I decided to use the great opportunity to LVM2 Logical Volume Manager for Linux.
Creating a partition on monnouveau Disk / dev / sdb with fdisk / dev / sdb and type 8e Linux LVM which gives me:
calc0nokvm: / var / lib / vz # fdisk / dev / sdb-l Disk / dev / sdb: 899.9 GB, 899995926528 bytes 255 heads
, 63 sectors / track, 109418 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System /dev/sdb1 1 109418 878900053+ 8e Linux LVM
Initialisation du disque au niveau lvm avec un
pvcreate /dev/sdb1
calc0nokvm:/var/lib/vz#
pvcreate /dev/sdb1 Physical volume "/dev/sdb1" successfully created
calc0nokvm:/var/lib/vz# pvdisplay
--- Physical volume ---
PV Name /dev/sda2 VG Name pve PV Size 68.00 GB / not usable 0
Allocatable yes PE Size (KByte) 4096 Total PE 17407
Free PE 1022 Allocated PE 16385 PV UUID 3bzRCT-gRWz-7EGo-IMAl-rIAq-SJDz-rLqOOd
--- NEW Physical volume --- PV Name /dev/sdb1
VG Name PV Size 838.18 GB Allocatable NO
PE Size (KByte) 0 Total PE 0 Free PE 0
Allocated PE 0 PV UUID B1eOlE-xfwe-wDDJ-E15M-o5lV-VMSC-LJ9Hdk

Ajout de ce nouveau volume physique au "volume groupe" pve calc0nokvm:/var/lib/vz#
vgextend pve /dev/sdb1 Volume group "pve" successfully extended
calc0nokvm:/var/lib/vz# vgdisplay
--- Volume group --- VG Name pve System ID
Format lvm2 Metadata Areas 2

Metadata Sequence No 5
VG Access read/write
VG Status resizable MAX LV 0 Cur LV 3
Open LV 3
Max PV 0
Cur PV 2
Act PV 2
VG Size 906.18 GB
PE Size 4.00 MB
Total PE 231982
Alloc PE / Size 16385 / 64.00 GB
Free PE / Size 215597 / 842.18 GB
VG UUID qbzjFv-rYTt-pRGE-FfUP-BgXr-z54a-5QNmXh

Increasing the size of a logical volume, in this case I add 100GB to volume / dev / pve / data
calc0nokvm: / var / lib / vz # lvextend-L 100 G / dev / PvE / data

Extending logical volume data to 140.00 GB Logical volume data
Successfully resized

not forget to update the filesystem with
calc0nokvm: / var / lib / vz #
resize2fs 1.40-WIP (14-Nov-2006) Filesystem at / dev / pve / SI data mounted on / var / lib / vz; on-line resizing required old
desc_blocks = 3 = 9 new_desc_blocks Performing an on-line resize of / dev / pve / data to 36701184 (4k) blocks. The filesystem on / dev / pve / data Is Now 36701184 blocks long. calc0nokvm: / var / lib / vz # df-hT

Filesystem Type Size Used Avail Use% Mounted on / dev / pve / root ext3 17G 750M 16G 5% / tmpfs tmpfs 3.9G 0 3.9G 0% / lib / init / rw udev tmpfs 10M
64K 10M 1% / dev tmpfs tmpfs 3.9G
0 3.9G 0% / dev / shm
/ dev / mapper / pve-data ext3 138G 21G
118g 16% / var / lib / vz
/ dev/sda1 ext3 35M 496m 436M 8% / boot

Now as you can see from the above, I df well and truly won over 100GB on my pve-data logical volume or is "mount" my / var / lib / vz. And all without service interruption. I can any time restart the command to add space on one of my logical volumes. This flexibility is not beautiful? "

This article on Wikipedia
is useful, as the LVM Howto

Yellow To Brown Behind

Tuesday, June 9, 2009

Kottakkal Arya Vaidya Salahair Fall

Thursday, June 4, 2009

Does My Dog Have Ringworm Or Hot Spot?