Recovering from missing (or wrong) LVM2 (EVMS) metadata

I just want to document one possible procedure, as it may prove helpful to others.

The setup of the system was as follows: Three disks in a RAID5 array, LVM2 with evms metadata on top and a single ext3 filesystem within an LVM2 logical volume spanning the whole RAID array. Two disks of the array were removed by the operating system (probably because of broken sectors) and the array was marked as corrupt. Unfortunately the failed disks caused problems in the LVM2 metadata and the logical volumes did not mount anymore after forcing the RAID to start (passing the –force parameter to the mdadm assemble command). Please note that the force option did not work on Ubuntu Edgy, probably due to a bug. I upgraded to Feisty during the recovery process.

I took advantage of the fact that I simply used the whole md0 as a container for my filesystem wrapped in LVM2. Because there was no striping over several PVs involved, I was sure that the filesystem is somewhere on md0 starting at block n. I simply dumped the first few hundred kibibytes from md0 and looked for an ext2/3 header. In order to sharpen your pattern recognition system for ext2/3 headers, generate a few filesystems of various sizes within a file and have a look at them. The amount of sectors (512 bytes) used by LVM2 at the beginning seems to be constant. It uses 384 sectors. Then I created a new device with dmsetup which started at the offset I just calculated and ended at the end of md0. A simple mount and I was up and running.

Say ‘blockdev –getsize /dev/md0’ returns 976783616 sectors. 976783616 – 384 = 976783232

sudo dmsetup create md0hack –table “0 976783232 linear /dev/md0 384”
sudo mount /media/mapper/md0hack /media/something

And you are up and running.

One thought on “Recovering from missing (or wrong) LVM2 (EVMS) metadata

Leave a Reply

Your email address will not be published. Required fields are marked *