The FreeBSD Diary

The FreeBSD Diary (TM)

Providing practical examples since 1998

If you buy from Amazon USA, please support us by using this link.
[ HOME | TOPICS | INDEX | WEB RESOURCES | BOOKS | CONTRIBUTE | SEARCH | FEEDBACK | FAQ | FORUMS ]
vinum - A simple install to improve performance 10 August 1999
Need more help on this topic? Click here
This article has 12 comments
Show me similar articles
vinum is a great tool.  As the author says:

The Vinum Volume Manager is a block device driver which implements virtual disk drives. It isolates disk hardware from the block device interface and maps data in ways which result in an increase in flexibility, performance and reliability compared to the traditional slice view of disk storage. Vinum implements the RAID-0, RAID-1 and RAID-5 models, both individually and in combination.

The vinum home page is at http://www.vinumvm.org/.   There is also lost article by Jake Burkholder which I used for the basis of this exercise.  You should read vinum(4) and vinum(8) and before continuing with this article.  It explains the concepts and terms used within this article (such as plexes, subdisks, and volumes) and which are necessary if you wish to implement vinum.

There is also an article by Jake Burkholder in FreeBSD'Zine

Why use vinum?
Well, in my case, it was to create a larger volume and to increase performance.   The box I was working on was destined to become the www and cvsup mirror for New Zealand FreeBSD User Group. But the box had only 1GB.  So I removed those two 500MB drives and added in three 1GB drives.   You may wish to read how I did the drive swap.  If you wish to learn more about striping and RAID, please read http://www.lemis.com/vinum.html as I will not cover those topics at all.
The best vinum resources
Remember your size
The first step was to figure out what size my disks were.  My first attempts involved assuming they were both 1GB drives.  There weren't.  And this caused problems.  So here's how to find out the size.  The drives I'm using are SCSI drives da1 and da2.
# fdisk da1
******* Working on device /dev/rda1 *******
parameters extracted from in-core disklabel are:
cylinders=496 heads=118 sectors/track=35 (4130 blks/cyl)

parameters to be used for BIOS calculations are:
cylinders=496 heads=118 sectors/track=35 (4130 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
    start 0, size 2051000 (1001 Meg), flag 80 (active)
        beg: cyl 0/ sector 1/ head 0;
        end: cyl 342/ sector 35/ head 117
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>
# fdisk da2
******* Working on device /dev/rda2 *******
parameters extracted from in-core disklabel are:
cylinders=4085 heads=14 sectors/track=36 (504 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=4085 heads=14 sectors/track=36 (504 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
    start 0, size 2059140 (1005 Meg), flag 80 (active)
        beg: cyl 0/ sector 1/ head 0;
        end: cyl 953/ sector 36/ head 13
The data for partition 2 is:
<UNUSED><unused>
The data for partition 3 is:
<UNUSED><unused>
The data for partition 4 is:<unused>
<UNUSED>

If you look above, you'll find that da1 is 1001MB and da2 is 1005MB.

Disk partitions
I am assuming your have your disk partitions and slices ready to go.  If you don't, please see Formatting Media For Use With FreeBSD but for your reference, my two disk are "dangerously" dedicated.
Addenda - 26 August 1999
When I originally wrote the section on File System Types, my default editor was vi.   Then a reader sent me a message telling me telling me that that various UNIX commands use the EDITOR command.  Thus, you can set your default editor for various commands.  I thought that was brilliant!  Of course I rushed off to write a new article about that, only to find I'd already written one and forgotten about it!

If you want to see how to set your default editor, see changing your default editor.

File system types
vinum only works on vinum type file systems.  By default, the file systems are of type unused. You can change the file system type (fstype) to "vinum" by using disklabel.
# disklabel -e da1

If you know how to use vi, then go ahead and ignore the following instructions.   Otherwise this should help a big help:

  1. use the arrow keys to position yourself at the start of where it says unused under the fstype column.
  2. press x to delete the existing fstype entry
  3. press i to start insert
  4. type vinum
  5. press escape to end insert mode
  6. press :wq, then enter to save your changes and exit
  7. OR presse :q! to exit without saving.

Here is a properly configured disk:

[root@fred:~] # disklabel da1
# /dev/rda1c:
type: SCSI
disk: da1s1
label: 
flags:
bytes/sector: 512
sectors/track: 35
tracks/cylinder: 171
sectors/cylinder: 5985
cylinders: 342
sectors/unit: 2051000
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # milliseconds
track-to-track seek: 0  # milliseconds
drivedata: 0 

8 partitions:
#        size offset fstype   [fsize bsize bps/cpg]
  c:  2051000      0 unused        0     0  # (Cyl.    0 - 342*)
  e:  2051000      0  vinum                 # (Cyl.    0 - 342*)
Protect your data
The process of creating a vinum volume will erase the data which exists on the file system.  If you are converting existing disks to use vinum, back up your data first.  You can restore the data later.
Create the configuration file

WARNING: Grey Lehey has said not to create striped plexes with a power of 2 stripe size. DO NOT USE THE 256k VALUES I USED.

As mentioned in Jake's article, I used /etc/vinum.conf as my configuration file.  Here is what it contains:

drive d1 device /dev/da1e
drive d2 device /dev/da2e

volume test

 plex org striped 256k  <==== do not use a power of 2 like I did
  sd length 1001m drive d1
  sd length 1001m drive d2

See the vinum documentation for more information.  The new volume device I am creating will be called test.  Also note that I am striping and specifying the same size (1001MB) for both drives.  This is despite that fact that one drive actually contains 1005 MB.  See below for more information on the error messages I encountered when I didn't specify the correct sizes.

man vinum also contains a "HOW TO SET UP VINUM" section which gives various options.  Please refer to that when deciding what to put into your configuration file.

Create the vinum devices
vinum uses it's own set of devices to implement what it needs to do what it does best.   These devices are located in /dev/vinum.  To create these devices, here's what I did:
# vinum create -f /etc/vinum.conf 
Configuration summary

Drives:         2 (4 configured)
Volumes:        1 (4 configured)
Plexes:         1 (8 configured)
Subdisks:       2 (16 configured)

D d1          State: up  Device /dev/da1e   Avail: 0/1001 MB (0%)
D d2          State: up  Device /dev/da2e   Avail: 4/1005 MB (0%)

V test        State: up  Plexes:       1 Size:       2002 MB

P test.p0   S State: up  Subdisks:     2 Size:       2002 MB

S test.p0.s0  State: up  PO:        0  B Size:       1001 MB
S test.p0.s1  State: up  PO:      256 kB Size:       1001 MB
Creating the new file system
The next step is to create the new file system.  That's really easy actually.
# newfs -v /dev/vinum/test
newfs: /dev/vinum/test: not a character-special device
/dev/vinum/test:    4100096 sectors in 1001 cylinders of 1 tracks,
4096 sectors
        2002.0MB in 63 cyl groups (16 c/g, 32.00MB/g, 7936 i/g)
super-block backups (for fsck -b #) at:
 32, 65568, 131104, 196640, 262176, 327712, 393248, 458784,
 524320, 589856, 655392, 720928, 786464, 852000, 917536, 983072,
 etc...

The -v option is very interesting:


-v      Specify that the disk does not contain any partitions, and that
        newfs should build a file system on the whole disk.  This option
        is useful for synthetic disks such as vinum.
Creation errors
If you don't change your fstype to vinum, you will get the following error when you try to create.
1: drive d1 device /dev/da1e
** 1 Drive d1 has invalid partition type: Inappropriate file 
                                 type or format
2: drive d2 device /dev/da2e
** 2 Drive d2 has invalid partition type: Inappropriate file 
                                 type or format

Always check /var/log/messages after doing a vinum create.  When I used the following configuration, I encountered my first problem:

sd length 1g drive d1
sd length 1g drive d2

But this configuration resulted in the following:

# vinum create -f /etc/vinum.conf 
   7:   sd length 1g drive d1
** 7 No space for test.p0.s0 on d1: No space left on device
   8:   sd length 1g drive d2
** 8 Unnamed sd is not associated with a plex: Invalid argument
Configuration summary

Drives:         2 (4 configured)
Volumes:        1 (4 configured)
Plexes:         1 (8 configured)
Subdisks:       0 (16 configured)

D d1       State: up   Device /dev/da1e  Avail: 1001/1001 MB (99%)
D d2       State: up   Device /dev/da2e  Avail: 1005/1005 MB (99%)

V test     State: down     Plexes:       1 Size:       1024 MB

P test.p0  S State: faulty   Subdisks:     1 Size:       1024 MB

S test.p0.s0 State: down     PO:        0  B Size:       1024 MB

The log messages for this are:

Aug  9 20:30:45 fred /kernel: vinum: drive d1 is up
Aug  9 20:30:45 fred /kernel: vinum: drive d2 is up
Aug  9 20:30:45 fred /kernel: vinum: plex test.p0 does not have at 
                                              least 2 subdisks
Aug  9 20:30:45 fred last message repeated 2 times
Aug  9 20:30:45 fred /kernel: vinum: test.p0 is faulty

So then I changed the configuration to:

sd length 1001m drive d1
sd length 1005m drive d2

And I tried again:

# vinum create -f /etc/vinum.conf 
Configuration summary

Drives:         2 (4 configured)
Volumes:        2 (4 configured)
Plexes:         2 (8 configured)
Subdisks:       2 (16 configured)

D d1           State: up   Device /dev/da1e Avail: 0/1001 MB (0%)
D d2           State: up   Device /dev/da2e Avail: 0/1005 MB (0%)

V test         State: down     Plexes:       2 Size:      2006 MB

P test.p0    S State: faulty   Subdisks:     1 Size:      1024 MB
P test.p1    S State: down     Subdisks:     2 Size:      2006 MB

S test.p0.s0   State: down     PO:        0  B Size:      1024 MB
S test.p1.s0   State: stale    PO:        0  B Size:      1001 MB
S test.p1.s1   State: stale    PO:      256 kB Size:      1005 MB

Well, that looks much better!  We're making progress.  At least, that's what I thought.  Note the faulty and down entries in the above.  And if you check the logs, you'll find.

Aug  9 20:34:28 fred /kernel: vinum: plex test.p0 does not have at
                                             least 2 subdisks
Aug  9 20:34:28 fred /kernel: vinum: test.p1 must have 
                                             equal sized subdisks
Aug  9 20:34:28 fred /kernel: vinum: test.p1 is down

And if you try the newfs, you'll get this:

# newfs -v /dev/vinum/test
newfs: /dev/vinum/test: Input/output error

So the moral of the story: always check /var/log/messages if encounter an unusual error.  And please note that after every failed vinum create, you must reset your vinum configuration.  See the next section for more detail on how to do that.

Vinum reset
One thing to remember is that you sometimes have to remove your existing vinum configuration.  If you encounter any errors during the create, you should clear out the vinum devices.  This is because each create is cumulative.  If youi keep doing creates, you will wind up with more and more devices.  Which is not necesarily what you want.

Please note that resetting the configuration does not delete your data but you must do another create with your original configuration.

Here's how I cleared out the vinum configuration:

# vinum resetconfig
 WARNING!  This command will completely wipe out your vinum 
                                                  configuration.
 All data will be lost.  If you really want to do this, enter 
                                                    the text

 NO FUTURE
 Enter text -> NO FUTURE
 Vinum configuration obliterated

Don't use resetconfig unless you want to lose everything. Instead, use rm.

Mounting the vinum volume
This too is quite easy.
# mount /dev/vinum/test /mnt

There you go!  Welcome to the vinum world!  And if I typed mount, here is what I found:

/dev/vinum/test on /mnt (local, writes: sync 2 async 1)

After all that, here is a set of successful logs from the above:

Aug  9 19:47:53 fred /kernel: vinum: drive d1 is up
Aug  9 19:47:53 fred /kernel: vinum: drive d2 is up
Aug  9 19:47:53 fred /kernel: vinum: test.p0.s0 is up
Aug  9 19:47:53 fred /kernel: vinum: test.p0.s1 is up
Aug  9 19:47:53 fred /kernel: vinum: test.p0 is up
Aug  9 19:47:53 fred /kernel: vinum: test is up
Aug  9 19:47:53 fred /kernel: vinum: test.p0 is up
Making it all automagical
This section shows how I made sure the vinum volume was always mounted when the system started up.

First, I found the vinum options which are recognized by /etc/rc.conf.

# grep vinum /etc/defaults/rc.conf
start_vinum=""  # set to YES to start vinum

Then I added the following to /etc/rc.conf.

start_vinum="YES"                  # set to YES to start vinum
vinum_drives="/dev/da1e /dev/da2e"
             # put in names of disks containing vinum drives
             # to start vinum on only specific disks.

Here's what I found in dmesg:

# dmesg | grep vinum
vinum: loaded
vinum: reading configuration from /dev/da2e
vinum: updating configuration from /dev/da1e

Need more help on this topic? Click here
This article has 12 comments
Show me similar articles