Venti on Linux via p9p

The plan is to backup arenas to DVD
DVD-Rs store 4707319808 bytes
Four arenas should be enough to start with

(when I posted this mycrotiv shoed me his venti setup script)

% echo '4 * 4707319808 / 1024' | hoc
18387968
% time dd if=/dev/zero of=arenas bs=1024 count=18387968 # stupid linux dd and its =
real 14m0.207s

venti(8) recommends an index size that is 5% of the active data log.

It doesn't describe the difference between the active data log and any other sort so I'll assume 5% of the arena size which in this case is
% echo '0.05 * 4 * 4707319808 / 1024' | hoc
919398.4
% dd if=/dev/zero of=isect bs=1024 count=919399

Seeing as I'm using it for backup all sorts of shizzle, I'm going to include the bloom filter.

The bloom filter thus has two parameters: nhash (maximum 32) and the total bitmap size (maximum 512MB, 232 bits). nhash × nblock <= 0.7 × b, where nblock is the expected number of blocks stored on the server and b is the bitmap size in bits.
b = 232, nash = 32
32 × nblock <= 0.7 × 232
nblock <= 0.7 × 232 / 32
nblock <= 5.075
so I guess nblock = 5, anyway says the maximum size is 512Mb
% echo '512 * 1024' | hoc
524288
% dd if=/dev/zero of=bloom bs=1024 count=524288

That's the storage reserved now for formatting

I'm going to round down the actual arena size a bit
% time venti/fmtareanas -a 4705000000 -Z arenas0. /home/venti/arenas # I dd'd in zero
fmtarenas /home/venti/arenas: 5 arenas, 18,828,484,608 bytes storage, 524,288 bytes for index map
real 0m1.312s
% time venti/fmtisect -Z isect0. /home/venti/isect
fmtisect /home/venti/isect: 114,827 buckets of 215 entries, 524,288 bytes for index map
real 0m0.513s
% time venti/fmtbloom -N 32 /home/venti/bloom
fmtbloom: using 512MB, 32 hashes/score, best up to 95,443,717 blocks
real 0m26.742s


Ok that looks like the formatting is done, set up venti.conf
% cat venti.conf
index main
isect /home/venti/isect
arenas /home/venti/arenas
bloom /home/venti/bloom
httpdaddr tcp!127.0.0.1!808
% time /usr/local/plan9/bin/venti/fmtindex venti.conf
fmtindex: 5 arenas, 114,827 index buckets, 18,828,402,688 bytes storage
real 0m0.483s


hmm, ok , lets run this sucker

% /usr/local/plan9/bin/venti/venti
2010/0127 21:12:22 venti: conf...httpd tcp!127.0.0.1!808...init...icache 0 bytes = 1,000 entries; 4 scache
sync...announce tcp!*!venti...serving.
% wget -q -O - http://127.0.0.1:808/storage
index=main
total arenas=5 active=0
total space=18,828,402,688 used=0
clumps=0 compressed clumps=0 data=0 compressed data=0

as usual, despressingly straight forward, I didn't even need a loopback

Restoring Venti from dumped arenas using Qemu

I've been using this script to dump my venti arenas from my laptop to RAID.

When I'm on the LAN I run this by hand as bootes

#!/bin/rc
test -e /n/daph/home || 9fs daph
bk = /n/daph/home/bootes/t23/arenas
if(! test -e $bk)
mkdir -p $bk
if(! test -e $bk)
status = $mk ^missing && exit
/usr/maht/bin/rc/vbackup /dev/sdC0 $bk

and I kept saying to myself - "I must do a test restore".

Naturally fate intervened and my HD decided that I didn't want to read my isect partiton after all and my hand was forced.

So after a couple of false starts trying to do it with the bootable CD I fired up Qemu with a blank 3G image - my arenas are 1.6Gb - bit of headroom, and installed fossil+venti.

I turned it into a cpu/auth with my script to make logging into it a bit easier

and then as bootes
% 9fs daph; cd /n/daph/arenas/
% venti/wrarena sdC0.arenas00
% venti/wrarena sdC0.arenas01
% time venti/wrarena sdC0.arenas02
1.67u 46.65s 496.71r
% ls -l sdC0.arenas02
--rw-rw-rw- M 85 bootes bootes 536870912 Dec 16 06:46 sdC0.arenas02

which seems easy enough, 1Mb sec

boot the laptop from the CDrom, edit plan9.ini to use my qemu is its venti (using -redir tcp:17034::17034

had to add
venti=/dev/sdC0/arenas tcp!*!17034
to qemu's plan9.ini

balls the qemu is sending a reset instead of data - I think there's a problem in there somewhere after a few different tries, fossil/flfmt dies when using my qemu venti as a host too


OK so with the Venti install I added a new disk - recover

My invocation for qemu venti now looks like this :
qemu.exe -L . -m 1024 -hda venti -hdb recover -localtime -M pc -redir tcp:3131::3131 -vnc :8 -snapshot -redir tcp:5356::5356 -redir tcp:17010::17010 -redir tcp:567::567 -redir tcp:17007::17007 -redir tcp:17034::17034

to get my data I did this as bootes


disk/mbr -m /386/mbr /dev/sdC1/data
disk/fdisk -baw /dev/sdC1/data
disk/prep -bw -a fossil /dev/sdC1/plan9
fossil/flfmt -v 22d...ff3 /dev/sdC1/fossil # 22d...ff3 is the score from fossil/last
fossil/conf -w /dev/sdC1/fossil <<EOF
fsys main config /dev/sdC1/fossil
fsys main open -AWP
fsys main
srv -p ricons
srv rico
EOF
fossil/fossil -f /dev/sdC1/fossil


drawtermed in again as me
mount /srv/rico /n/rico
cd /n/rico/usr/maht
tar cp . > /n/daph/home/maht/plan9_usr_maht.tar


You beauty!!