Index of /~jokey/squashed-portage-tree

 Name                    Last modified      Size  Description
 Parent Directory                             -   
 README.html             30-May-2007 17:38  1.8K  
 conf.d/                 11-Mar-2007 19:43    -   
 init.d/                 11-Mar-2007 19:43    -   

Slow rsync's and how to tweak it

Well yeah, rsync as such is fast, no argument.
Though the portage tree (and the following copy of the metadata cache)
takes a nice amount of
a) time to sync up and
b) space on disk due to about 150k files on disk
so let's tweak it a bit so we have it a bit faster.

Approach: Squashfs the tree and unionfs it for changes

There are already some hints out there to use this method though most of them
rebuild the whole tree each time, which is imho a bit senseless as squashfs
has an append feature.
But let's do the setup first and talk about open issues afterwards

I. support in kernel
Device Drivers -> Block Devices -> Loopback device support 
File systems  ->  Miscellaneous Filesystems  -> SquashFS 

Ia. external module
emerge unionfs

II. required utilities
emerge sys-fs/squashfs-tools

III. needed dir
mkdir -p /var/portage

IV. tweak make.conf for distfiles
echo DISTDIR=\"/var/portage/distfiles\" >> /etc/make.conf
echo PKGDIR=\"/var/portage/packages\" >> /etc/make.conf

V. move old stuff
mv /usr/portage/distfiles /var/portage
mv /usr/portage/packages /var/portage

VI. get the first snapshot up and kill old stuff
mksquashfs /usr/portage /var/portage/portage-$(date +%F).sqfs -check_data
ln -s /var/portage/portage-$(date +%F).sqfs /var/portage/portage.sqfs
rm -rf /usr/portage

VII. put files in this dir to /etc
cd /etc/conf.d
wget http://people.gentoo.org/jokey/squashed-portage-tree/conf.d/squash_portage
cd /etc/init.d
wget http://people.gentoo.org/jokey/squashed-portage-tree/init.d/squash_portage
chmod a+x squashed_portage

VIII. start it
/etc/init.d/squash_portage start

IX. add it to default runlevel
rc-update add squash_portage default

your /usr/portage should now be back

Current issues:
- you manually need to nuke older rebuilds