Re: создание/удаление миллиона файлов
- From
- Vlad Gnatov (2:5020/400)
- To
- Eugene Grosbein
- Date
- 2007-01-15T07:07:32Z
- Area
- RU.UNIX.BSD
From: Vlad Gnatov <stalker@rm-rf.kiev.ua>
Eugene Grosbein пишет:
EG>>> Сейчас запустил тот же самый скрипт на FreeBSD 6.2-PRE, Pentium-D
EG>>> 2x2.8Gb,
EG>>> 1Gb RAM, диск WDC 80G WD800JB-00FMA0 в режиме ATA100 (без TQ),
EG>>> noasync+softupdates, раздел UFS2, 22G с 2Gb и 2.5 миллионов inode
EG>>> свободных, ядро SMP, vfs.ufs.dirhash_maxmem: 8388608.
EG>>> Скорость создания - 666 файлов в секунду против 10000 на четверке.
EG>>> то есть 6% от прежней. Вот это прогресс...
VG>> И снова, шо я делаю нетак?
VG>> (Все тот же RELENG_6, x2 4200, 4G, sg 320G ST3320620A)
VG>> $mount |grep garbage3
VG>> /dev/ad8s1d on /mnt/garbage3 (ufs, local, with quotas, soft-updates)
EG> SATA? TQ работает?
Как я и предполагал, дело совсем не в железе.
Алокатор dirhash тупой. Т.к. dirhash при resize создается заново,
то начиная с некоторого размера линейное увеличение невыгодно.
С этим можно бороться как поправив алгоритм, так и более простым
хаком, увеличив expand ratio.
- --- ufs/ufs/ufs_dirhash.c
+++ ufs_dirhash.c.new
@@ -79,7 +79,9 @@
static int ufs_dirhashcheck = 0;
SYSCTL_INT(_vfs_ufs, OID_AUTO, dirhash_docheck, CTLFLAG_RW,
&ufs_dirhashcheck,
0, "enable extra sanity tests");
-
+static int ufs_dirhashexpandratio = 150; /* ($var * 3 + 1) / 2 */
+SYSCTL_INT(_vfs_ufs, OID_AUTO, dirhash_expandratio, CTLFLAG_RW,
&ufs_dirhashexpandratio,
+ 0, "dirhash expand ratio in percents");
static int ufsdirhash_hash(struct dirhash *dh, char *name, int namelen);
static void ufsdirhash_adjfree(struct dirhash *dh, doff_t offset, int diff);
@@ -150,15 +152,19 @@
if (ip->i_effnlink == 0)
return (-1);
+ /* if expandratio below minimal set him to default */
+ if (ufs_dirhashexpandratio < 110 )
+ ufs_dirhashexpandratio = 150;
+
vp = ip->i_vnode;
/* Allocate 50% more entries than this dir size could ever need. */
KASSERT(ip->i_size >= DIRBLKSIZ, ("ufsdirhash_build size"));
nslots = ip->i_size / DIRECTSIZ(1);
- nslots = (nslots * 4 + 1) / 2;
+ nslots = (nslots * ufs_dirhashexpandratio) / 100;
narrays = howmany(nslots, DH_NBLKOFF);
nslots = narrays * DH_NBLKOFF;
dirblocks = howmany(ip->i_size, DIRBLKSIZ);
- nblocks = (dirblocks * 4 + 1) / 2;
+ nblocks = (dirblocks * ufs_dirhashexpandratio) / 100;
memreqd = sizeof(*dh) + narrays * sizeof(*dh->dh_hash) +
narrays * DH_NBLKOFF * sizeof(**dh->dh_hash) +
default:
(нужно быть осторожным, играясь с expandratio: если следующая итерация
dirhash_mem невлезет в dirhash_maxmem, вместо улучшения получим
катастрофическое падения производительности)
$./testm
vfs.ufs.dirhash_maxmem: 15000000
vfs.ufs.dirhash_expandratio=150
creating...
created 20000: 1 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2142441
created 40000: 2 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2272652
created 60000: 3 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2388738
created 80000: 4 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2579625
created 100000: 5 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2563757
created 120000: 6 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2882232
created 140000: 7 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2882232
created 180000: 9 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3360971
created 200000: 10 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3360971
created 220000: 11 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3360971
created 240000: 12 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3360971
created 260000: 14 sec, 18571 files/sec, vfs.ufs.dirhash_mem: 4078567
created 280000: 15 sec, 18666 files/sec, vfs.ufs.dirhash_mem: 4078567
created 300000: 16 sec, 18750 files/sec, vfs.ufs.dirhash_mem: 4078567
created 320000: 17 sec, 18823 files/sec, vfs.ufs.dirhash_mem: 4078567
created 340000: 18 sec, 18888 files/sec, vfs.ufs.dirhash_mem: 4078567
created 360000: 19 sec, 18947 files/sec, vfs.ufs.dirhash_mem: 4078567
created 380000: 23 sec, 16521 files/sec, vfs.ufs.dirhash_mem: 5154961
created 400000: 24 sec, 16666 files/sec, vfs.ufs.dirhash_mem: 5154961
created 420000: 25 sec, 16800 files/sec, vfs.ufs.dirhash_mem: 5154961
created 440000: 25 sec, 17600 files/sec, vfs.ufs.dirhash_mem: 5154961
created 460000: 26 sec, 17692 files/sec, vfs.ufs.dirhash_mem: 5154961
created 480000: 27 sec, 17777 files/sec, vfs.ufs.dirhash_mem: 5154961
created 500000: 28 sec, 17857 files/sec, vfs.ufs.dirhash_mem: 5158421
created 520000: 29 sec, 17931 files/sec, vfs.ufs.dirhash_mem: 5158421
created 540000: 30 sec, 18000 files/sec, vfs.ufs.dirhash_mem: 5158421
created 560000: 37 sec, 15135 files/sec, vfs.ufs.dirhash_mem: 6772498
created 580000: 38 sec, 15263 files/sec, vfs.ufs.dirhash_mem: 6772498
created 600000: 39 sec, 15384 files/sec, vfs.ufs.dirhash_mem: 6772498
created 620000: 40 sec, 15500 files/sec, vfs.ufs.dirhash_mem: 6772498
created 640000: 40 sec, 16000 files/sec, vfs.ufs.dirhash_mem: 6772498
created 660000: 41 sec, 16097 files/sec, vfs.ufs.dirhash_mem: 6772498
created 680000: 42 sec, 16190 files/sec, vfs.ufs.dirhash_mem: 6772498
created 700000: 43 sec, 16279 files/sec, vfs.ufs.dirhash_mem: 6772498
created 720000: 44 sec, 16363 files/sec, vfs.ufs.dirhash_mem: 6772498
created 740000: 45 sec, 16444 files/sec, vfs.ufs.dirhash_mem: 6772498
created 760000: 46 sec, 16521 files/sec, vfs.ufs.dirhash_mem: 6772498
created 780000: 47 sec, 16595 files/sec, vfs.ufs.dirhash_mem: 6772498
created 800000: 48 sec, 16666 files/sec, vfs.ufs.dirhash_mem: 6772498
created 820000: 61 sec, 13442 files/sec, vfs.ufs.dirhash_mem: 9195155
created 840000: 62 sec, 13548 files/sec, vfs.ufs.dirhash_mem: 9195155
created 860000: 63 sec, 13650 files/sec, vfs.ufs.dirhash_mem: 9195155
created 880000: 64 sec, 13750 files/sec, vfs.ufs.dirhash_mem: 9195155
created 900000: 65 sec, 13846 files/sec, vfs.ufs.dirhash_mem: 9195155
created 920000: 66 sec, 13939 files/sec, vfs.ufs.dirhash_mem: 9195155
created 940000: 67 sec, 14029 files/sec, vfs.ufs.dirhash_mem: 9195155
created 960000: 68 sec, 14117 files/sec, vfs.ufs.dirhash_mem: 9195155
created 980000: 69 sec, 14202 files/sec, vfs.ufs.dirhash_mem: 9195155
created 1000000: 70 sec, 14285 files/sec, vfs.ufs.dirhash_mem: 9195155
removing...removed: 46 sec, 21739 files/sec
$./testm
vfs.ufs.dirhash_maxmem: 15000000
vfs.ufs.dirhash_expandratio=200
creating...
created 20000: 1 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 1688793
created 40000: 2 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 1821145
created 60000: 3 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2085849
created 80000: 4 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2085849
created 100000: 5 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2615257
created 120000: 6 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2615257
created 140000: 7 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2615257
created 160000: 8 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 2615257
created 180000: 8 sec, 22500 files/sec, vfs.ufs.dirhash_mem: 2615257
created 200000: 10 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3674073
created 220000: 11 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3674073
created 240000: 12 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3674073
created 280000: 14 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3674073
created 300000: 15 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3674073
created 320000: 16 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3674073
created 340000: 17 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3674073
created 360000: 18 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 3674073
created 380000: 21 sec, 18095 files/sec, vfs.ufs.dirhash_mem: 5791705
created 400000: 22 sec, 18181 files/sec, vfs.ufs.dirhash_mem: 5791705
created 420000: 23 sec, 18260 files/sec, vfs.ufs.dirhash_mem: 5791705
created 440000: 24 sec, 18333 files/sec, vfs.ufs.dirhash_mem: 5791705
created 460000: 25 sec, 18400 files/sec, vfs.ufs.dirhash_mem: 5791705
created 480000: 25 sec, 19200 files/sec, vfs.ufs.dirhash_mem: 5791705
created 500000: 26 sec, 19230 files/sec, vfs.ufs.dirhash_mem: 5791705
created 520000: 27 sec, 19259 files/sec, vfs.ufs.dirhash_mem: 5791705
created 540000: 28 sec, 19285 files/sec, vfs.ufs.dirhash_mem: 5791705
created 560000: 29 sec, 19310 files/sec, vfs.ufs.dirhash_mem: 5791705
created 580000: 30 sec, 19333 files/sec, vfs.ufs.dirhash_mem: 5791705
created 600000: 31 sec, 19354 files/sec, vfs.ufs.dirhash_mem: 5791705
created 620000: 32 sec, 19375 files/sec, vfs.ufs.dirhash_mem: 5791705
created 640000: 33 sec, 19393 files/sec, vfs.ufs.dirhash_mem: 5791705
created 660000: 34 sec, 19411 files/sec, vfs.ufs.dirhash_mem: 5791705
created 680000: 35 sec, 19428 files/sec, vfs.ufs.dirhash_mem: 5791705
created 700000: 36 sec, 19444 files/sec, vfs.ufs.dirhash_mem: 5791705
created 720000: 47 sec, 15319 files/sec, vfs.ufs.dirhash_mem: 10061489
created 740000: 48 sec, 15416 files/sec, vfs.ufs.dirhash_mem: 10061489
created 760000: 49 sec, 15510 files/sec, vfs.ufs.dirhash_mem: 10061489
created 780000: 50 sec, 15600 files/sec, vfs.ufs.dirhash_mem: 10061489
created 800000: 51 sec, 15686 files/sec, vfs.ufs.dirhash_mem: 10061489
created 820000: 52 sec, 15769 files/sec, vfs.ufs.dirhash_mem: 10061489
created 840000: 54 sec, 15555 files/sec, vfs.ufs.dirhash_mem: 10061489
created 860000: 55 sec, 15636 files/sec, vfs.ufs.dirhash_mem: 10061489
created 880000: 56 sec, 15714 files/sec, vfs.ufs.dirhash_mem: 10061489
created 900000: 57 sec, 15789 files/sec, vfs.ufs.dirhash_mem: 10061489
created 920000: 58 sec, 15862 files/sec, vfs.ufs.dirhash_mem: 10061489
created 940000: 58 sec, 16206 files/sec, vfs.ufs.dirhash_mem: 10061489
created 960000: 59 sec, 16271 files/sec, vfs.ufs.dirhash_mem: 10061489
created 980000: 60 sec, 16333 files/sec, vfs.ufs.dirhash_mem: 10061489
created 1000000: 61 sec, 16393 files/sec, vfs.ufs.dirhash_mem: 10061489
removing...removed: 44 sec, 22727 files/sec
$./testm
vfs.ufs.dirhash_maxmem: 15000000
vfs.ufs.dirhash_expandratio=300
creating...
created 20000: 1 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 1740061
created 40000: 2 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 1740061
created 60000: 2 sec, 30000 files/sec, vfs.ufs.dirhash_mem: 2186749
created 80000: 3 sec, 26666 files/sec, vfs.ufs.dirhash_mem: 2186749
created 100000: 4 sec, 25000 files/sec, vfs.ufs.dirhash_mem: 2186749
created 120000: 5 sec, 24000 files/sec, vfs.ufs.dirhash_mem: 2186749
created 140000: 6 sec, 23333 files/sec, vfs.ufs.dirhash_mem: 3526813
created 160000: 7 sec, 22857 files/sec, vfs.ufs.dirhash_mem: 3526813
created 180000: 8 sec, 22500 files/sec, vfs.ufs.dirhash_mem: 3526813
created 200000: 9 sec, 22222 files/sec, vfs.ufs.dirhash_mem: 3526813
created 220000: 10 sec, 22000 files/sec, vfs.ufs.dirhash_mem: 3526813
created 240000: 11 sec, 21818 files/sec, vfs.ufs.dirhash_mem: 3526813
created 260000: 12 sec, 21666 files/sec, vfs.ufs.dirhash_mem: 3526813
created 280000: 12 sec, 23333 files/sec, vfs.ufs.dirhash_mem: 3526813
created 300000: 13 sec, 23076 files/sec, vfs.ufs.dirhash_mem: 3526813
created 320000: 14 sec, 22857 files/sec, vfs.ufs.dirhash_mem: 3526813
created 340000: 15 sec, 22666 files/sec, vfs.ufs.dirhash_mem: 3526813
created 360000: 18 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 380000: 19 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 400000: 20 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 420000: 21 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 440000: 22 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 460000: 23 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 480000: 24 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 500000: 25 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 520000: 26 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 540000: 27 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 560000: 28 sec, 20000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 580000: 28 sec, 20714 files/sec, vfs.ufs.dirhash_mem: 7548036
created 600000: 29 sec, 20689 files/sec, vfs.ufs.dirhash_mem: 7548036
created 620000: 30 sec, 20666 files/sec, vfs.ufs.dirhash_mem: 7548036
created 640000: 31 sec, 20645 files/sec, vfs.ufs.dirhash_mem: 7548036
created 660000: 32 sec, 20625 files/sec, vfs.ufs.dirhash_mem: 7548036
created 680000: 33 sec, 20606 files/sec, vfs.ufs.dirhash_mem: 7548036
created 700000: 34 sec, 20588 files/sec, vfs.ufs.dirhash_mem: 7548036
created 720000: 35 sec, 20571 files/sec, vfs.ufs.dirhash_mem: 7548036
created 740000: 36 sec, 20555 files/sec, vfs.ufs.dirhash_mem: 7548036
created 760000: 37 sec, 20540 files/sec, vfs.ufs.dirhash_mem: 7548036
created 780000: 37 sec, 21081 files/sec, vfs.ufs.dirhash_mem: 7548036
created 800000: 38 sec, 21052 files/sec, vfs.ufs.dirhash_mem: 7548036
created 820000: 40 sec, 20500 files/sec, vfs.ufs.dirhash_mem: 7548036
created 840000: 40 sec, 21000 files/sec, vfs.ufs.dirhash_mem: 7548036
created 860000: 42 sec, 20476 files/sec, vfs.ufs.dirhash_mem: 7548036
created 880000: 43 sec, 20465 files/sec, vfs.ufs.dirhash_mem: 7548036
created 900000: 44 sec, 20454 files/sec, vfs.ufs.dirhash_mem: 7548036
created 920000: 45 sec, 20444 files/sec, vfs.ufs.dirhash_mem: 7548036
created 940000: 46 sec, 20434 files/sec, vfs.ufs.dirhash_mem: 7548036
created 960000: 47 sec, 20425 files/sec, vfs.ufs.dirhash_mem: 7548036
created 980000: 48 sec, 20416 files/sec, vfs.ufs.dirhash_mem: 7548036
created 1000000: 49 sec, 20408 files/sec, vfs.ufs.dirhash_mem: 7548036
removing...removed: 45 sec, 22222 files/sec
--
Отправлено через сервер Форумы@mail.ru - http://talk.mail.ru
--- ifmail v.2.15dev5.3
* Origin: Talk.ru (2:5020/400)
SEEN-BY: 50/12 400/814 450/159 1024 461/43 132 640 469/999 4616/3 4625/8
SEEN-BY: 4641/444 5000/76 5000 5006/1 5007/1 5010/70 5011/13 5012/46 5015/28
SEEN-BY: 5019/26 5020/18 175 194 400 545 982 1057 1909 1922 2238 2395 2871
SEEN-BY: 5020/4441 5021/29 5025/3 5026/14 45 5027/12 5030/1080 1957 5034/10 13
SEEN-BY: 5035/38 5036/1 5045/7 5049/1 5051/15 5054/1 4 8 9 11 28 35 36 37 45
SEEN-BY: 5054/66 67 70 75 84 85 5059/9 5060/88 5061/15 5062/10 5063/3 5064/7
SEEN-BY: 5066/18 5075/5 5076/1 5077/70 5084/9 5085/13 5095/20 5096/18 6001/10
PATH: 5020/400 545 5054/1 37