solaris service contract
- From
- Vova Uralsky (2:5030/257)
- To
- Andy Igoshin
- Date
- 2009-02-23T22:03:28Z
- Area
- RU.UNIX.SOLARIS
Hi!
Что-то я номера патча не вижу....
Document Audience: SPECTRUM
Document ID: 6579274
Title: Share memory unmap stays in infinite loop (32bit intel)
Copyright Notice: Copyright (c) 2009 Sun Microsystems, Inc. All Rights Reserved
Update Date: Thu Feb 05 00:00:00 MST 2009
Bug ID: 6579274
Synopsis: Share memory unmap stays in infinite loop (32bit intel)
Category: kernel
Subcategory: vm
State: 10-Fix Delivered
Description:
PostgreSQL initdb command invokes kernel hang. Kernel stays in infinite loop in hat_unshare function. See stack trace. It is reproducible on 32bit kernel on intel machine (Sony Vaio) and Nevada build68.
d4176e48 mutex_enter+0x11(d3ed3b90)
d4176e7c hat_unshare+0x8e(d9b13d88, d0800000, 800000, 1)
d4176ea0 segspt_shmunmap+0x77(d40e6468, d0800000, 800000)
d4176ef0 as_unmap+0xe3(d3c0ed28, d0800000, 800000)
d4176f24 shm_detach+0x4d(d3c67118, d40f5c80)
d4176f64 shmdt+0x8f(d0800000)
d4176f84 shmsys+0x58()
d4176fac sys_sysenter+0x106()
Date Modified: 2007-07-11 15:35:53 GMT+00:00
Reproduce scenario:
Run as normal user (no root) following command:
initdb /var/tmp/test
it hangs on following line:
selecting default max_connections ...
Date Modified: 2007-07-11 15:59:01 GMT+00:00
I reproduced this on a lab machine and indeed, we are stuck in an infinite loop. I set breakpoints to see the parameters to htable_lookup, and here's what we got:
ca116e30 htable_lookup+3(cf36fa88, c0000000, 1)
ca116e6c hat_unshare+0x86(cf36fa88, c4400000, 800000, 1)
ca116e90 segspt_shmunmap+0x8d(c9dbe340, c4400000, 800000)
ca116ee0 as_unmap+0xf9(c7a73a20, c4400000, 800000)
ca116f14 shmsys`shm_detach+0x4d(c9f02c58, caeeacc8)
ca116f54 shmsys`shmdt+0x8f(c4400000)
ca116f7c shmsys`shmsys+0x58()
ca116fa4 sys_sysenter+0x1a4()
ca116e30 htable_lookup+3(cf36fa88, 0, 1)
ca116e6c hat_unshare+0x86(cf36fa88, c4400000, 800000, 1)
ca116e90 segspt_shmunmap+0x8d(c9dbe340, c4400000, 800000)
ca116ee0 as_unmap+0xf9(c7a73a20, c4400000, 800000)
ca116f14 shmsys`shm_detach+0x4d(c9f02c58, caeeacc8)
ca116f54 shmsys`shmdt+0x8f(c4400000)
ca116f7c shmsys`shmsys+0x58()
ca116fa4 sys_sysenter+0x1a4()
ca116e30 htable_lookup+3(cf36fa88, 40000000, 1)
ca116e6c hat_unshare+0x86(cf36fa88, c4400000, 800000, 1)
ca116e90 segspt_shmunmap+0x8d(c9dbe340, c4400000, 800000)
ca116ee0 as_unmap+0xf9(c7a73a20, c4400000, 800000)
ca116f14 shmsys`shm_detach+0x4d(c9f02c58, caeeacc8)
ca116f54 shmsys`shmdt+0x8f(c4400000)
ca116f7c shmsys`shmsys+0x58()
ca116fa4 sys_sysenter+0x1a4()
ca116e30 htable_lookup+3(cf36fa88, 80000000, 1)
ca116e6c hat_unshare+0x86(cf36fa88, c4400000, 800000, 1)
ca116e90 segspt_shmunmap+0x8d(c9dbe340, c4400000, 800000)
ca116ee0 as_unmap+0xf9(c7a73a20, c4400000, 800000)
ca116f14 shmsys`shm_detach+0x4d(c9f02c58, caeeacc8)
ca116f54 shmsys`shmdt+0x8f(c4400000)
ca116f7c shmsys`shmsys+0x58()
ca116fa4 sys_sysenter+0x1a4()
htable_lookup+3(cf36fa88, c0000000, 1)
hat_unshare+0x86(cf36fa88, c4400000, 800000, 1)
segspt_shmunmap+0x8d(c9dbe340, c4400000, 800000)
as_unmap+0xf9(c7a73a20, c4400000, 800000)
shmsys`shm_detach+0x4d(c9f02c58, caeeacc8)
shmsys`shmdt+0x8f(c4400000)
shmsys`shmsys+0x58()
sys_sysenter+0x1a4()
htable_lookup+3(cf36fa88, 0, 1)
hat_unshare+0x86(cf36fa88, c4400000, 800000, 1)
segspt_shmunmap+0x8d(c9dbe340, c4400000, 800000)
as_unmap+0xf9(c7a73a20, c4400000, 800000)
shmsys`shm_detach+0x4d(c9f02c58, caeeacc8)
shmsys`shmdt+0x8f(c4400000)
shmsys`shmsys+0x58()
sys_sysenter+0x1a4()
So as you can see from the above, we are calling htable lookup for level 1 with addresses of 0, 40000000, 80000000, c0000000, repeat endlessly.
Looking at the code in hat_unshare, we see the following logic that we are stuck in:
hat_unshare(hat_t *hat, caddr_t addr, size_t len, uint_t ismszc)
l = mmu.max_page_level;
if (l == mmu.max_level)
--l;
for (; l >= 0; --l) {
for (vaddr = (uintptr_t)addr; vaddr < eaddr;
vaddr = (vaddr & LEVEL_MASK(l + 1)) + LEVEL_SIZE(l + 1)) {
ASSERT(!IN_VA_HOLE(vaddr));
/*
* find a pagetable that maps the current address
*/
ht = htable_lookup(hat, vaddr, l);
if (ht == NULL)
continue;
if (ht->ht_flags & HTABLE_SHARED_PFN) {
/*
* clear page count, set valid_cnt to 0,
* let htable_release() finish the job
*/
hat->hat_ism_pgcnt -= ht->ht_valid_cnt <<
(LEVEL_SHIFT(ht->ht_level) - MMU_PAGESHIFT);
ht->ht_valid_cnt = 0;
need_demaps = 1;
}
htable_release(ht);
}
}
Our parameters to hat unshare:
hat_unshare(cf36fa88, c4400000, 800000, 1)
So, in the above logic, vaddr will start at c4400000 and eaddr will
be c4400000 + 800000 = c4c00000
So, htable_lookup has returned NULL we continue to the top and we align vaddr to 1G and then add 1G to it. And thus the spin. Now the question is, why did htable_lookup return NULL for an ISM segment? I believe all of the pages in an ISM segment should be mapped. Let's take a closer look at the process:
[0]> c9f02c58::ps -t
S PID PPID PGID SID UID FLAGS ADDR NAME
R 100830 100829 100826 100768 78595 0x4a004000 c9f02c58 postgres
T 0xc7d21220 <TS_ONPROC>
Looking at it's segments, we see our ism segment below:
[0]> c7a73a4c::walk avl | ::seg
SEG BASE SIZE DATA OPS
c9dbed80 8041000 7000 cd0dad48 segvn_ops
cc89b000 8050000 263000 cc89a1e0 segvn_ops
c9dbee40 82c2000 23000 cd0dae98 segvn_ops
c9dbedc0 82e5000 a1000 cd0dadb8 segvn_ops
c9dbe340 c4400000 800000 caee63f8 segspt_shmops
c9dbe900 c4c70000 10000 cd0da568 segvn_ops
cc89ea80 c4c90000 6000 cc89f0e8 segvn_ops
c9dbeec0 c4ca0000 6000 cd0daf78 segvn_ops
Let's look at the hat:
[0]> cf36fa88::print struct hat
{
hat_mutex = {
_opaque = [ 0, 0xbaddcafe ]
}
hat_as = 0xc7a73a20
hat_stats = 0
hat_pages_mapped = [ 0x386, 0x3 ]
hat_ism_pgcnt = 0
hat_cpus = 0x1
hat_flags = 0x2
hat_htable = 0xca396940
hat_next = 0xc9b5a780
hat_prev = 0xcf36f698
hat_num_hash = 0x100
hat_ht_hash = 0xc8e578b0
hat_ht_cached = 0
hat_vlp_ptes = [ 0x2829d001, 0, 0, 0x280dc001 ]
}
hat_ism_pgcnt is set to zero so we succesfully removed all of the ISM mappings :)
I'm going to guess that this is a side effect of some recent work which allowed ISM to share 2M pages. I'll get the author of that logic to take a look.
Date Modified: 2007-07-12 19:22:56 GMT+00:00
Work Around:
Suggested Fix:
Evaluation:
Fixed by patch:
Integrated in Build: snv_70
Duplicate of:
Related Change Request(s):
6529185 6770466 6782198
Date Modified: 2009-02-05 10:46:55 GMT+00:00
Public Summary:
Andy Igoshin -> All schrieb:
AI> From: Andy Igoshin <ai@vsu.ru>
AI> Hi!
AI> может у кого есть сервис-контракт на Solaris 10 x86?
AI> нужен вот такой патч:
AI> http://search.sun.com/main/index.jsp?qt=6579274&col=main-support-sunsolve
AI> С уважением,
--- Thunderbird 2.0.0.19 (Macintosh/20081209)
* Origin: Permission denied (2:5030/257)
SEEN-BY: 46/50 236/100 450/186 1024 451/30 452/25 461/1313 463/68 550/196
SEEN-BY: 5000/5000 5001/100 5001 5004/58 5011/13 5012/30 46 5015/28 5019/40
SEEN-BY: 5020/175 400 545 715 758 830 845 1042 1604 1641 2238 4441 5021/29 44
SEEN-BY: 5022/128 5023/11 5025/3 5029/60 5030/7 55 61 87 97 115 172 251 254
SEEN-BY: 5030/257 266 338 448 452 468 580 585 738 786 830 845 901 918 920 966
SEEN-BY: 5030/1014 1023 1115 1256 1340 1763 1900 1957 5035/38 5038/9 5045/7
SEEN-BY: 5052/4 5054/1 4 8 9 28 37 5058/1 42 5061/26 120 5062/10 5064/7
SEEN-BY: 5070/156 5071/166 5075/35 5080/68 80 111 1003 5082/6 5083/1 5085/13
SEEN-BY: 5085/87 5095/20 5096/18 5098/11 6001/10 6055/2 6083/11 12
PATH: 5030/257 1115 115 5080/1003 5020/4441 545 5054/1 37