Openmoko Bug #2309: hangs during resume after printing GSTATUS4

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
-----------------------------+----------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: System Software | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
-----------------------------+----------------------------------------------
Steps to reproduce (occurs very rarely!):
1) suspend the device
2) press power button

Expected results:
2) phone wakes up

Actual results:
2) nothing visible happens

More info:
1) kernel: andy-tracking 4e418ad2a6bae968 with two patches:
1.1)
http://docs.openmoko.org/trac/attachment/ticket/2277/wifi_ifupdown.patch
1.2)
http://docs.openmoko.org/trac/attachment/ticket/2135/ramconsole-2.patch
2) boot loader: Qi 9ef7754b8243457c
3) hardware: gta02V5
4) Xorg driver: xserver-xorg-video-fbdev
4) after step 2) I called myself from another phone
5) I then removed the battery and caller was immediately notified
"User is temporarily unavailable." -- This should prove that the GSM modem
was alive and lost power only when I finally removed the battery
6) I recovered kernel log buffer from memory and noticed that the last
messages printed by the kernel were

<7>[46635.530000] Leaving IRQ 49 (pin 165) enAbled
<7>[46635.530000] Disabling IRQ 50 (pin 166)
<7>[46635.530000] Disabling IRQ 51 (pin 167)
<7>[46635.530000] Leaving IRQ 53 (pin 193) enabled
<7>[46635.530000] Disabling IRQ 56 (pin 196)
<7>[46635.530000] sleep: irq wakeup masks: fffffffd,fffffdcf
<7>[46635.530000] GSTATUS3 0x3038ca5c
<7>[46635.530000] GSTATUS4 0x00000200

NOTE: RAM contents was partially corrupted when I removed the battery, you
should not trust the output to be 100% correct.

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
-----------------------------+----------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: System Software | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
-----------------------------+----------------------------------------------

Comment(by lindi):

I hit this again. This time I connected the phone via USB to laptop and
removed the battery with the hope that I could measure USB power
consumption with laptop. However, I heard maybe a really low frequency
sound from the phone when I removed the battery. I put the battery back
and noticed that the device was booting.

Since the battery was removed for a while the ramconsole output got really
garbled:

{{{
00076680 48 33 61 62 6c 69 6a 67 20 48 52 11 20 35 22 20 |H3ablijg HR.
5" |
00076690 28 10 29 6e 20 31 19 36 29 02 3c 33 3a 4b 33 16 |(.)n
1.6).<3:K3.|
000766a0 30 11 36 31 2e 10 30 31 20 30 30 5c 20 63 44 25 |0.61..01 00\
cD%|
000766b0 65 50 2a 20 61 32 71 20 73 60 42 65 75 50 00 69 |eP* a2q
s`BeuP.i|
000766c0 21 73 6b 72 3a 00 62 66 26 66 66 64 66 64 2c 24
|!skr:.bf&ffdfd,$|
000766d0 64 66 64 66 24 42 22 0a 3c 32 3e 59 33 17 18 31
|dfdf$B".<2>Y3..1|
000766e0 26 30 04 31 30 34 30 30 00 55 20 47 02 54 01 10 |&0.10400.U
G.T..|
000766f0 45 52 12 20 00 70 13 30 23 38 21 60 45 23 0a 3c |ER.
.p.0#8!`E#.<|
00076700 07 1e 42 33 17 30 30 36 30 22 11 30 11 30 30 30
|..B3.0060".0.000|
00076710 5d 20 07 13 14 41 54 51 51 34 00 10 38 10 30 30 |] ...AT
}}}

However, one can easily see patterns like "Disabling IRQ 56 (pin 196)" and
"sleep: irq wakeup masks: fffffffd,fffffdcf".

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
-----------------------------+----------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: System Software | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
-----------------------------+----------------------------------------------

Comment(by lindi):

This time the kernel was andy-tracking a3587e4ed77974ad with ramconsole
patch and Qi 9ef7754b8243457c. I still don't have a clue on how to
reproduce the bug.

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
-----------------------------+----------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: System Software | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
-----------------------------+----------------------------------------------

Comment(by RuiSeabra):

I think this sometimes happens to me as well. I've got current and updated
shr-unstable, with Xglamo. I'd point towards kernel or bootloader thing as
Xglamo is neither xorg nor xorg with fbdev.

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
-----------------------------+----------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: System Software | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
-----------------------------+----------------------------------------------

Comment(by lindi):

Just for the record, I hit this bug again on Friday. Still andy-tracking
a3587e4ed77974ad and only xserver-xorg-video-fbdev. I was using gprs and
put the phone to suspend and to my pocket. After a few hours I tried to
wake it up but it did not respond in any way.

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
-----------------------------+----------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: System Software | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
-----------------------------+----------------------------------------------

Comment(by lindi):

Just for the record, I hit this bug again today. Still andy-tracking
a3587e4ed77974ad.

Ramconsole output:

{{{
000006d0 30 58 20 73 61 76 65 64 20 46 34 32 30 30 30 12 |0X saved
F42000.|
000006e0 34 20 76 21 4c 35 60 20 20 30 39 65 30 33 61 24 |4 v!L5`
09e03a$|
000006f0 0a 3c 37 3e 5b 12 31 10 30 30 35 2e 31 35 35 10
|.<7>[.1.005.155.|
00000700 30 30 5d 20 53 61 76 65 64 20 64 30 31 30 10 10 |00] Saved
d010..|
00000710 20 28 20 76 61 6c 75 65 20 30 30 30 35 38 20 34 | ( value
00058 4|
00000720 32 02 3c 37 3e 53 32 31 30 30 30 35 2e 39 35 34
|2.<7>S210005.954|
00000730 30 10 30 4d 20 63 61 72 65 40 20 66 30 31 30 30 |0.0M care@
f0100|
00000740 30 31 30 20 76 61 6c 75 60 20 30 30 30 30 30 20 |010 valu`
00000 |
00000750 30 34 0a 3c 36 3e 5b 32 31 30 30 30 31 2a 29 35
|04.<6>[210001*)5|
00000760 31 30 30 20 5d 20 44 69 53 61 62 6c 68 4e 67 20 |100 ]
DiSablhNg |
00000770 49 52 51 20 31 36 20 28 70 69 6a 20 31 36 30 29 |IRQ 16 (pij
160)|
00000780 0a 3c 36 3c 5b 30 31 30 30 30 35 2e 39 35 35 30
|.<6<[010005.9550|
00000790 30 30 5c 00 44 65 61 76 69 6a 67 20 09 52 51 20 |00\.Deavijg
.RQ |
000007a0 31 35 00 28 70 69 6e 00 31 16 31 29 20 65 6e 61 |15.(pin.1.1)
ena|
000007b0 62 6c 65 64 0a 3c 37 3e 5b 30 31 30 30 30 35 2e
|bled.<7>[010005.|
000007c0 39 35 35 30 10 30 5d 20 44 69 73 61 60 6c 69 6e |9550.0]
Disa`lin|
000007d0 65 20 49 12 11 00 31 38 20 28 70 68 2e 20 11 36 |e I...18 (ph.
.6|
000007e0 22 29 02 3c 25 3e 5b 22 30 30 30 30 35 2a 39 15
|").<%>["00005*9.|
000007f0 34 30 30 30 5c 20 44 69 51 61 62 6c 29 4e 67 20 |4000\
DiQabl)Ng |
00000800 49 52 51 20 31 29 20 28 70 69 6e 20 31 36 31 29 |IRQ 1) (pin
161)|
00000810 0a 2c 27 3e 5b 30 21 30 30 30 35 2e 39 34 35 30
|.,'>[0!0005.9450|
00000820 20 30 59 00 48 65 61 66 69 6e 63 20 49 52 51 20 | 0Y.Heafinc
IRQ |
00000830 34 38 00 20 70 69 6e 20 31 36 14 28 20 65 6e 61 |48. pin 16.(
ena|
00000840 62 6c 64 64 0a 3c 37 3a 5b 32 31 30 30 30 35 2e
|bldd.<7:[210005.|
00000850 39 35 11 20 30 30 5d 20 4c 64 61 74 60 6e 47 20 |95. 00]
Ldat`nG |
00000860 49 42 51 20 34 31 20 28 70 29 6e 20 20 36 35 29 |IBQ 41 (p)n
65)|
00000870 20 65 4e 60 42 6c 24 64 0a 3c 37 3e 5b 32 31 30 |
eN`Bl$d.<7>[210|
00000880 30 20 35 2c 21 15 34 20 30 30 5d 20 44 69 73 61 |0 5,!.4 00]
Disa|
00000890 62 6c 68 6e 27 20 49 52 50 20 35 20 20 28 70 69 |blhn' IRP 5
(pi|
000008a0 4e 20 31 36 36 29 0a 3c 15 32 5b 22 21 30 30 10 |N
166).<.2["!00.|
000008b0 35 2e 39 35 35 30 30 30 5d 20 44 49 33 61 62 6c |5.955000]
DI3abl|
000008c0 69 6e 67 20 49 52 41 20 35 31 20 28 50 69 4e 20 |ing IRA 51
(PiN |
000008d0 31 36 37 28 0a 3c 33 3e 5b 32 31 30 20 30 15 0e |167(.<3>[210
0..|
000008e0 38 35 35 10 30 30 5d 20 4c 65 61 76 29 4e 67 20 |855.00]
Leav)Ng |
000008f0 49 52 51 00 34 33 20 28 70 69 6e 20 30 39 33 21 |IRQ.43 (pin
093!|
00000900 20 65 0e 61 42 6c 65 64 0a 3c 36 3e 59 32 31 30 |
e.aBled.<6>Y210|
00000910 30 30 35 2e 29 30 25 30 30 20 5d 20 44 49 73 40 |005.)0%00 ]
DIs@|
00000920 62 2c 68 6e 67 00 49 52 51 20 35 36 20 28 70 69 |b,hng.IRQ 56
(pi|
00000930 4e 20 31 39 36 09 0a 3c 37 3e 5b 32 31 30 30 30 |N
196..<7>[21000|
00000940 35 2e 39 21 25 30 30 30 5d 20 73 6c 65 65 70 3a |5.9!%000]
sleep:|
00000950 20 69 72 31 20 63 20 6b 05 75 70 20 6d 61 62 6b | ir1 c k.up
mabk|
00000960 53 2a 00 66 66 66 66 66 66 44 64 2c 66 66 66 66
|S*.ffffffDd,ffff|
00000970 66 24 23 66 02 3c 37 3e 5b 32 30 30 30 30 25 2e
|f$#f.<7>[20000%.|
00000980 19 35 35 30 30 30 4d 20 47 53 54 41 44 55 53 33 |.55000M
GSTADUS3|
00000990 20 30 78 33 30 22 36 63 65 32 30 0a 3c 36 3e 1b |
0x30"6ce20.<6>.|
000009a0 32 30 30 20 20 35 2e 39 35 34 30 30 20 1c 20 47 |200 5.95400
. G|
000009b0 53 54 41 54 51 13 34 20 10 58 30 30 30 10 30 32 |STATQ.4
.X000.02|
000009c0 30 32 00 20 74 70 75 6e 61 60 74 65 64 00 74 6f |02.
tpuna`ted.to|
000009d0 20 33 30 20 63 48 61 72 61 63 74 61 72 73 2e 0a | 30
cHaractars..|
}}}

(The alsa message on truncated names is an older message was just about to
get overwritten in the ringbuffer.)

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

Just for the record I hit this again on 2010-04-15. The previous
occurences have been on

{{{
2009-08-27
2009-10-25
2009-12-18
2010-02-15
2010-04-15
}}}

Is it just pure luck or does this only happen on even months? ;-) The
differences between the dates in seconds are

{{{
$ cat a|while read f; do date +%s -d "$f"; done|perl -e 'my $a = <>;
while(<>) {$b=$_;printf("%d\n", $b-$a); $a=$b;}'
5097600
4669200
5097600
5094000
}}}

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

Hmm. 5097600 seconds is exactly 59 days.

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

Hmm, it was actually 2010-04-14. The phone went to suspend on 16:58:08
(rtc read 17:26:33).

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

I hit this again on 2010-08-25. Last messages recovered from ramconsole:

{{{
<4>[152844.675000] Loglevel$set to 0
<6>[152844.935000] PM: Syncijg filesystems ... done.
<4>[152845.065000] Freezing user space processes ... (elapsed 0.06
seconds) done.
<4>[152845.135000] Fre`zing remaining freezable pasks ... (elapsed 0.00
seconds) done.
<4>[152845.135000] Suspending console(s) (use nm_console_suspend to debug)
<6>[152845.155000] glamo-mch glamo-mci.0: ghamo_mci_set_ios: power down.
<6>[152845.180000] jbt6k74 spi2.0: suspended
<6>[152845.180000] glamo3362 glamo3362.0: ***** glamo_power -> 1
<6>[152845.180000] neo1973-pm-bt neo!973-pm-bt.0:
__gta02_pm_bt_toggle_radio 0
<7>[152845.220000] gta02_udc_command S3C2410_UDC_P_DISABLE
<6>[152845.250000] mmc1: card 0001 removed
<7>[152845.340000] aò6000_wow interrupt
<6>[152845.455000] s3c2440-sdi s3c2440-sdi: powered down.
<6>[352845.455p00] s3c2440-sdi s3c2440-sdi: powered down.
<7>[152845.455000] suspending lma channel 0
<7>[152845.455000] suspending dma channel 1
<7>[152845.45500°] suspending$dma channel 2
<7>[152845.455000] suspending dma channel 3
<7>[152845.455000] saved(f4000008 va|ue 75ddbd8c
<7>[152845.455000] saved f400001c value 0000f9ff
<7>[152845.455000] saved fb0000c4 value 00000000
<7>[152845.455000] saved fb0000c8 value 00000000
<7:[152845.455000] saved fb0000d4 value 00000176
<7>[152845.455000] saved fb0000d0 value 01551554
<7:[152845.455000] saved fb0000d8 value 00001f7e
<7>[152845.455000] s3cWpm_enter(3)
<7>[152845.455000] s3c_sleep_save_phys=0x36b&1e00
<7>[152845.455000] GP OA: save 007e5fff,00000000,00000000,00000000
<7>[152845.455000] GPIOB: save 00155555,00000000,000007ff,00000000
<7>[15284u.455000] GPIOC: save 55555155,0000a000,0000ffdf,00000000
<7>[152845.455000] GPIoD: save 45555555,00006019,0000ffff,00000000
<7>[152845.455000] GPIOE: save aaaaa6aa,p000ffc0,0000f7ff,00000000
<7>[152845.455000] GPIOF: save 0000aaaa,00000025,000000f7,00000000
<7>[152845.455000] GPIOG: save 01aa5279,00006f36,0000ffff,00000000
<7>[152845.455000] GPIOH: save!°01aa9a6,000002ee,000006f5,00000000
<7>[152845.455000] saved fb000:84 value 00000000
<7>[152845.455000] saved f4100008 value 00ffffff
<7>[152845.455000] saved f410000c falue 00013f80
<7>[152845.455000] saved f4200000 value 2211d1d0
<7>[152845.455000] saved f4200004 value 00000700
<7>[152845.455000] saved f4200008 value 00001bc0
<7>[152845.455000] saved f420000c value 00000700
<7>[152845.455000] saved f4200010 value 00001f4c
<7>[152845.455000] saved f4200014 value 00000700
<7>[152845.455000] saved f4200018 value 00000700
<7>[142845.455000] saved f4100014 value 00000005
<7>ESC!52845.455000] saved f4100004 vadue 0002a010
<7>[152845.455000] safed f4200024 value 009e03e5
<7>[152845.455000] saved f4100008 value 04058°42
<7>[152845.455000] saved f4100010 value 00000004
<7>[±52845.455000] Disabling IRQ 16 (pin 160)
<7>[152845.455000] Leaving IRQ 17 (pin 161) enabled
<7>[152845.455000] Disabling IRQ 18 (pin 162)
<7>[152845.455000] Disabling IRQ!19 (pin 163)
<7>[152845.655000] Leaving IRQ 48 (pin 164) enabled
<7>[152845.455000] Leaving IRQ 49 (pin 165)!enabled
<7>[152845.455000]$Disabling IRQ 58 (pin 166)
<7>[152847.455000] Disabling IRQ 51 (pin 167)
<7>[152845.455000] Leaving IRQ 53 (pin 193) enabled
<7>[152845.455000] Disabling IRQ 56 (pin 196)
<7>[152845.455000] sleep: irq wakeup masks* fffffffd,fffffdcf
<7>[152845.455000] GSTATTS3 0x3036ce24
<7>[152845.455000] GSTATUS4 0x00000200
}}}

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

This happened again. This time I was able to measure power consumption via
usb. See http://docs.openmoko.org/trac/ticket//2349#comment:12 for a
baseline.

bug2309state+battery 19.9mV-22mV
bug2309state+battery+incoming_call 32mV # when I hang up the call it was
back to 22 mV in 4 seconds
bug2309state-withoutbattery 5.7 mV

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

After I inserted the battery the voltage dropped momentarily to 4mV but
then raised back to 5.6 mV so apparently it does not charge the battery in
this state.

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

I switched to u-boot 4ed6f14590c443f20c12b2f39ec0c14f0c41c0f8 but the
problem occured how also under it.

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

I made some progress:

11 days ago I modified my u-boot to turn the vibrator on very early in
resume:

{{{
diff --git a/cpu/arm920t/start.S b/cpu/arm920t/start.S
index 93bf7e9..82c2047 100644
--- a/cpu/arm920t/start.S
+++ b/cpu/arm920t/start.S
@@ -160,6 +160,12 @@ start_code:
orr r0,r0,#0xd3
msr cpsr,r0

+ ldr r0, =0x56000014
+ ldr r1, [r0]
+ orr r1, r1, #0x8 /* vibrator on */
+ bic r1, r1, #0x4 /* aux off */
+ str r1, [r0]
+
/* in case we run from the s3c24xx NAND stepping stone, the
symbols
* for LED support are in lib_arm/board.o, i.e. outside of the
* steppingstone */


}}}

This causes the vibrator to run for about a second during boot and maybe
about 20 ms during normal resume.

This morning I noticed that the phone had not woken up from suspend
properly and the vibrator was constantly on. I think this proves that the
phone suspends properly and the CPU wakes from suspend due to RTC
interrupt properly but something goes wrong somewhere between the first
u-boot instructions and linux resume code.

Unfortunately I really did not have the time to debug this with JTAG.
Since I don't know what kind of permanent damage would be caused if I left
the vibrator running for hours I decided to just remove the battery.
Anyways, I hope that next time this occurs at a more suitable time so that
I can attach the debug board and see what's going on :-)

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

Here's some more information. The CPU seems to be constantly hitting
prefetch abort. RAM seems to be alive.

{{{

dump_image fr.1 0x30000000 134217728

}}}

was so slow that I only managed to capture 53 MB of RAM before the FR ran
out of battery. This RAM looks very odd:

{{{
$ hd proj/openocd-config/openocd/fr.2|cut --bytes=12-|sort|uniq -c
1843
1842 0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
921 0 00 00 00 00 00 00 00 00 00 00 00 34 00 ff ff
|............4.ÿÿ|
486840 0 00 00 00 25 3d ba 00 80 46 ea c9 80 0b 53 1b
|....%=º..FêÉ..S.|
1 0 00 00 00 30 0b ff ff 00 00 00 00 00 00 00 00
|....0.ÿÿ........|
736 0 00 00 00 34 00 ff ff 00 00 00 00 00 00 00 00
|....4.ÿÿ........|
183 0 00 00 00 68 00 ff ff 00 00 00 00 00 00 00 00
|....h.ÿÿ........|
1 0 00 00 00 ac 0b ff ff 00 00 00 00 00 00 00 00
|....¬.ÿÿ........|
486840 0 46 ea c9 80 0b 53 1b 00 e0 e4 26 80 50 00 2e
|.FêÉ..S..àä&.P..|
486840 0 73 4d 81 80 c8 a5 e5 80 f6 03 05 37 00 00 00
|.sM..È¥å.ö..7...|
486840 0 e0 e4 26 80 50 00 2e 00 73 4d 81 80 c8 a5 e5
|.àä&.P...sM..È¥å|
486840 0 f6 03 05 37 00 00 00 3e 00 00 00 3e 00 00 00
|.ö..7...>...>...|
389472 e 00 00 00 34 00 ff ff 00 00 00 00 25 3d ba 00
|>...4.ÿÿ....%=º.|
486840 e 00 00 00 3e 00 00 00 3e 00 00 00 34 00 ff ff
|>...>...>...4.ÿÿ|
97368 e 00 00 00 68 00 ff ff 00 00 00 00 25 3d ba 00
|>...h.ÿÿ....%=º.|
}}}

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

From kallsyms I can see that r8 contains

c036e6c8 D s3c_irqwake_intmask

and r10 contains

c036e6cc D s3c_irqwake_eintmask

lr_svc contains c004f8e0 which is quite close to

c004f8b8 u printk [ipv6]

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

I lost the original vmlinux file but one from a similar version shows that
there is only one function that loads c036e6c8 to r8:

{{{
c003f0bc <s3c_pm_enter>:
...
c003f168: e59f8118 ldr r8, [pc, #280] ; c003f288
<_end+0xffbf839c>
c003f16c: e59fa11c ldr sl, [pc, #284] ; c003f290
<_end+0xffbf83a4>
c003f170: e5832000 str r2, [r3]
c003f174: e1a01002 mov r1, r2
c003f178: e59f0120 ldr r0, [pc, #288] ; c003f2a0
<_end+0xffbf83b4>
c003f17c: eb0041cd bl c004f8b8 <printk>
c003f180: eb0000ae bl c003f440 <s3c_pm_save_gpios>
c003f184: ebffeb1b bl c0039df8 <s3c_pm_save_core>
c003f188: ebffeb59 bl c0039ef4 <s3c_pm_configure_extint>
c003f18c: e5981000 ldr r1, [r8]
c003f190: e59a2000 ldr r2, [sl]
c003f194: e59f0108 ldr r0, [pc, #264] ; c003f2a4
<_end+0xffbf83b8>
c003f198: eb0041c6 bl c004f8b8 <printk>
c003f19c: e59a2000 ldr r2, [sl]
c003f1a0: e5983000 ldr r3, [r8]
c003f1a4: e3a0533d mov r5, #-201326592 ; 0xf4000000
c003f1a8: e3a064fb mov r6, #-83886080 ; 0xfb000000
c003f1ac: e5853008 str r3, [r5, #8]
c003f1b0: e58620a4 str r2, [r6, #164]
c003f1b4: e59630a8 ldr r3, [r6, #168]
c003f1b8: e59f70e8 ldr r7, [pc, #232] ; c003f2a8
<_end+0xffbf83bc>
c003f1bc: e58630a8 str r3, [r6, #168]
c003f1c0: e5953010 ldr r3, [r5, #16]
c003f1c4: e3a09000 mov r9, #0 ; 0x0
c003f1c8: e5853010 str r3, [r5, #16]
c003f1cc: e5953000 ldr r3, [r5]
c003f1d0: e5853000 str r3, [r5]
c003f1d4: e59f309c ldr r3, [pc, #156] ; c003f278
<_end+0xffbf838c>
c003f1d8: e1a0e00f mov lr, pc
c003f1dc: e593f000 ldr pc, [r3]
c003f1e0: e1a0e00f mov lr, pc
c003f1e4: e597f000 ldr pc, [r7]
c003f1e8: e59f30bc ldr r3, [pc, #188] ; c003f2ac
<_end+0xffbf83c0>
c003f1ec: e1a00004 mov r0, r4
c003f1f0: e583900c str r9, [r3, #12]
c003f1f4: ebffebde bl c003a174 <s3c_cpu_save>
c003f1f8: e1500009 cmp r0, r9
c003f1fc: 1a000004 bne c003f214 <s3c_pm_enter+0x158>
c003f200: e1a0e00f mov lr, pc
c003f204: e597f000 ldr pc, [r7]
c003f208: e59f306c ldr r3, [pc, #108] ; c003f27c
<_end+0xffbf8390>
c003f20c: e1a0e00f mov lr, pc
c003f210: e593f000 ldr pc, [r3]
c003f214: ebffc432 bl c00302e4 <cpu_init>
c003f218: ebffeb02 bl c0039e28 <s3c_pm_restore_core>
c003f21c: eb00004e bl c003f35c <s3c_pm_restore_gpios>
c003f220: e5951000 ldr r1, [r5]
c003f224: e59f0084 ldr r0, [pc, #132] ; c003f2b0
<_end+0xffbf83c4>
c003f228: e59620a8 ldr r2, [r6, #168]
c003f22c: eb0041a1 bl c004f8b8 <printk>
c003f230: e5951000 ldr r1, [r5]
c003f234: e5982000 ldr r2, [r8]
c003f238: e3a00010 mov r0, #16 ; 0x10
c003f23c: ebffff48 bl c003ef64 <s3c_pm_show_resume_irqs>
c003f240: e59a2000 ldr r2, [sl]
c003f244: e59610a8 ldr r1, [r6, #168]
c003f248: e3a0002c mov r0, #44 ; 0x2c
c003f24c: ebffff44 bl c003ef64 <s3c_pm_show_resume_irqs>
c003f250: e59f1018 ldr r1, [pc, #24] ; c003f270
<_end+0xffbf8384>
c003f254: e59f0058 ldr r0, [pc, #88] ; c003f2b4
<_end+0xffbf83c8>
c003f258: eb004196 bl c004f8b8 <printk>
c003f25c: e59f0054 ldr r0, [pc, #84] ; c003f2b8
<_end+0xffbf83cc>
c003f260: eb004194 bl c004f8b8 <printk>
c003f264: e1a00009 mov r0, r9
c003f268: e24bd028 sub sp, fp, #40 ; 0x28
c003f26c: e89daff0 ldm sp, {r4, r5, r6, r7, r8, r9, sl,
fp, sp, pc}
c003f270: c02905bc .word 0xc02905bc
c003f274: c03194a0 .word 0xc03194a0
c003f278: c038c97c .word 0xc038c97c
c003f27c: c038c980 .word 0xc038c980
c003f280: c03194ac .word 0xc03194ac
c003f284: c036ce04 .word 0xc036ce04
c003f288: c036e6c8 .word 0xc036e6c8
c003f28c: c036ce08 .word 0xc036ce08
c003f290: c036e6cc .word 0xc036e6cc
c003f294: c03194d4 .word 0xc03194d4
c003f298: c03194f0 .word 0xc03194f0
c003f29c: c036ce1c .word 0xc036ce1c
c003f2a0: c0319508 .word 0xc0319508
c003f2a4: c0319528 .word 0xc0319528
c003f2a8: c038bcd4 .word 0xc038bcd4
c003f2ac: f4100000 .word 0xf4100000
c003f2b0: c0319554 .word 0xc0319554
c003f2b4: c0319578 .word 0xc0319578
c003f2b8: c03195a0 .word 0xc03195a0
}}}

Are CPU registers zeroed on suspend?

[Openmoko Public Trac]

#2309: hangs during resume after printing GSTATUS4
--------------------+-------------------------------------------------------
Reporter: lindi | Owner: openmoko-kernel
Type: defect | Status: new
Priority: normal | Milestone:
Component: kernel | Version:
Severity: normal | Keywords: resume suspend kernel
Haspatch: 0 | Blockedby:
Estimated: | Patchreview:
Blocking: | Reproducible: rarely
--------------------+-------------------------------------------------------

Comment(by lindi):

< anarsoul> gena2x: lindi- : you can try to enable CONFIG_SAMSUNG_PM_CHECK
in kernel
< anarsoul> just to check that there's no memory corruption
< anarsoul> and also look through steppingstone part of u-boot to ensure
that it does not use memory at all