Page 1 of 1

Hunting down offending function crash with EPC1=0x40101074

PostPosted: Mon Oct 07, 2019 5:44 pm
by Agentsmithers
*To anyone who thinks they know how to solve this with Objdump or simular methods I will paypal you $50 as a reward for your help!*

Sending Config[../library/webserver.h][WebServerConn_connect_callback][441] - Client connected
[../library/webserver.h][WebServerConn_connect_callback][441] - Client connected
[../library/webserver.h][HttpSendWithHeader][21] - Freeheap 12592 vs. 12592
HTTP Response sent bytes: 4885
Heap allocated for SSID's: 2048 Freeheap: 15568
Heap allocated for SSID's left: 1968 Freeheap: 15568
(3,"Rob Stone-2G",-96,"34:6b:46:2d:4c:86",6)
[../library/webserver.h][HttpSendWithHeader][21] - Freeheap 7368 vs. 7368
Unable to send, need a moment to breath to free memoryE:M 8200
Fatal exception 29(StoreProhibitedCause):
�pc1=0x4000e1b2, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x00000000
ets Jan 8 2013,rst cause:2, boot mode:(3,6)

load 0x3ffe8000, len 2192, room 16
tail 0
chksum 0x8c
load 0x3ffe8890, len 22144, room 8
tail 8
chksum 0x1c
load 0x40100000, len 30784, room 0
tail 0
chksum 0x13
csum 0x13
sleep disable

Starting ESP8266 Standard!
SDK version:2.1.0(116b762)
Loaded from: 0x00
Vdd33_Const: ff
data : 0x3ffe8000 ~ 0x3ffe8890, len: 2192
rodata: 0x3ffe8890 ~ 0x3ffedf10, len: 22144
bss : 0x3ffedf10 ~ 0x3fff4a48, len: 27448
heap : 0x3fff4a48 ~ 0x3fffc000, len: 30136
reset reason: 2
GET GetEXCCount 1
[StoreEXCCount] creating with: 2
epc1=0x4000e1b2, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000000, depc=0x00000000
System halt!

I keep receiving the following exception, then I use this

[osboxes@osboxes Main]$ locate elf-objdump
[osboxes@osboxes Main]$ /home/osboxes/esp-open-sdk/xtensa-lx106-elf/bin/xtensa-lx106-elf-objdump -d Main.o -S --start-address=0x4000

Code: Select allMain.o:     file format elf32-xtensa-le

Disassembly of section .irom0.text:

00004000 <init_ThermostatSettings+0xa0>:
    4000:   000021           l32r   a2, fffc4000 <user_init+0xfffb8e20>
    4003:   000001           l32r   a0, fffc4004 <user_init+0xfffb8e24>
    4006:   0000c0           callx0   a0
    4009:   000846           j   402e <init_ThermostatSettings+0xce>
    400c:   280000           excw
    400f:   0f             .byte 0xf
    4010:   0c1266           bnei   a2, 1, 4020 <init_ThermostatSettings+0xc
    4013:   000021           l32r   a2, fffc4014 <user_init+0xfffb8e34>
    4016:   000001           l32r   a0, fffc4018 <user_init+0xfffb8e38>
    4019:   0000c0           callx0   a0
    401c:   000386           j   402e <init_ThermostatSettings+0xce>
    401f:   0f2800           excw
    4022:   082266           bnei   a2, 2, 402e <init_ThermostatSettings+0xc
    4025:   000021           l32r   a2, fffc4028 <user_init+0xfffb8e48>
    4028:   000001           l32r   a0, fffc4028 <user_init+0xfffb8e48>
    402b:   0000c0           callx0   a0
    402e:   0f1d         mov.n   a1, a15
    4030:   b108         l32i.n   a0, a1, 44
    4032:   a1f8         l32i.n   a15, a1, 40
    4034:   30c112           addi   a1, a1, 48
    4037:   f00d         ret.n
    4039:   000000           ill
        0000b1e0 <user_init>:
    b1e0:   f0c112           addi   a1, a1, -16
    b1e3:   3109         s32i.n   a0, a1, 12
    b1e5:   21f9         s32i.n   a15, a1, 8
    b1e7:   01fd         mov.n   a15, a1
    b1e9:   020c         movi.n   a2, 0
    b1eb:   000005           call0   b1ec <user_init+0xc>
    b1ee:   020c         movi.n   a2, 0
    b1f0:   b6a232           movi   a3, 0x2b6
    b1f3:   000001           l32r   a0, fffcb1f4 <user_init+0xfffc0014>
    b1f6:   0000c0           callx0   a0
    b1f9:   000005           call0   b1fc <user_init+0x1c>
    b1fc:   000005           call0   b200 <user_init+0x20>
    b1ff:   000021           l32r   a2, fffcb200 <user_init+0xfffc0020>
    b202:   000001           l32r   a0, fffcb204 <user_init+0xfffc0024>
    b205:   0000c0           callx0   a0
    b208:   0f1d         mov.n   a1, a15
    b20a:   3108         l32i.n   a0, a1, 12
    b20c:   21f8         l32i.n   a15, a1, 8
    b20e:   10c112           addi   a1, a1, 16
    b211:   f00d         ret.n

It seems that the address 0x4000e1b2 does not exist in Main.O as it ends on 0x4000b211... Anyone know any background details on how I can narrow down the root cause. Thank you!!

As for addtional detail, it seems in https://www.espressif.com/sites/default ... ses_en.pdf on page 3/4 that this may be more of a ROM issue?

Update: It seems that I found a feed of the obj dump on google.
4000e190: 743030 extui a3, a3, 0, 8
4000e193: 117380 slli a7, a3, 8
4000e196: 203370 or a3, a3, a7
4000e199: 117300 slli a7, a3, 16
4000e19c: 203370 or a3, a3, a7
4000e19f: 205220 or a5, a2, a2
4000e1a2: cee207 bbsi a2, 0, 4000e174 <memmove+0x128>
4000e1a5: d8e217 bbsi a2, 1, 4000e181 <memmove+0x135>
4000e1a8: 417440 srli a7, a4, 4
4000e1ab: 179c beqz.n a7, 4000e1c0 <memset+0x30>
4000e1ad: 1167c0 slli a6, a7, 4
4000e1b0: 665a add.n a6, a6, a5
4000e1b2: 0539 s32i.n a3, a5, 0
4000e1b4: 1539 s32i.n a3, a5, 4
4000e1b6: 2539 s32i.n a3, a5, 8
4000e1b8: 3539 s32i.n a3, a5, 12
4000e1ba: 10c552 addi a5, a5, 16
4000e1bd: f12567 blt a5, a6, 4000e1b2 <memset+0x22>
4000e1c0: 056437 bbci a4, 3, 4000e1c9 <memset+0x39>
4000e1c3: 0539 s32i.n a3, a5, 0
4000e1c5: 1539 s32i.n a3, a5, 4
4000e1c7: 558b addi.n a5, a5, 8
4000e1c9: 036427 bbci a4, 2, 4000e1d0 <memset+0x40>
4000e1cc: 0539 s32i.n a3, a5, 0
4000e1ce: 554b addi.n a5, a5, 4
4000e1d0: 046417 bbci a4, 1, 4000e1d8 <memset+0x48>
4000e1d3: 005532 s16i a3, a5, 0
4000e1d6: 552b addi.n a5, a5, 2
4000e1d8: 026407 bbci a4, 0, 4000e1de <memset+0x4e>
4000e1db: 004532 s8i a3, a5, 0
4000e1de: f00d ret.n

I think my program is crashing within a memset call.. Maybe one of my alloc's is sliently failing cauing memset in my userland code to trigger this exception?

https://0x04.net/~mwk/doc/xtensa.pdf <-- I then found this and looked up s32i.n and found this on page 78/662

Code: Select allTable 4–27. Code Density Option Instruction Additions
Instruction1 Format Definition
ADD.N RRRN Add two registers (same as ADD instruction but with a 16-bit encoding).
ADDI.N RRRN Add register and immediate (-1 and 1..15).
BEQZ.N RI16 Branch if register is zero with a 6-bit unsigned offset (forward only).
BNEZ.N RI16 Branch if register is non-zero with a 6-bit unsigned offset (forward only).
BREAK.N2 RRRN This instruction is the same as BREAK but with a 16-bit encoding.
L32I.N RRRN Load 32 bits, 4-bit offset
MOV.N RRRN Narrow move
MOVI.N RI7 Load register with immediate (-32..95).
NOP.N RRRN This instruction performs no operation. It is typically used for instruction alignment.
RET.N RRRN The same as RET but with a 16-bit encoding.
RETW.N3 RRRN The same as RETW but with a 16-bit encoding.
S32I.N RRRN Store 32 bits, 4-bit offset

if seems when executing S32I.N RRRN Store 32 bits, 4-bit offset it triggers the issue, This may be more information then I need but I think I have an idea of what is triggering it but I am uncertain on how to locate the calling function without laying a bunch of printf's everywhere. Anyone know how to get a call stack or get additional details?

So I appended this check to each of my alloc's before the memset's got to them and it seemed to have resolved the issue.
Code: Select allif (FormatedHTMLPage == 0x0)
         os_printf("[%s][%s][%d] - Unable to Alloc memory, exiting\r\n", __FILE__ ,__func__, __LINE__);

The larger issue is I only have about 25k of heap to have a client connect via WPA-PSK2 and then send it a detailed webpage.
The page itself is almost 8k in side. Anyone know of an easy way to send this waypage with dynamic configurations inside?
The way I have it now is its loaded in flash and it reads it into a GlobalVar at the start. Yes sadly this takes a var kinda like this
char webpage[8192] = SPiRead(WebpageInSPIRom)
Then I have to allocate another for sprintf to then change my variables before I shoot it off to the client.
Problem is thats 16k raw just to format the data in memory. Any pointers would be greatly appreciated! :)

Update3: Here is my latest error @ 0x40101074
Code: Select allIP Address:
Netmask   :
Gateway   :
Calling myConnectToStronestWifiCallback callback
AP Already Loaded[../library/webserver.h][init_webserver][786]
Setting Callback for WebServer
-=-=-=-=-=-=-=-=- START DumpSPItable -=-=-=-=-=-=-=-=-
Read from: 0x67004-0x67007, Pending ff: EOF!
-=-=-=-=-=-=-=-=- END DumpSPItable -=-=-=-=-=-=-=-=-
IGMP Joining: 3301a8c0 faffffef,  joined
 Fatal exception 9(LoadStoreAlignmentCause):
epc1=0x40101074, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000003, depc=0x0�000000
 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x3ffe8000, len 2192, room 16
tail 0
chksum 0xcd
load 0x3ffe8890, len 22224, room 8
tail 8
chksum 0x44
load 0x40100000, len 30784, room 0
tail 0
chksum 0xc8
csum 0xc8
sleep disable

Starting ESP8266 Standard!
SDK version:2.1.0(116b762)
Loaded from: 0x00
Vdd33_Const: ff
data  : 0x3ffe8000 ~ 0x3ffe8890, len: 2192
rodata: 0x3ffe8890 ~ 0x3ffedf60, len: 22224
bss   : 0x3ffedf60 ~ 0x3fff4a98, len: 27448
heap  : 0x3fff4a98 ~ 0x3fffc000, len: 30056
reset reason: 2
[StoreEXCCount] creating with: 1
epc1=0x40101074, epc2=0x00000000, epc3=0x00000000, excvaddr=0x00000003, depc=0x00000000
System halt!

Anyone know how to see the function call at address's greater then 0x40100000? With this new EPC1 value it seems its assigned to the mapping of IRAM1.. Am I on the right track, should I objdump the Main.o and review the offset 0x1074 within?

Any pointers in the right direciton would be greatly appreicated.

Re: Hunting down offending function crash with EPC1=0x401010

PostPosted: Sun Apr 12, 2020 7:06 am
by eriksl
Congratulations, you apparently have a bug where either some random RAM is trashed or one of the SDK or user stack. At some point this will make the code jump to a random location and the CPU will start executing nonsense instructions which will lead to either an illegal instruction or some of the memory addressing exceptions.

The best way to debug is to rollback all modifications until the issue no longer occurs. Then study the code that was added.