1017504 - [Sora][FOTA] after FOTA updates,the system crash

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Vendcom, defect, P1)

Description

[sync-1]

Firefox OS v1.3
 AU_LINUX_GECKO_B2G_JB_3.2.01.03.00.112.301
 Mozilla build ID:20140422024003
 
 DEFECT DESCRIPTION:
 
  after FOTA updates,the system crash
 
  REPRODUCING PROCEDURES:
 
 1.download SW 12C+ZZ10,write CU:4019X-2CALEU0,check the updates and download the diff package.
 2.install the diff package,after install ,the device ask whether to send crash reports to mozilla,and there is no updates message in status bar to tell us the upgrade is sucessfull.--KO
 
 中文描述：
 1.下载SW12C+ZZ10,写好CU:4019X-2CALEU0,进入system updates下载好差分包；
 2.安装好差分包重启手机，会问你是否需要把crash reports发送给mozilla,在status bar上也没有升级成功的消息--KO
 
  EXPECTED BEHAVIOUR:
 
 after system updates,the system should not crash
 
 tel:[phone]
 reproducing rate:60%
 
  ASSOCIATE SPECIFICATION:
 
  TEST PLAN REFERENCE:
 
  TOOLS AND PLATFORMS USED:
 
  USER IMPACT:
 
  REPRODUCING RATE:
 
  For FT PR, Please list reference mobile's behavior:

Comment 1

[jian.bai]

after FOTA updates,the system crash
please view:
https://crash-stats.mozilla.com/report/index/93bb119d-45a4-4160-8b6e-a99ed2140528

https://crash-stats.mozilla.com/report/index/1e41d4a9-5acc-440d-a0fb-e36c42140527

Thanks

baijian

Comment 2

[sync-1]

Attachment: SW12A+ZZ10-->SW12F+ZZ10 dmesg log

Comment 3

[sync-1]

Attachment: SW12A+ZZ10-->SW12F+ZZ10 logcat

Comment 4

[sync-1]

Attachment: SW12C+ZZ10-->SW12F+ZZ10 logcat

Comment 5

[sync-1]

Attachment: SW12A+ZZ10-->SW12F+ZZ10 Crash Reports

Comment 6

[Preeti Raghunath(:Preeti)]

Andrew,

Please review and reassign

Comment 7

[Andrew Overholt [:overholt]]

Dave/Ben, any thoughts?

Comment 8

[Ben Turner (not reading bugmail, use the needinfo flag!)]

Assuming this is using v1.3 (the filenames in the report don't line up to anything easily identifiable)?

If so, we're here:

http://mxr.mozilla.org/mozilla-b2g28_v1_3/source/ipc/chromium/src/base/message_loop.cc#340

The crash address is not 0, so I'm going to guess that we're trying to run a Task that has already been deleted.

Crash-stats shows we have an extremely small number of crashes with this signature so I'm inclined to think that some local modification has caused this.

Comment 9

[Dave Hylands [:dhylands]]

I took a look at the crash. Both of the reports from comment 1 are caused by a segmentation fault on address 0x6567617a

I think it's crashing while trying to dereference the this pointer, and 0x6567617a isn't a valid value for a this pointer (since its not 4 byte aligned).

Maybe conincidental, but 0x6567617a is made up entirely of ASCII characters, which would look like "zage" in memory. So it's quite probable that we've got a memory trample.

I don't have a Sora device, so I probably can't do much more investigation.

To investigate further, I think we'd need to flash Sora device with the image in question, and have the exact update which is causing the problem.

What type of update was this? Were files replaced? Or patched? If the files were patched, and an non-matching base file was patched, then I wouldn't be surprised by a crash.

Comment 10

[Andrew Overholt [:overholt]]

Please help out with answers to Dave's questions in comment 9, Baijian.

Comment 11

[Dave Hylands [:dhylands]]

And I guess that you should also do a comparison.

You have a device with image A that you want to update to image B.

And you have some update which takes you from A to B.

It isn't clear to me exactly when you're seeing the crash. I've been assuming that you see the crash after booting up with A + update applied. Do you also see the crash when you flash B directly?

If you don't see the crash when B is flashed directly, then you should pull the files from the phone after flashing with A and applying the update and identify what's different from applying B directly.

Comment 12

[jian.bai]

Sorry for delay

I used google OTA update, and upgrade the images:boot.img, system.img, recovery.img, modem(firmware), custpack.img(gaia:apps) and so on. The base files is the same as files in phone before update.

There is the recovery log:

Installing update...
Verifying current system...
partition read matched size 7813120 sha cc7299cee2e8ae478626e3edc6f1e1bd88b0c49d
partition read matched size 353136 sha c1642426b92beff52bd308c68b6c43a33b77dc32
partition read matched size 146884 sha 319f1ae5c04f9c00ea40f6d0a2b96bc7ad772665
partition read matched size 338988 sha 85ae36242d71db1d1a5ee6b895d6def47ce0120a
57356288 bytes free on /cache (26158588 needed)
Removing unneeded files...patch /custpack/b2g/defaults/settings.json: 
Patching system files...
now dadd25aa
patch /custpack/build.prop: now cb60f098
patch /custpack/webapps/alcatelhelp.gaiamobile.org/application.zip: now 4040add8
patch /custpack/webapps/bluetooth.gaiamobile.org/application.zip: now 169bad1d
patch /custpack/webapps/bluetooth.gaiamobile.org/manifest.webapp: now 60a9ce8d
patch /custpack/webapps/browser.gaiamobile.org/application.zip: now 8b2be2f4
patch /custpack/webapps/calendar.gaiamobile.org/application.zip: now d242adcb
patch /custpack/webapps/camera.gaiamobile.org/application.zip: now fdb63c13
patch /custpack/webapps/clock.gaiamobile.org/application.zip: now 41ab8bcc
patch /custpack/webapps/communications.gaiamobile.org/application.zip: now 2dc5e986
patch /custpack/webapps/costcontrol.gaiamobile.org/application.zip: now f27208a1
patch /custpack/webapps/costcontrol.gaiamobile.org/manifest.webapp: now 304e5c16
patch /custpack/webapps/email.gaiamobile.org/application.zip: now cfc19e84
patch /custpack/webapps/filemanager.gaiamobile.org/application.zip: now 76ce4283
patch /custpack/webapps/fl.gaiamobile.org/application.zip: now e7b7638e
patch /custpack/webapps/fl.gaiamobile.org/manifest.webapp: now dfd4feaa
patch /custpack/webapps/fm.gaiamobile.org/application.zip: now 8f612c28
patch /custpack/webapps/gallery.gaiamobile.org/application.zip: now 9cbc05c9
patch /custpack/webapps/homescreen.gaiamobile.org/application.zip: now 24c437cf
patch /custpack/webapps/keyboard.gaiamobile.org/application.zip: now 4dbf2de9
patch /custpack/webapps/keyboard.gaiamobile.org/manifest.webapp: now dd354644
patch /custpack/webapps/mmitest.gaiamobile.org/application.zip: now c49f6327
patch /custpack/webapps/music.gaiamobile.org/application.zip: now 1e15c680
patch /custpack/webapps/pdfjs.gaiamobile.org/application.zip: now 9bd76c2f
patch /custpack/webapps/pdfjs.gaiamobile.org/manifest.webapp: now 52c0ca44
patch /custpack/webapps/ringtones.gaiamobile.org/application.zip: now 9ff4a570
patch /custpack/webapps/setringtone.gaiamobile.org/application.zip: now ec138b13
patch /custpack/webapps/settings.gaiamobile.org/application.zip: now 5e3dfcfa
patch /custpack/webapps/sms.gaiamobile.org/application.zip: now 292a7d0d
patch /custpack/webapps/system.gaiamobile.org/application.zip: now 4aedfe9a
patch /custpack/webapps/video.gaiamobile.org/application.zip: now f71a1424
patch /custpack/webapps/wallpaper.gaiamobile.org/application.zip: now 37b14331
patch /custpack/webapps/wallpaper.gaiamobile.org/manifest.webapp: now 62a90419
patch /custpack/webapps/wappush.gaiamobile.org/application.zip: now 2bd44ddd
patch /custpack/webapps/wappush.gaiamobile.org/manifest.webapp: now f8a7b0ae
patch /firmware/IMAGE/MODEM.B00: now 65bd43d9
patch /firmware/IMAGE/MODEM.B01: now 8a82d39e
patch /firmware/IMAGE/MODEM.B04: now 1a31f233
patch /firmware/IMAGE/MODEM.B05: now 19948f27
patch /firmware/IMAGE/MODEM.B06: now cd155766
patch /firmware/IMAGE/MODEM.B09: now 2fb2bb25
patch /firmware/IMAGE/MODEM.B16: now 8b6675ac
patch /firmware/IMAGE/MODEM.B17: now 38ea9b82
patch /firmware/IMAGE/MODEM.B18: now 7b9e84eb
patch /firmware/IMAGE/MODEM.B22: now 2765ff48
patch /firmware/IMAGE/MODEM.B23: now 636e40fd
patch /firmware/IMAGE/MODEM.B27: now b3343107
patch /firmware/IMAGE/MODEM.B28: now 7b97704e
patch /firmware/IMAGE/MODEM.B29: now 7e2ffbd9
patch /firmware/IMAGE/MODEM.MDT: now db512224
patch /system/b2g/application.ini: now e61817ed
patch /system/b2g/b2g: now 30e542ea
patch /system/b2g/distribution/bundles/libqc_b2g_location/libqc_b2g_location.so: now 41314627
patch /system/b2g/distribution/bundles/libqc_b2g_ril/content_helper/QCContentHelper.js: now 04ec9b00
patch /system/b2g/distribution/bundles/libqc_b2g_ril/libqc_b2g_ril.so: now ae4a29e1
patch /system/b2g/libfreebl3.so: now fd2125cf
patch /system/b2g/libmozglue.so: now c941867a
patch /system/b2g/libnss3.so: now cd718a9e
patch /system/b2g/libnssckbi.so: now 9eccb535
patch /system/b2g/libsoftokn3.so: now 359c7b27
patch /system/b2g/libxul.so: now 6d56fd99
patch /system/b2g/omni.ja: now 04ab7f39
patch /system/b2g/plugin-container: now 1d609d48
patch /system/b2g/updater: now cd09d841
patch /system/bin/debuggerd: now 85c96cc3
patch /system/bin/mcDriverDaemon: now cdfe00e9
patch /system/bin/mdnsd: now d29c5330
patch /system/bin/time_daemon: now d2128554
patch /system/bin/trace_util: now 93238243
patch /system/etc/plmn-list.conf: now 5f17becd
patch /system/etc/recovery-resource.dat: now 7cd63d7b
patch /system/lib/libLLVM.so: now db5fbad1
patch /system/lib/libRS.so: now 077059be
patch /system/lib/libRSCpuRef.so: now 1dc9f7e5
patch /system/lib/libRSDriver.so: now ce6af356
patch /system/lib/libbcc.sha1.so: now 05b9386f
patch /system/lib/libbcc.so: now f7be4412
patch /system/lib/libcompiler_rt.so: now 56ab1f59
patch /system/lib/libmdnssd.so: now 008f70f2
patch /system/lib/libstagefright_soft_aacdec.so: now 5746538b
patch /system/lib/libstagefright_soft_aacenc.so: now 97d4aca1
patch /system/lib/libwebrtc_audio_preprocessing.so: now bf0d9433
patch /system/lib/modules/ansi_cprng.ko: now 8e40c5a0
patch /system/lib/modules/coresight-event.ko: now 7f00a951
patch /system/lib/modules/dma_test.ko: now 00be69cf
patch /system/lib/modules/evbug.ko: now 03aadf4a
patch /system/lib/modules/gpio_axis.ko: now aaee6190
patch /system/lib/modules/gpio_event.ko: now 0ce4eb23
patch /system/lib/modules/gpio_input.ko: now 88b5abc3
patch /system/lib/modules/gpio_matrix.ko: now 03d7bc3c
patch /system/lib/modules/gpio_output.ko: now 3eed0301
patch /system/lib/modules/mmc_test.ko: now ea47b2a0
patch /system/lib/modules/msm-buspm-dev.ko: now c909111c
patch /system/lib/modules/oprofile.ko: now 805fb4a3
patch /system/lib/modules/pronto/pronto_wlan.ko: now 01564f9e
patch /system/lib/modules/qcedev.ko: now f3c1adbd
patch /system/lib/modules/qcrypto.ko: now 972e0d96
patch /system/lib/modules/radio-iris-transport.ko: now 3a83cca0
patch /system/lib/modules/reset_modem.ko: now e7e942ca
patch /system/lib/modules/spidev.ko: now 91c557f7
patch /system/vendor/lib/libril-qc-qmi-1.so: now 023b37e1
Patching boot image...
patch EMMC:/dev/block/platform/msm_sdcc.1/by-name/boot:7813120:cc7299cee2e8ae478626e3edc6f1e1bd88b0c49d:7813120:42480fa2206be18431f0dedb7c94de84b39161ad: partition read matched size 7813120 sha cc7299cee2e8ae478626e3edc6f1e1bd88b0c49d
57356288 bytes free on /cache (7813120 needed)
now 42480fa2
raw O_SYNC write /dev/block/platform/msm_sdcc.1/by-name/boot attempt 1 start at 0
  caches dropped
verification read succeeded (attempt 1)
sleeping after close
get_Partition_info partition : /dev/block/platform/msm_sdcc.1/by-name/fsg byte_size : 180000
mmc_raw_erase partition : /dev/block/platform/msm_sdcc.1/by-name/fsg  byte_size : 180000
Writing study img...
mmc_raw_erase done!
get_Partition_info partition : /dev/block/platform/msm_sdcc.1/by-name/modemst1 byte_size : 180000
mmc_raw_erase partition : /dev/block/platform/msm_sdcc.1/by-name/modemst1  byte_size : 180000
mmc_raw_erase done!
get_Partition_info partition : /dev/block/platform/msm_sdcc.1/by-name/modemst2 byte_size : 180000
mmc_raw_erase partition : /dev/block/platform/msm_sdcc.1/by-name/modemst2  byte_size : 180000
mmc_raw_erase done!
Patching emmcboot image...patch EMMC:/dev/block/platform/msm_sdcc.1/by-name/aboot:353136:c1642426b92beff52bd308c68b6c43a33b77dc32:353136:225fa998c283f879505a2a7a3ae0ec9990240cd0: partition read matched size 353136 sha c1642426b92beff52bd308c68b6c43a33b77dc32
57356288 bytes free on /cache (353136 needed)

now 225fa998
raw O_SYNC write /dev/block/platform/msm_sdcc.1/by-name/aboot attempt 1 start at 0
  caches dropped
verification read succeeded (attempt 1)
sleeping after close
Patching rpm image...patch EMMC:/dev/block/platform/msm_sdcc.1/by-name/rpm:146884:319f1ae5c04f9c00ea40f6d0a2b96bc7ad772665:146884:ab4251d6f39536e755e151765c9638cb8a9ccf46: partition read matched size 146884 sha 319f1ae5c04f9c00ea40f6d0a2b96bc7ad772665
57356288 bytes free on /cache (146884 needed)

now ab4251d6
raw O_SYNC write /dev/block/platform/msm_sdcc.1/by-name/rpm attempt 1 start at 0
  caches dropped
verification read succeeded (attempt 1)
sleeping after close
Patching tz image...patch EMMC:/dev/block/platform/msm_sdcc.1/by-name/tz:338988:85ae36242d71db1d1a5ee6b895d6def47ce0120a:338988:bd47116ffc20c1ae71b120fe3f28998ad669d8cc: 
partition read matched size 338988 sha 85ae36242d71db1d1a5ee6b895d6def47ce0120a
57356288 bytes free on /cache (338988 needed)
now bd47116f
raw O_SYNC write /dev/block/platform/msm_sdcc.1/by-name/tz attempt 1 start at 0
  caches dropped
verification read succeeded (attempt 1)
sleeping after close
patch EMMC:/dev/block/platform/msm_sdcc.1/by-name/abootbk:353136:c1642426b92beff52bd308c68b6c43a33b77dc32:353136:225fa998c283f879505a2a7a3ae0ec9990240cd0: partition read matched size 353136 sha c1642426b92beff52bd308c68b6c43a33b77dc32
57356288 bytes free on /cache (353136 needed)
Patching abootbk image...
now 225fa998
raw O_SYNC write /dev/block/platform/msm_sdcc.1/by-name/abootbk attempt 1 start at 0
  caches dropped
verification read succeeded (attempt 1)
sleeping after close
patch EMMC:/dev/block/platform/msm_sdcc.1/by-name/rpmbk:146884:319f1ae5c04f9c00ea40f6d0a2b96bc7ad772665:146884:ab4251d6f39536e755e151765c9638cb8a9ccf46: Patching rpmbk image...partition read matched size 146884 sha 319f1ae5c04f9c00ea40f6d0a2b96bc7ad772665
57356288 bytes free on /cache (146884 needed)

now ab4251d6
raw O_SYNC write /dev/block/platform/msm_sdcc.1/by-name/rpmbk attempt 1 start at 0
  caches dropped
verification read succeeded (attempt 1)
sleeping after close
patch EMMC:/dev/block/platform/msm_sdcc.1/by-name/tzbk:338988:85ae36242d71db1d1a5ee6b895d6def47ce0120a:338988:bd47116ffc20c1ae71b120fe3f28998ad669d8cc: Patching tz image...partition read matched size 338988 sha 85ae36242d71db1d1a5ee6b895d6def47ce0120a
57356288 bytes free on /cache (338988 needed)

now bd47116f
raw O_SYNC write /dev/block/platform/msm_sdcc.1/by-name/tzbk attempt 1 start at 0
  caches dropped
verification read succeeded (attempt 1)
sleeping after close
minzip: Extracted 3 file(s)
minzip: Extracted 1 file(s)
minzip: Extracted 1 file(s)
minzip: Extracted 2 file(s)
Unpacking new files...
Unpacking new recovery...
Symlinks and permissions...
script result was [/system]
Installation success.
package install result:INSTALL SUCCESS
dir_name = /data/fota




(In reply to Dave Hylands [:dhylands] from comment #9)
> I took a look at the crash. Both of the reports from comment 1 are caused by
> a segmentation fault on address 0x6567617a
> 
> I think it's crashing while trying to dereference the this pointer, and
> 0x6567617a isn't a valid value for a this pointer (since its not 4 byte
> aligned).
> 
> Maybe conincidental, but 0x6567617a is made up entirely of ASCII characters,
> which would look like "zage" in memory. So it's quite probable that we've
> got a memory trample.
> 
> I don't have a Sora device, so I probably can't do much more investigation.
> 
> To investigate further, I think we'd need to flash Sora device with the
> image in question, and have the exact update which is causing the problem.
> 
> What type of update was this? Were files replaced? Or patched? If the files
> were patched, and an non-matching base file was patched, then I wouldn't be
> surprised by a crash.

Comment 13

[jian.bai]

I will do a comparison.The crash happened after booting up with A + update applied first time.Applying B directly,there is no crash.

(In reply to Dave Hylands [:dhylands] from comment #11)
> And I guess that you should also do a comparison.
> 
> You have a device with image A that you want to update to image B.
> 
> And you have some update which takes you from A to B. 
> 
> It isn't clear to me exactly when you're seeing the crash. I've been
> assuming that you see the crash after booting up with A + update applied. Do
> you also see the crash when you flash B directly?
> 
> If you don't see the crash when B is flashed directly, then you should pull
> the files from the phone after flashing with A and applying the update and
> identify what's different from applying B directly.

Comment 14

[jian.bai]

Hi 

I had a test 
http://mxr.mozilla.org/mozilla-central/source/dom/apps/src/Webapps.jsm#58

change "let runUpdate = AppsUtils.isFirstRun(Services.prefs)" to "let runUpdate = true",

Then every time boot-up ,there will be a cash:
https://crash-stats.mozilla.com/report/index/da9ad316-a242-437d-87ad-0927a2140529
Those two crash is related ??

Thanks

Comment 15

[Dave Hylands [:dhylands]]

I built 1.3 for my hamachi, and I observed a crash at the end of FTU. It was a segfault on address 0. So this may be the same crash.

I'll dig into this further on Monday (I just thought I update where I got to today).

The crash I ran into has nothing to do with updates. It happened on the first run after flashing.

Comment 16

[Dave Hylands [:dhylands]]

Well it only seems to have happened once for. So far, all further attempts to reproduce have failed.

Looking at comment 12, it appears that the update was applied fine, and that the problem is a bug with the updated version of SW.

Comment 17

[Dave Hylands [:dhylands]]

OK - I got something reproducible.

The secret seems to be to get the "First Time User" app to run. For me, on my hamachi, the crash I'm seeing is at the very end of the FTU run, just after you hit "Skip".

BrowserElementChildPreload.js's _takeScreenShot function winds up calling ctx.scale which eventually tries to call dlopen on /system/lib/egl/libGLES_android.so

That's the last place I get anything sensible from the debugger.

I've been doing:

> PRODUCTION=1 make -C gaia reset-gaia

which will clean things up so that the FTU app runs. I then just click Next through the FTU app, and after pressing Skip, I get the crash (in the Communications app, which is where FTU is).

Here's the backtrace just before calling dlopen:

(gdb) break Loader.cpp:278
Breakpoint 1 at 0x4005ca04: file frameworks/base/opengl/libs/EGL/Loader.cpp, line 278.
(gdb) c
Continuing.

Breakpoint 1, android::Loader::load_driver (this=0x45441020, kind=0x400625e6 "GLES", tag=0x45441370 "android", cnx=0x40068d10, mask=7) at frameworks/base/opengl/libs/EGL/Loader.cpp:278
278	    void* dso = dlopen(driver_absolute_path, RTLD_NOW | RTLD_LOCAL);
(gdb) bt
#0  android::Loader::load_driver (this=0x45441020, kind=0x400625e6 "GLES", tag=0x45441370 "android", cnx=0x40068d10, mask=7) at frameworks/base/opengl/libs/EGL/Loader.cpp:278
#1  0x4005cb3c in android::Loader::open (this=0x45441020, display=<value optimized out>, impl=<value optimized out>, cnx=0x40068d10) at frameworks/base/opengl/libs/EGL/Loader.cpp:188
#2  0x4004f43a in egl_init_drivers_locked () at frameworks/base/opengl/libs/EGL/egl.cpp:261
#3  android::egl_init_drivers () at frameworks/base/opengl/libs/EGL/egl.cpp:289
#4  0x4005161e in eglGetDisplay (display=0x0) at frameworks/base/opengl/libs/EGL/eglApi.cpp:138
#5  0x40be56c0 in mozilla::gl::GLLibraryEGL::fGetDisplay (this=0x42afeb94) at /home/work/B2G-hamachi-1.3/gecko/gfx/gl/GLLibraryEGL.h:139
#6  mozilla::gl::GLLibraryEGL::EnsureInitialized (this=0x42afeb94) at /home/work/B2G-hamachi-1.3/gecko/gfx/gl/GLLibraryEGL.cpp:198
#7  0x40be1a6c in mozilla::gl::GLContextProviderEGL::CreateOffscreen (size=..., caps=..., flags=mozilla::gl::ContextFlagsNone) at /home/work/B2G-hamachi-1.3/gecko/gfx/gl/GLContextProviderEGL.cpp:904
#8  0x412a25d8 in mozilla::dom::CanvasRenderingContext2D::EnsureTarget (this=0x441eb800) at /home/work/B2G-hamachi-1.3/gecko/content/canvas/src/CanvasRenderingContext2D.cpp:910
#9  0x412a3194 in mozilla::dom::CanvasRenderingContext2D::TransformWillUpdate (this=0x7) at /home/work/B2G-hamachi-1.3/gecko/content/canvas/src/CanvasRenderingContext2D.cpp:2074
#10 0x412a33f6 in mozilla::dom::CanvasRenderingContext2D::Scale (this=0x7, x=-6.2943654484115541e-06, y=1, error=...) at /home/work/B2G-hamachi-1.3/gecko/content/canvas/src/CanvasRenderingContext2D.cpp:1207
#11 0x40cc84ee in scale (cx=0x4045e4a0, obj=<value optimized out>, self=0x441eb800, args=...) at /home/work/B2G-hamachi-1.3/objdir-gecko-debug-userdebug/dom/bindings/CanvasRenderingContext2DBinding.cpp:882
#12 0x40cbc9f2 in genericMethod (cx=0x4045e4a0, argc=<value optimized out>, vp=<value optimized out>)
    at /home/work/B2G-hamachi-1.3/objdir-gecko-debug-userdebug/dom/bindings/CanvasRenderingContext2DBinding.cpp:4853
#13 0x41e89d90 in js::CallJSNative (cx=0x4045e4a0, native=0x40cbc939 <genericMethod>, args=...) at /home/work/B2G-hamachi-1.3/gecko/js/src/jscntxtinlines.h:220
#14 0x41e9d2e2 in js::Invoke (cx=0x4045e4a0, args=..., construct=js::NO_CONSTRUCT) at /home/work/B2G-hamachi-1.3/gecko/js/src/vm/Interpreter.cpp:463
#15 0x41e90ba0 in Interpret (cx=0x4045e4a0, state=<value optimized out>) at /home/work/B2G-hamachi-1.3/gecko/js/src/vm/Interpreter.cpp:2511
#16 0x41e9cbee in js::RunScript (cx=0x4045e4a0, state=...) at /home/work/B2G-hamachi-1.3/gecko/js/src/vm/Interpreter.cpp:420
#17 0x41e9d278 in js::Invoke (cx=0x4045e4a0, args=..., construct=js::NO_CONSTRUCT) at /home/work/B2G-hamachi-1.3/gecko/js/src/vm/Interpreter.cpp:482
#18 0x41e9dbde in js::Invoke (cx=0x4045e4a0, thisv=..., fval=..., argc=0, argv=0xbeda8a90, rval=...) at /home/work/B2G-hamachi-1.3/gecko/js/src/vm/Interpreter.cpp:519
#19 0x41d5e1b2 in JS_CallFunctionValue (cx=0x4045e4a0, objArg=<value optimized out>, fval=..., argc=0, argv=0xbeda8a90, rval=0xbeda8b40) at /home/work/B2G-hamachi-1.3/gecko/js/src/jsapi.cpp:5008
#20 0x40fd65e2 in nsXPCWrappedJSClass::CallMethod (this=0x45385d00, wrapper=<value optimized out>, methodIndex=<value optimized out>, info_=0x43e60dc0, nativeParams=0xbeda8cd8)
    at /home/work/B2G-hamachi-1.3/gecko/js/xpconnect/src/XPCWrappedJSClass.cpp:1413
#21 0x40fd2acc in nsXPCWrappedJS::CallMethod (this=0x448f7100, methodIndex=3, info=0x43e60dc0, params=0xbeda8cd8) at /home/work/B2G-hamachi-1.3/gecko/js/xpconnect/src/XPCWrappedJS.cpp:479
#22 0x4089bd4c in PrepareAndDispatch (self=<value optimized out>, methodIndex=<value optimized out>, args=<value optimized out>)
    at /home/work/B2G-hamachi-1.3/gecko/xpcom/reflect/xptcall/src/md/unix/xptcstubs_arm.cpp:105
#23 0x4089b384 in SharedStub () from /home/work/B2G-hamachi-1.3/objdir-gecko-debug-userdebug/dist/bin/libxul.so
#24 0x40866ac0 in Run (this=0x453695c0) at /home/work/B2G-hamachi-1.3/gecko/xpcom/base/nsMessageLoop.cpp:113
#25 0x40a37ce8 in MessageLoop::RunTask (this=0xbeda9894, task=0x453695c0) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:340
#26 0x40a37d2c in MessageLoop::ProcessNextDelayedNonNestableTask (this=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:238
#27 0x40a37d3c in MessageLoop::DoIdleWork (this=0xbeda8cd8) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:479
#28 0x40a46c52 in mozilla::ipc::MessagePump::Run (this=0x40402b20, aDelegate=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/glue/MessagePump.cpp:116
#29 0x40a46d98 in mozilla::ipc::MessagePumpForChildProcess::Run (this=0x40402b20, aDelegate=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/glue/MessagePump.cpp:250
#30 0x40a381f2 in MessageLoop::RunInternal (this=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:222
#31 0x40a3820a in MessageLoop::RunHandler (this=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:215
#32 MessageLoop::Run (this=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:189
#33 0x40f7e04a in nsBaseAppShell::Run (this=0x44163c40) at /home/work/B2G-hamachi-1.3/gecko/widget/xpwidgets/nsBaseAppShell.cpp:161
#34 0x41918e06 in XRE_RunAppShell () at /home/work/B2G-hamachi-1.3/gecko/toolkit/xre/nsEmbedFunctions.cpp:679
#35 0x40a46d02 in mozilla::ipc::MessagePumpForChildProcess::Run (this=0x40402b20, aDelegate=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/glue/MessagePump.cpp:217
#36 0x40a381f2 in MessageLoop::RunInternal (this=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:222
#37 0x40a3820a in MessageLoop::RunHandler (this=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:215
#38 MessageLoop::Run (this=0xbeda9894) at /home/work/B2G-hamachi-1.3/gecko/ipc/chromium/src/base/message_loop.cc:189
#39 0x419196b0 in XRE_InitChildProcess (aArgc=2, aArgv=0xbeda99b0, aProcess=1078209536) at /home/work/B2G-hamachi-1.3/gecko/toolkit/xre/nsEmbedFunctions.cpp:516
#40 0x00008894 in main (argc=7, argv=0xbeda9a34) at /home/work/B2G-hamachi-1.3/gecko/ipc/app/MozillaRuntimeMain.cpp:137
(gdb) print driver_absolute_path
$1 = "/system/lib/egl/libGLES_android.so\000\276\270fھ\260\344E@HgھH\232(D\001\344E@s0\336A\003\000\000\000\001\000\000\000\254\344E@hfھ\000\062(D\001gھ\254\344E@\240\344E@Q\000\000\000Hgھ,\241\231B\bgھ\fgھ\330mھ\030gھ\203s\336A%\276\324AQ", '\000' <repeats 11 times>"\300, hھ", '\000' <repeats 12 times>, "8gھ\334hھ@\254\212Bpsھ\345\377\377\377\340\344E@\001QaE\210gھ\214gھ\001\001\000\000\001gھ\315h\323@\001gھ\210gھ\320gھ\204hھ\270\227\231B\001\000\000\000\304\344E@0tھ\000\246*D\001\274\335A4eھ\000\000\000\000\000gھ\b\000\000\000\260mھ\260mھ\202\377\377\377\063\350\330A\001nھTg"...
(gdb) n

Program received signal SIGSEGV, Segmentation fault.
0xb0005496 in unwind_phase2_forced (ucbp=0x0, entry_vrs=<value optimized out>, resuming=12288)
    at /tmp/android-build-bb7e003d31d08f72cabc269a652912b7/src/build/../gcc/gcc-4.4.3/libgcc/../gcc/config/arm/unwind-arm.c:717
717	/tmp/android-build-bb7e003d31d08f72cabc269a652912b7/src/build/../gcc/gcc-4.4.3/libgcc/../gcc/config/arm/unwind-arm.c: No such file or directory.
	in /tmp/android-build-bb7e003d31d08f72cabc269a652912b7/src/build/../gcc/gcc-4.4.3/libgcc/../gcc/config/arm/unwind-arm.c

Comment 18

[Dave Hylands [:dhylands]]

Oh yeah - this was running whatever the latest on v1.3 for hamachi

Unassigning myself and ni overholt to reassign to somebody in graphics...

Comment 19

[Dave Hylands [:dhylands]]

Is this code path even valid in a child process? I'm not familiar enough with the graphics stuff to know whether this code should even be expected to work in a child process (as I mentioned before, this all happens in the Communications app, NOT the main app).

Comment 20

[Andrew Overholt [:overholt]]

Milan, can you fit this into the graphics team's work list?

Comment 21

[Milan Sreckovic [:milan] (needinfo for best results)]

Hmm, part of the bug update stayed in my head.  CC-d Kats, just in case this is related to the event issue he's looking at.  Jeff, can you take a peak at comment 17/18 and shed some light for us?

Comment 22

[Kartikaya Gupta (email:[e-mail])]

Doesn't appear related to anything I'm currently looking at.

Comment 23

[Jeff Muizelaar [:jrmuizel]]

So there doesn't look to be anything unusual going on here. We're just loading up the gl driver for our first use of it. Can you get more information about the actual crash?

Comment 24

[Dave Hylands [:dhylands]]

(In reply to Jeff Muizelaar [:jrmuizel] from comment #23)
> So there doesn't look to be anything unusual going on here. We're just
> loading up the gl driver for our first use of it. Can you get more
> information about the actual crash?

I'm not sure what kind of more information you want?

Now I'm sooo confused.

If I run b2g under the debugger, then I get the weirdness where it's crashing while loading EGL.

If I don't run it under the debugger, then it loads EGL fine, takes the snapshot and I see an assert:

Assertion failure: mHandleCreatedByOtherProcessWasUsed, at /home/work/B2G-hamachi-1.3/gecko/ipc/glue/FileDescriptor.cpp:69

I'll see if I can figure out anything else.

Comment 25

[Dave Hylands [:dhylands]]

Yeah - adding some code and putting a breakpoint in FileDescriptor.cpp it still crashes while loading the EGL library, and even just attaching via the debugger and letting the program run with no breakpoints does the crash at library load time.

If I run the program not under the debugger, then it loads the EGL library fine, and does the extra prints and hits the assert in FileDescriptor.cpp

So it seems that the EGL library thing is just red-herring and some type of interaction between the debugger and the EGL library.

Comment 26

[jian.bai]

Attachment: crash reporter first boot up after fota upgrade

Hi try it locally in those days and found another crash. It may be related to this crash.

Comment 27

[jian.bai]

Comment on attachment 8434616 [details]
crash reporter  first boot up after fota upgrade

[Approval Request Comment]
Regression caused by (bug #): 
User impact if declined: 
Testing completed (on m-c, etc.): 
Risk to taking this patch (and alternatives if risky):
String or IDL/UUID changes made by this patch:

NOTE: This flag is now for security issues only. Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 
User impact if declined: 
Testing completed: 
Risk to taking this patch (and alternatives if risky): 
String or UUID changes made by this patch:

Comment 28

[Jason Smith [:jsmith]]

Comment on attachment 8434616 [details]
crash reporter  first boot up after fota upgrade

These flags are not appropriate for a crash stack.

Comment 29

[jian.bai]

Attachment: crash reporter first boot up after fota upgrade (reback some patch)

please do it as soon as possible

Thanks

Comment 30

[Andrew Overholt [:overholt]]

I suspect it's you that's being asked for more info here, Dave.

Comment 31

[Dave Hylands [:dhylands]]

Reassigning back to me. I'm still investigating.

Comment 32

[Dave Hylands [:dhylands]]

Attachment: Crash at end of FTU app (on hamachi)

This is the only crash I've been able to get out of my hamachi. The callstack here doesn't look at all like the original one reported, so I'm going to open a new bug.

I think I'll need a Sora phone, with the builds that are crashing, and instructions on creating those builds before I can proceed any further.

Comment 33

[Dave Hylands [:dhylands]]

I filed bug 1023513 to cover the crash that I saw on my hamachi.

It would be good if the reporter could try the patch in that bug and report back on whether it solves the problem in this bug.

Comment 34

[Dave Hylands [:dhylands]]

Actually, it looks like bug 918595 is a better fix for the problem (I'll probably close 1023513 as invalid)

So it would good to know if bug 918595 (which wasn't backported to 1.3) fixes this problem.

Comment 35

[jian.bai]

Attachment: TabChild.cpp(Modified)

Our source code is based on AU_LINUX_GECKO_B2G_JB_3.2.01.03.00.112.301. The file(TabChild.cpp) is very difference from file modified by Bug 9918595. But we still added the patch into the TabChild.cpp and had a try. The problem is still exist.

Comment 36

[jian.bai]

Attachment: it crash again,but it is different from before

It is crashed dump file


(In reply to jian.bai from comment #35)
> Created attachment 8438189 [details]
> TabChild.cpp(Modified)
> 
> Our source code is based on AU_LINUX_GECKO_B2G_JB_3.2.01.03.00.112.301. The
> file(TabChild.cpp) is very difference from file modified by Bug 9918595. But
> we still added the patch into the TabChild.cpp and had a try. The problem is
> still exist.

Comment 37

[Dave Hylands [:dhylands]]

The crash address in the second example is 0x0070697e which looks like the end of an ASCII string "~iF" (might just be a coincidence).

I'm unable to reproduce anymore crashes on my Hamachi 1.3 phone, so before I can investigate this further I'll need to be able to build a copy of FirefoxOS for the Sora (i.e. I'll need a copy of the source actually used to build gecko and gaia which will be our tree plus whatever modifications the vendor made), and get a Sora phone with an appropriate image on it.

Comment 38

[jian.bai]

Hi 

Because The file TabChild.cp based on partch of bug 918595 is different from our source code. This is our modified:https://bugzilla.mozilla.org/attachment.cgi?id=8438837.Can you help us to review it ? Thanks.
By the way, we found a strange phenomenon: we crated a file(data/fota/result.txt) and wrote something into it when the fota upgrade is finished,but when the phone start-up, we found the file is missed. I sure the file we didn't delete it.

Thanks

(In reply to Dave Hylands [:dhylands] from comment #37)
> The crash address in the second example is 0x0070697e which looks like the
> end of an ASCII string "~iF" (might just be a coincidence).
> 
> I'm unable to reproduce anymore crashes on my Hamachi 1.3 phone, so before I
> can investigate this further I'll need to be able to build a copy of
> FirefoxOS for the Sora (i.e. I'll need a copy of the source actually used to
> build gecko and gaia which will be our tree plus whatever modifications the
> vendor made), and get a Sora phone with an appropriate image on it.

Comment 39

[jian.bai]

Hi Dave

I'm sad to tell you the source code we couldn't provide to. But we can provide some file.

Jian Bai

Comment 40

[jian.bai]

TabChild.cpp :  https://bug1017504.bugzilla.mozilla.org/attachment.cgi?id=8438189

(In reply to jian.bai from comment #38)
> Hi 
> 
> Because The file TabChild.cp based on partch of bug 918595 is different from
> our source code. This is our
> modified:https://bugzilla.mozilla.org/attachment.cgi?id=8438837.Can you help
> us to review it ? Thanks.
> By the way, we found a strange phenomenon: we crated a
> file(data/fota/result.txt) and wrote something into it when the fota upgrade
> is finished,but when the phone start-up, we found the file is missed. I sure
> the file we didn't delete it.
> 
> Thanks
> 
> (In reply to Dave Hylands [:dhylands] from comment #37)
> > The crash address in the second example is 0x0070697e which looks like the
> > end of an ASCII string "~iF" (might just be a coincidence).
> > 
> > I'm unable to reproduce anymore crashes on my Hamachi 1.3 phone, so before I
> > can investigate this further I'll need to be able to build a copy of
> > FirefoxOS for the Sora (i.e. I'll need a copy of the source actually used to
> > build gecko and gaia which will be our tree plus whatever modifications the
> > vendor made), and get a Sora phone with an appropriate image on it.

Comment 41

[Andrew Overholt [:overholt]]

Attachment: TabChild diff

This is what I get as a diff between my b2g28_v1_3 branch and the attachment from comment 40.

Comment 42

[Dave Hylands [:dhylands]]

(In reply to Andrew Overholt [:overholt] from comment #41)
> Created attachment 8440082 [details] [diff] [review]
> TabChild diff
> 
> This is what I get as a diff between my b2g28_v1_3 branch and the attachment
> from comment 40.

I was going to say that's just the patch from bug 918595, but it must have been hand typed? Because it doesn't quite match.

For example, the for loop just has for (...stuff..; ..stuff...; index) but the real patch has index++

It isn't clear to me why the patch from bug 918595 wasn't just applied directly.

I was able to do the following:
> wget -O bug-918595.patch 'https://bugzilla.mozilla.org/attachment.cgi?id=8412884'
> patch -p1 < bug-918595.patch 

and it applied cleanly:

> patching file dom/ipc/TabChild.cpp
> Hunk #1 succeeded at 1207 (offset 3 lines).

It would be good if the reporter could undo his patch, and apply the patch from bug 918995 using the above commands (from within the gecko directory).

Comment 43

[jian.bai]

Attachment: crashreporter11(after apply the patch)

Hi 

I apply the patch and had a try. Another crash was occured.

Comment 44

[jian.bai]

Attachment: logcat_log

This is the crash adb log. There is something:

06-11 15:36:12.399 I/Gecko   (  905): 
06-11 15:36:12.399 I/Gecko   (  905): ###!!! [Child][MessageChannel::SendAndWait] Error: Channel error: cannot send/recv
06-11 15:36:12.399 I/Gecko   (  905): 
06-11 15:36:12.399 I/Gecko   (  905): [Child 905] ###!!! ABORT: constructor for actor failed: file /local/code/soul3.5/soul3.5_0528/out/target/product/msm8610/obj/objdir-gecko/ipc/ipdl/PLayerTransactionChild.cpp, line 122
06-11 15:36:12.399 I/Gecko   (  905): 
06-11 15:36:12.399 I/Gecko   (  905): ###!!! [Child][MessageChannel] Error: Channel error: cannot send/recv
06-11 15:36:12.399 I/Gecko   (  905):

Comment 45

[Dave Hylands [:dhylands]]

Those messages are all from the parent after the child crashed.

Comment 46

[jian.bai]

Attachment: downCrash00

When I downloaded fota upgrade package for some times, I shut down the phone,the crash occured.

Comment 47

[Jason Smith [:jsmith]]

Comment on attachment 8444181 [details]
downCrash00

Please don't set approval flags on attachments that aren't patches.

Comment 48

[Hema Koka [:hema]]

(In reply to jian.bai from comment #46)
> Created attachment 8444181 [details]
> downCrash00
> 
> When I downloaded fota upgrade package for some times, I shut down the
> phone,the crash occured.

Is this a new crash? Can you please provide more information so we can take the right action?

Comment 49

[jian.bai]

Yes,it is a new crash.I don't patch TabChild diff. I just downloaded my fota upgrad package and deleted it for some times.Then shut down ,So the crash was occured.

Comment 50

[Vance Chen [:vchen][skype:vance.lucida][[e-mail]]]

Hi Bai Jian, per discussion, for this new crash issue please submit another Bug ID if you want Mozilla to help to check. As to the original crash mentioned in Comment#0, it is caused by your own patch and now has been solved. So lets close this issue and track the new crash in another one

Comment 51

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

bug 1031224 was filed as followup.

Comment 52

[jian.bai]

Attachment: This bug come back

Hi, After last version, the rate is lower.But it is still.

Comment 53

[jian.bai]

Attachment: After I omitted our code. The crash is still.

The crash is still.Please help

Comment 54

[Dave Hylands [:dhylands]]

(In reply to jian.bai from comment #53)
> Created attachment 8449920 [details]
> After I omitted our code. The crash is still.
> 
> The crash is still.Please help

This is a crash but it is totally different from the earlier crashes reported in this bug.
Please open a new bug.

Comment 55

[jian.bai]

a new bug:https://bugzilla.mozilla.org/show_bug.cgi?id=1033966

(In reply to Dave Hylands [:dhylands] from comment #54)
> (In reply to jian.bai from comment #53)
> > Created attachment 8449920 [details]
> > After I omitted our code. The crash is still.
> > 
> > The crash is still.Please help
> 
> This is a crash but it is totally different from the earlier crashes
> reported in this bug.
> Please open a new bug.

Comment 56

[sync-1]

It has been fixed. Please close it