# You have an app that is segfaulting you as the dev are expected to be # able todo basic debugging on the app to take user submitted reports to # the next level (upstream). Thankfully the linux kernel is able to # write core dumps upon a given program crashing with signal 11/segfault # to the current directory. By default core dumping are disabled as they # can cause a harddrive to fill if not tended to. # There are two basic ways to invoke gdb. 'gdb ' then from # within gdb you pass 'run ' where are options passed to # the program upon startup. GDB (extracted from manpage) Here are some of the most frequently needed GDB commands: break [file:]function Set a breakpoint at function (in file). run [arglist] Start your program (with arglist, if specified). bt Backtrace: display the program stack. print expr Display the value of an expression. c Continue running your program (after stopping, e.g. at a breakpoint). next Execute next program line (after stopping); step over any function calls in the line. edit [file:]function look at the program line where it is presently stopped. list [file:]function type the text of the program in the vicinity of where it is presently stopped. step Execute next program line (after stopping); step into any function calls in the line. help [name] Show information about GDB command name, or general information about using GDB. quit Exit from GDB. ## Analyzing an initial segfault with gdb # First we need to know how to build our bins correctly for dedugging. # If you are using SSP(propolice) or PIE(position independent executables) # you would need to compile like the follwing user@devbox $ cc program.c -o program -fno-stack-protector -nopie -g3 -ggdb # else you can drop the -fno-stack-protector -nopie flags. user@devbox $ cc program.c -o program -g3 -ggdb # If you are running with PaX you will need to disable executable # memory protections to allow for the setting of breakpoints and to # allow the debuger to read from /proc/$pid/maps user@devbox ~/c/program $ /sbin/paxctl -permsx program user@devbox ~/c/program $ gdb -q ./program Using host libthread_db library "/lib/libthread_db.so.1". (gdb) b main Breakpoint 1 at 0x8048595: file program.c, line 23. (gdb) r 012345678790 Starting program: /home/user/c/program 012345678790 Breakpoint 1, main (argc=2, argv=0xbfffeb04, envp=0xbfffeb10, auxv=0x0) at program.c:23 23 char buf[10] = ""; (gdb) c Continuing. 0x8048578 main(argc=0xbfffeab0, argv=0xbfffeab4, envp=0xbfffeab8, auxv=0xbfffeabc) __guard=0x804a010 (Stack) Copying [12] of data into [10] of space Program exited normally. (gdb) r 01234567890123456789 Starting program: /home/user/c/program 01234567890123456789 Breakpoint 1, main (argc=2, argv=0xbfffe924, envp=0xbfffe930, auxv=0x0) at program.c:23 23 char buf[10] = ""; (gdb) step 25 char buf1[] = "01234567890123456789"; (gdb) step 26 printf("%p main(argc=%p, argv=%p, envp=%p, auxv=%p) __guard=%p\n", (gdb) step 0x8048578 main(argc=0xbfffe8d0, argv=0xbfffe8d4, envp=0xbfffe8d8, auxv=0xbfffe8dc) __guard=0x804a010 36 fflush(stdout); (gdb) step 40 printf("(Stack) "); (gdb) step 3 51 printf("Copying [%d] of data into [%d] of space\n", (gdb) step (Stack) Copying [20] of data into [10] of space 53 strcpy(buf, argv[1]); (gdb) step 59 return 0; (gdb) step 60 } (gdb) step 0x400427b6 in __libc_start_main () from /lib/libc.so.6 (gdb) step Single stepping until exit from function __libc_start_main, which has no line number information. Program exited normally. (gdb) r 0123456789012345678901234567890 Starting program: /home/user/c/program 0123456789012345678901234567890 Breakpoint 1, main (argc=2, argv=0xbfffebc4, envp=0xbfffebd0, auxv=0x0) at program.c:23 23 char buf[10] = ""; (gdb) step 5 0x8048578 main(argc=0xbfffeb70, argv=0xbfffeb74, envp=0xbfffeb78, auxv=0xbfffeb7c) __guard=0x804a010 42 if (argc > 1) { (gdb) step 43 if (strcmp(argv[1], "-a") == 0) { (gdb) step 51 printf("Copying [%d] of data into [%d] of space\n", (gdb) step (Stack) Copying [31] of data into [10] of space 53 strcpy(buf, argv[1]); (gdb) step 59 return 0; (gdb) step 60 } (gdb) step Warning: Cannot insert breakpoint 0. Error accessing memory address 0x2: Input/output error. (gdb) ## Ouch! is this a security problem? (gdb) r AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA The program being debugged has been started already. Start it from the beginning? (y or n) y Starting program: /home/user/c/program AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Breakpoint 1, main (argc=2, argv=0xbffff334, envp=0xbffff340, auxv=0x0) at program.c:23 23 char buf[10] = ""; (gdb) step 5 0x8048578 main(argc=0xbffff2e0, argv=0xbffff2e4, envp=0xbffff2e8, auxv=0xbffff2ec) __guard=0x804a010 42 if (argc > 1) { (gdb) step 43 if (strcmp(argv[1], "-a") == 0) { (gdb) s 51 printf("Copying [%d] of data into [%d] of space\n", (gdb) (Stack) Copying [41] of data into [10] of space 53 strcpy(buf, argv[1]); (gdb) 59 return 0; (gdb) 60 } (gdb) Warning: Cannot insert breakpoint 0. Error accessing memory address 0x41414141: Input/output error. (gdb) r AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA The program being debugged has been started already. Start it from the beginning? (y or n) y Starting program: /home/user/c/program AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Breakpoint 1, main (argc=2, argv=0xbfffe724, envp=0xbfffe730, auxv=0x0) at program.c:23 23 char buf[10] = ""; (gdb) c Continuing. 0x8048578 main(argc=0xbfffe6d0, argv=0xbfffe6d4, envp=0xbfffe6d8, auxv=0xbfffe6dc) __guard=0x804a010 (Stack) Copying [41] of data into [10] of space Program received signal SIGSEGV, Segmentation fault. 0x41414141 in ?? () # man 3 ascii shows us that 41 in Hex is the letter A and A is the # letter we used when passing to much data to the program. So not only # is this program faultly but that memory addresses can be overwritten. Oct Dec Hex Char 101 65 41 A # The above example is a harmless command line overflow and rarely would # pose a danger in the real world of gentoo. A bug should be filed none # the less. # Other useful debugging tools dev-util/ltrace ltrace shows runtime library call information for dynamically linked executables dev-util/strace A useful diagnostic, instructional, and debugging tool dev-util/valgrind An open-source memory debugger for x86-GNU/Linux and ppc-GNU/Linux The envionment variable LD_PRELOAD= can be quite helpful for quickly replacing a symbol in a program with one you define. This can save you also of time with compiling.