GDB Tips and Tricks #3: Saving and Restoring Breakpoints Using Files

You spent the last 10 minutes littering your application with breakpoints in all the places where you think bad things might be happening. Then you run your application. Maybe you missed something. Maybe you didn’t. But now after a few minutes of debugging, both your application and GDB appear to be in a funky state. What you want to do is just quit out of everything and start fresh with a new session. But what about all those breakpoints? Typing in the “b” commands was such a chore. Even if you could remember where they all were, the mere thought of doing so is exhausting. And what if you need to start over yet again? Surely there’s a better way.

Did you know you could save your breakpoints to a file? All you need to do is issue the “save breakpoints” command like so…

(gdb) save breakpoints my.brk
Saved to file 'my.brk'.

This will save all types of breakpoints (regular breakpoints, watchpoints, and catchpoints) to the file my.brk. You can in fact name the file whatever you’d like.

Later when you’re ready to reload your breakpoints, you can issue the source command.

(gdb) source my.brk

There’s nothing special about this breakpoints file. It’s just a text file containing a list of gdb commands separated by newlines. In fact, not only can you amend it manually with more breakpoint commands, but you can add in just about any other gdb command in the process.

It’s worth noting that the “source” command isn’t actually specific to breakpoints. It will execute anything in the specified file as written, which makes the “source” command a handy tool in its own right.

GDB Tips and Tricks #2: Setting Breakpoints with Regular Expressions

When debugging applications under gdb, the meek and mighty breakpoint is the cornerstone of just about everything we do. Breakpoints give us the ability to freeze time so we can inspect, and sometimes modify, the world in which our code is running. They can be used to “hit count” a body of count. You can even use breakpoints to automatically run commands within gdb.

So it’s no surprise that “break, or “b”, is one of the first few gdb commands we learn about. The problem with the “break” command, however, is that it allows you to set one breakpoint at a time (That is, unless you specify an overloaded function name as your break location. You’ll end up with multiple breakpoints in that case). If you need to set a lot of breakpoints, “break” can get very repetitive very fast.

Enter “rbreak”, or “rb”, to the rescue. The gdb “rbreak” command allows you to use grep-style regular expressions to set breakpoint locations. As an example, consider the following test fixture class.

struct TestFixture
{
    void setUp() { std::cout << "Method setUp()\n"; }
    void tearDown() { std::cout << "Method tearDown()\n"; }
    void testA() { std::cout << "Method testA()\n"; }
    void testB() { std::cout << "Method testB()\n"; }
    // ...
};

Let’s suppose that we want to add breakpoints on only the methods that start with “test”.

(gdb) rb TestFixture::test.*
Breakpoint 1 at 0x4008f2: file gdbtest.cpp, line 7.
void TestFixture::testA();
Breakpoint 2 at 0x400910: file gdbtest.cpp, line 8.
void TestFixture::testB();
(gdb) info breakpoints
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00000000004008f2 in TestFixture::testA() at gdbtest.cpp:7
2       breakpoint     keep y   0x0000000000400910 in TestFixture::testB() at gdbtest.cpp:8

This gave us two breakpoints in this example.

What if I want to set breakpoints for every function in a given file? While not really a regular expression, “rbreak” allows you set these like so.

(gdb) rb TestFixture.h:.
Breakpoint 1 at 0x4008b6: file TestFixture.h, line 5.
void TestFixture::setUp();
Breakpoint 2 at 0x4008d4: file TestFixture.h, line 6.
void TestFixture::tearDown();
Breakpoint 3 at 0x4008f2: file TestFixture.h, line 7.
void TestFixture::testA();
Breakpoint 4 at 0x400910: file TestFixture.h, line 8.
void TestFixture::testB();
(gdb) info breakpoints
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00000000004008b6 in TestFixture::setUp() at TestFixture.h:5
2       breakpoint     keep y   0x00000000004008d4 in TestFixture::tearDown() at TestFixture.h:6
3       breakpoint     keep y   0x00000000004008f2 in TestFixture::testA() at TestFixture.h:7
4       breakpoint     keep y   0x0000000000400910 in TestFixture::testB() at TestFixture.h:8

Note the single colon when using file names vs double colons when specifying class names.

There’s even a way to specify that you want breakpoints for all functions everywhere.

(gdb) rbreak .
Breakpoint 5 at 0x4008b6: file TestFixture.h, line 5.
void TestFixture::setUp();
Breakpoint 6 at 0x4008d4: file TestFixture.h, line 6.
void TestFixture::tearDown();
Breakpoint 7 at 0x4008f2: file TestFixture.h, line 7.
void TestFixture::testA();
Breakpoint 8 at 0x400910: file TestFixture.h, line 8.
void TestFixture::testB();
Breakpoint 9 at 0x4007e5: file gdbtest.cpp, line 4.
int main(int, char**);
Breakpoint 10 at 0x400899: file gdbtest.cpp, line 13.
static void _GLOBAL__sub_I_main();
Breakpoint 11 at 0x400865: file gdbtest.cpp, line 13.
static void __static_initialization_and_destruction_0(int, int);
Breakpoint 12 at 0x400640
<function, no debug info> _init;
Breakpoint 13 at 0x400670
<function, no debug info> std::ios_base::Init::Init()@plt;
Breakpoint 14 at 0x400680
<function, no debug info> __libc_start_main@plt;
Breakpoint 15 at 0x400690
<function, no debug info> __cxa_atexit@plt;
Breakpoint 16 at 0x4006a0
<function, no debug info> std::ios_base::Init::~Init()@plt;
Breakpoint 17 at 0x4006b0
<function, no debug info> std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*)@plt;
Breakpoint 18 at 0x4006c0
<function, no debug info> __stack_chk_fail@plt;
Breakpoint 19 at 0x4006e0
<function, no debug info> _start;
Breakpoint 20 at 0x400710
<function, no debug info> deregister_tm_clones;
Breakpoint 21 at 0x400750
<function, no debug info> register_tm_clones;
Breakpoint 22 at 0x400790
<function, no debug info> __do_global_dtors_aux;
Breakpoint 23 at 0x4007b0
<function, no debug info> frame_dummy;
Breakpoint 24 at 0x400930
<function, no debug info> __libc_csu_init;
Breakpoint 25 at 0x4009a0
<function, no debug info> __libc_csu_fini;
Breakpoint 26 at 0x4009a4
<function, no debug info> _fini;

This, of course, includes a lot of standard library stuff that you probably don’t care about. So it’s not as interesting as it first might seem.

“rbreak” can save you a lot of time typing and trying to remember exact function names. It’s a super-handy tool to keep in your toolbox.

(p.s., if you accidentally set more breakpoints than you had intended, you can delete a specific breakpoint by using the “delete breakpoint [breakpoint-number]” command. You can get the breakpoint number from “info breakpoints”. To delete all breakpoints, use the “delete breakpoints” command).

GDB Tips and Tricks #1: A Tale of Two Terminals

It’s occasionally handy to be able to separate an application’s console I/O from that of gdb. Perhaps you’re debugging a particularly chatty application that’s preventing you from seeing what’s going on with gdb. Or maybe you’re debugging an ncurses application, a situation where gdb’s terminal I/O can actually interfere with the application’s UI.

Fortunately, there’s a mechanism that allows your application’s console I/O to be handled in a completely different terminal than that of gdb. The gdb command is called “tty” (not to be confused with the command line tool “tty” which we’ll also be using here).

Step 1: Figure out the device file for the terminal where you want your app’s console I/O to be handled. In that terminal…

skirk@dormouse:~$ tty
/dev/pts/18

Note: It probably goes without saying, but the device path will vary from terminal to terminal.

Step 2: Put the shell in that terminal to sleep so it’s not competing with your application for I/O.

skirk@dormouse:~$ sleep 1000000

Step 3: In a second terminal, fire up gdb with your application.

skirk@dormouse:~$ gdb ./myapp
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./ttytest...done.
(gdb)

Step 4: Within gdb, set the device file path for terminal where your app’s I/O should be handled.

(gdb) tty /dev/pts/18

Step 5: Debug your app! Note: You may see a warning about GDB not being able to set the controlling terminal. This is safe to ignore.