Prevention and detection of heap corruption and memory leakage with OpenBSD’s memory allocator security mitigations
NOTE: Some MALLOC_OPTIONS such as Dump (D) are introduced in the latest release (OpenBSD 7.4).
Lately I’ve been exploring functionality and possibilities of various memory allocators to see which one provides decent level of safety for the average developer. Talking about safety in memory allocators, one has to have in mind that there is an issue of performance vs security and when designing these complicated algorithms one has to think about proper balance between the two. For example, it is convenient and quite speedy to allocate pages or chunks of memory in contiguous fashion but quite unsafe if out-of-bounds write occurs as data gets overwritten unintentionally (heap buffer overflow) opening doors for serious bugs that can result in code execution. Better design choice would be to randomize the allocation and scatter those pages and chunks throughout the process’s virtual memory space but there’s a price to pay: random allocation requires separate data structure holding those addresses and now allocator has to deal both with memory objects as well as their meta data each time malloc(3) and free(3) is called which certainly reflects on the performance. Nevertheless, as security people, we prefer to swap the speed for consistency and memory safety, and that is why I decided to show all the goodies that otto malloc, modern memory allocator designed and implemented by Otto Moerbeek for OpenBSD (operating system that I am huge fan of and been using it since 3.2) provides. Known for its proactive security and strict code audits, this 4.4BSD-based UNIX is the prime choice for those who want to run a stable and “secure by default” system on many architectures. As security philosophy is deeply embedded into OpenBSD, it isn’t strange that memory allocators were also improving throughout the releases introducing new security mitigations.
Let’s jump into code and show what options OpenBSD malloc provides in malloctest.c:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char *argv[])
{
u_char *buff = malloc(20);
u_char *buff1 = malloc(20);
memset(buff, 'A', 60);
free(buff);
free(buff1);
u_int *pi = NULL;
for (int i=0; i<5; i++)
{
pi = malloc(500);
}
free(pi);
return 0;
}We can see that this code suffers from buffer overflow and memory leaks.
Randomization
In line 7 there’s an allocation of 20 bytes for buffer pointed to by pointer buff that is getting overwritten behind the boundaries with call to memset(3). Now let’s think for a second about the values that buff1 holds after the out-of-bound write. In order to compare, we will execute same code on Kali Linux and analyze memory allocations of buff and buff1 in gdb:
(gdb) b 9 Breakpoint 1 at 0x1184: file malloctest.c, line 9. (gdb) r Starting program: /home/rsrdjan/prog/malloc-tests/malloctest [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Breakpoint 1, main (argc=1, argv=0x7fffffffdef8) at malloctest.c:9 9 memset(buff, 'A', 60); (gdb) info locals buff = 0x5555555592a0 "" buff1 = 0x5555555592c0 "" pi = 0x0 (gdb) n 10 free(buff); (gdb) info locals buff = 0x5555555592a0 'A' <repeats 60 times> buff1 = 0x5555555592c0 'A' <repeats 28 times> pi = 0x0 (gdb) x/20xb buff 0x5555555592a0: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x5555555592a8: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x5555555592b0: 0x41 0x41 0x41 0x41 (gdb) x/20xb buff1 0x5555555592c0: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x5555555592c8: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x5555555592d0: 0x41 0x41 0x41 0x41 (gdb)
What can we observe? Linux malloc(3) is allocating memory in contiguous fashion (buff and buff1 chunks are stored one after another, memset(3) overwrites all values in buff, buff1 + 20 bytes with 0x41). Now let see what is happening with OpenBSD:
(gdb) b 9 Breakpoint 1 at 0x1ac3: file malloctest.c, line 9. (gdb) r Starting program: /home/rsrdjan/prog/malloc-tests/malloctest Breakpoint 1, main (argc=1, argv=0x71c3cf013798) at malloctest.c:9 9 memset(buff, 'A', 60); (gdb) info locals buff = 0x9c769841e80 '\337' <repeats 200 times>... buff1 = 0x9c769841440 '\337' <repeats 200 times>... pi = 0x9c7517997c0 <_dl_dtors> (gdb) n 10 free(buff); (gdb) info locals buff = 0x9c769841e80 'A' <repeats 60 times>, '\337' <repeats 140 times>... buff1 = 0x9c769841440 '\337' <repeats 200 times>... pi = 0x9c7517997c0 <_dl_dtors> (gdb) x/40xb buff 0x9c769841e80: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x9c769841e88: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x9c769841e90: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x9c769841e98: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x9c769841ea0: 0x41 0x41 0x41 0x41 0x41 0x41 0x41 0x41 (gdb) x/20xb buff1 0x9c769841440: 0xdf 0xdf 0xdf 0xdf 0xdf 0xdf 0xdf 0xdf 0x9c769841448: 0xdf 0xdf 0xdf 0xdf 0xdf 0xdf 0xdf 0xdf 0x9c769841450: 0xdf 0xdf 0xdf 0xdf (gdb)
Here we’re inspecting the contents of memory in 40 byte range after buff and it is clear that memset(3) writes behind the boundary, so clearly there’s an buffer overflow. But, as buff1 (0x9c769841440) isn’t located next to buff (0x9c769841e80) it doesn’t get overwritten. As a matter of fact, OpenBSD malloc(3) enables ASLR on the heap by default and it cannot be disabled which greatly reduces the possibility of successful heap exploitation.
Canaries
What OpenBSD memory allocator makes great is the fact that it provides developers with options that can help them in preventing and sanitizing heap corruption bugs not only during development process, but also in the production stage. One of those mitigations is the introduction of canaries, randomly chosen byte values written immediately behind the boundary of allocated buffer that, when gets overwritten, indicates BOF state and triggers SIGABRT. Note that there are several ways by which you can enable or disable malloc options (in this order): by setting vm.malloc_conf system variable with sysctl(2), by setting environment variable MALLOC_OPTIONS, and by explicitly setting global variable malloc_options in the program. Canaries are enabled by setting any of these to ‘C’ and disabled with lower-case letter ‘c’. Presence of malloc(3) options are checked when free(3) is called.
Following our code in malloctest.c, let see how this works:
openbsd$ export MALLOC_OPTIONS="C"
openbsd$ ./malloctest
malloctest(31676) in free(): canary corrupted 0x56d816d21a0 0x14@0x14
Abort trap (core dumped)
openbsd$ gdb malloctest malloctest.core
GNU gdb (GDB) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from malloctest...
[New process 234638]
Core was generated by `malloctest'.
Program terminated with signal SIGABRT, Aborted.
#0 thrkill () at /tmp/-:3
3 /tmp/-: No such file or directory.
(gdb) bt full
#0 thrkill () at /tmp/-:3
No locals.
#1 0xa3179372836b9ffe in ?? ()
No symbol table info available.
#2 0x0000056d6cd144b2 in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51
sa = {__sigaction_u = {__sa_handler = 0x3000000010, __sa_sigaction = 0x3000000010}, sa_mask = 1553334416,
sa_flags = 31891}
mask = 4294967263
#3 0x0000056d6cd2a81e in wrterror (d=0x56d6d8b5998, msg=0x56d6cc77f02 "canary corrupted %p %#tx@%#zx%s")
at /usr/src/lib/libc/stdlib/malloc.c:343
ap = {{gp_offset = 48, fp_offset = 48, overflow_arg_area = 0x7c935c960190, reg_save_area = 0x7c935c960090}}
saved_errno = 0
#4 0x0000056d6cd2f6c7 in validate_canary (d=0x0,
ptr=0x56d6ccd6739 <_thread_sys_sigprocmask+41> "r\vH\205\322t\002\211\002\061\300\353\017d\211\004% ",
sz=136972355370720, allocated=<optimized out>) at /usr/src/lib/libc/stdlib/malloc.c:1197
check_sz = <optimized out>
p = <optimized out>
q = <optimized out>
#5 find_chunknum (d=0x0, info=<optimized out>, ptr=<optimized out>, check=<optimized out>)
at /usr/src/lib/libc/stdlib/malloc.c:1222
chunknum = <optimized out>
#6 0x0000056d6cd2b85f in ofree (argpool=0x7c935c960220, p=0x56d816d21a0, clear=<optimized out>, check=0,
argsz=<optimized out>) at /usr/src/lib/libc/stdlib/malloc.c:1608
info = 0x6
i = <optimized out>
tmp = <optimized out>
saved_function = 0x56d546fae10 <environ> "\b\003\226\\\223|"
pool = 0x56d6d8b5998
r = 0x56d6ccd6739 <_thread_sys_sigprocmask+41>
--Type <RET> for more, q to quit, c to continue without paging--c
sz = 32
#7 0x0000056d6cd2b553 in _libc_free (ptr=0x56d816d21a0) at /usr/src/lib/libc/stdlib/malloc.c:1678
saved_errno = 0
d = 0x56d6d8b5998
#8 0x0000056ac49fbae6 in main (argc=1, argv=0x7c935c9602f8) at malloctest.c:10
buff = 0x56d816d21a0 'A' <repeats 60 times>, '\337' <repeats 140 times>...
buff1 = 0x56d81699820 '\337' <repeats 20 times>, '~' <repeats 12 times>, '\337' <repeats 168 times>...
pi = 0x56d54602ef0 <_dl_dtors>
(gdb)We can observe that process received SIGABRT signal. Full back trace shows that function validate_canary in malloc.c failed and called wrterror(), leading to abort(3). If we want to see how the checking is done, we can analyze lines 12-19.
static void
validate_canary(struct dir_info *d, u_char *ptr, size_t sz, size_t allocated)
{
size_t check_sz = allocated - sz;
u_char *p, *q;
if (check_sz > CHUNK_CHECK_LENGTH)
check_sz = CHUNK_CHECK_LENGTH;
p = ptr + sz;
q = p + check_sz;
while (p < q) {
if (*p != (u_char)mopts.chunk_canaries && *p != SOME_JUNK) {
wrterror(d, "canary corrupted %p %#tx@%#zx%s",
ptr, p - ptr, sz,
*p == SOME_FREEJUNK ? " (double free?)" : "");
}
p++;
}
}Pointer ptr points to the beginning of buff, sz is the size of buff (20), while allocated is the size of the allocated chunk. While loop checks if any of the bytes from the end of the buff to the beginning of the next allocated chunk are different from mopts.chunk_canaries (holds the output value of previously called arc4random(3)) and SOME_JUNK macro (0xdb). If it is, it ends with abort(3).
Memory leakage
As we noted previously, for loop at lines 14 – 17 of malloctest.c leaks memory by assigning same pointer to newly allocated buffer each time it executes and as a result unreferenced memory never gets freed. How can this be detected?
OpenBSD malloc has the option D (dump) which, when enabled, utilizes utrace(2) syscall for adding records to the process trace which can be collected with ktrace(1) utility that enables kernel trace logging for the specified processes and writes it to the file ktrace.out. This dump can then be read with kdump(1).
This is how it works for our test program:
openbsd$ ktrace -tu ./malloctest
openbsd$ ls -la ktrace.out
-rw------- 1 rsrdjan rsrdjan 1046 Mar 16 22:40 ktrace.out
openbsd$ kdump -u malloc
******** Start dump malloctest *******
M=8 I=1 F=0 U=0 J=1 R=0 X=0 C=0 cache=64 G=0
Leak report:
f sum # avg
0x0 1536 3 512 addr2line -e . 0x0
0x4ba90b3b12 512 1 512 addr2line -e ./malloctest 0x1b12
******** End dump malloctest *******We’re using ktrace(1) with -tu switch, meaning we want to trace user data coming from utrace(2). kdump(1)‘s -u switch means that we would like to output data labeled ‘malloc’. Sum column of the report shows how many bytes of memory did our program leak. If we want to search for the exact line(s) of code that caused the leakage, on condition that we compiled the executable with debugging info (-g), we can use addr2line(1) command.
Use after free and double free
Consider the following code (uaftest.c):
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
typedef struct person{
char name[10];
u_int years;
}PER;
int main(int argc, char *argv[])
{
PER *ptPer = malloc(sizeof(PER));
strcpy(ptPer->name,"Srdjan");
ptPer->years = 41;
free(ptPer);
ptPer->years = 40;
PER *ptPer1 = malloc(sizeof(PER));
free(ptPer1);
return 0;
}In line 16 we have use after free vulnerability as we have freed memory pointed to by ptPer in line 15 and now we’re trying to write to that location another value. In order to detect UAF vulnerabilities we are going to set environment variable MALLOC_OPTIONS to ‘F‘ (“Freecheck” – enables more extensive double free and write after free detection). Let’s debug this code and find out what is happening:
(gdb) b 15 Breakpoint 1 at 0x1b00: file uaftest.c, line 15. (gdb) b 19 Breakpoint 2 at 0x1b22: file uaftest.c, line 19. (gdb) r Starting program: /home/rsrdjan/prog/malloc-tests/uaftest Breakpoint 1, main (argc=1, argv=0x724a95e4da78) at uaftest.c:15 15 free(ptPer); (gdb) info locals ptPer = 0x7f961f657e0 ptPer1 = 0x724a95e4da78 (gdb) x/4x ptPer 0x7f961f657e0: 0x6a647253 0xdf006e61 0xdfdfdfdf 0x00000029 (gdb) n 16 ptPer->years = 40; (gdb) x/4x ptPer 0x7f961f657e0: 0xdfdfdfdf 0xdfdfdfdf 0xdfdfdfdf 0xdfdfdfdf (gdb) n 18 PER *ptPer1 = malloc(sizeof(PER)); (gdb) x/4x ptPer 0x7f961f657e0: 0xdfdfdfdf 0xdfdfdfdf 0xdfdfdfdf 0x00000028 (gdb) c Continuing. Breakpoint 2, main (argc=1, argv=0x724a95e4da78) at uaftest.c:19 19 free(ptPer1); (gdb) n uaftest(36859) in free(): write after free 0x7f961f657e0 Program received signal SIGABRT, Aborted. thrkill () at /tmp/-:3 3 /tmp/-: No such file or directory. (gdb)
We are setting two breakpoints (calls to free(ptPer) and free(ptPer1)) and after first breakpoint hit we see that structure ptPer fields ptPer.name is indeed set to “Srdjan” (first two words 0x6a647253 and 0xdf006e61 – mind the endianness) and ptPer.years = 41 (0x29 in fourth word). Then we’re calling free(3) on ptPer and on the next step can see that memory is freed. Now things are getting interested: We’re writing a new value on ptPer.name (0x28 = 40 in decimal) although we have previously freed the memory at that location and it passes! Why? Because OpenBSD memory allocator implementation follows deferred coalescing policy, meaning that, because of performance reasons, free list is checked when next free(3) of previously allocated memory object is called. This is what happens when we execute next step after breakpoint 2 and program ends with SIGABRT.
Double free example is straightforward (dfreetest.c):
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char *argv[])
{
u_int *ptInt = malloc(sizeof(u_int));
*ptInt = 50;
free(ptInt);
u_int *ptInt1 = malloc(sizeof(u_int));
free(ptInt);
return 0;
}Execution ends with SIGABRT on second free(3) call in line 11 which shows typical programming mistake.
There are more malloc options for which you can find additional information in superb OpenBSD’s man pages. I encourage you to do so if you want to find out more about security mitigations of otto malloc. If you wish to enable most of them, I advise you to set your MALLOC_OPTIONS environment variable to ‘S’ and use it throughout the development or debugging process.