1/* Copyright (C) 2012-2013 2 Free Software Foundation 3 4 This file is part of GCC. 5 6 GCC is free software; you can redistribute it and/or modify 7 it under the terms of the GNU General Public License as published by 8 the Free Software Foundation; either version 3, or (at your option) 9 any later version. 10 11 GCC is distributed in the hope that it will be useful, 12 but WITHOUT ANY WARRANTY; without even the implied warranty of 13 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 14 GNU General Public License for more details. 15 16 Under Section 7 of GPL version 3, you are granted additional 17 permissions described in the GCC Runtime Library Exception, version 18 3.1, as published by the Free Software Foundation. 19 20 You should have received a copy of the GNU General Public License and 21 a copy of the GCC Runtime Library Exception along with this program; 22 see the files COPYING3 and COPYING.RUNTIME respectively. If not, see 23 <http://www.gnu.org/licenses/>. */ 24 25/* This file is part of the vtable security feature implementation. 26 The vtable security feature is designed to detect when a virtual 27 call is about to be made through an invalid vtable pointer 28 (possibly due to data corruption or malicious attacks). The 29 compiler finds every virtual call, and inserts a verification call 30 before the virtual call. The verification call takes the actual 31 vtable pointer value in the object through which the virtual call 32 is being made, and compares the vtable pointer against a set of all 33 valid vtable pointers that the object could contain (this set is 34 based on the declared type of the object). If the pointer is in 35 the valid set, execution is allowed to continue; otherwise the 36 program is halted. 37 38 There are several pieces needed in order to make this work: 1. For 39 every virtual class in the program (i.e. a class that contains 40 virtual methods), we need to build the set of all possible valid 41 vtables that an object of that class could point to. This includes 42 vtables for any class(es) that inherit from the class under 43 consideration. 2. For every such data set we build up, we need a 44 way to find and reference the data set. This is complicated by the 45 fact that the real vtable addresses are not known until runtime, 46 when the program is loaded into memory, but we need to reference the 47 sets at compile time when we are inserting verification calls into 48 the program. 3. We need to find every virtual call in the program, 49 and insert the verification call (with the appropriate arguments) 50 before the virtual call. 4. We need some runtime library pieces: 51 the code to build up the data sets at runtime; the code to actually 52 perform the verification using the data sets; and some code to set 53 protections on the data sets, so they themselves do not become 54 hacker targets. 55 56 To find and reference the set of valid vtable pointers for any given 57 virtual class, we create a special global varible for each virtual 58 class. We refer to this as the "vtable map variable" for that 59 class. The vtable map variable has the type "void *", and is 60 initialized by the compiler to NULL. At runtime when the set of 61 valid vtable pointers for a virtual class, e.g. class Foo, is built, 62 the vtable map variable for class Foo is made to point to the set. 63 During compile time, when the compiler is inserting verification 64 calls into the program, it passes the vtable map variable for the 65 appropriate class to the verification call, so that at runtime the 66 verification call can find the appropriate data set. 67 68 The actual set of valid vtable pointers for a polymorphic class, 69 e.g. class Foo, cannot be built until runtime, when the vtables get 70 loaded into memory and their addresses are known. But the knowledge 71 about which vtables belong in which class' hierarchy is only known 72 at compile time. Therefore at compile time we collect class 73 hierarchy and vtable information about every virtual class, and we 74 generate calls to build up the data sets at runtime. To build the 75 data sets, we call one of the functions we add to the runtime 76 library, __VLTRegisterPair. __VLTRegisterPair takes two arguments, 77 a vtable map variable and the address of a vtable. If the vtable 78 map variable is currently NULL, it creates a new data set (hash 79 table), makes the vtable map variable point to the new data set, and 80 inserts the vtable address into the data set. If the vtable map 81 variable is not NULL, it just inserts the vtable address into the 82 data set. In order to make sure that our data sets are built before 83 any verification calls happen, we create a special constructor 84 initialization function for each compilation unit, give it a very 85 high initialization priority, and insert all of our calls to 86 __VLTRegisterPair into our special constructor initialization 87 function. */ 88 89/* This file contains the main externally visible runtime library 90 functions for vtable verification: __VLTChangePermission, 91 __VLTRegisterPair, and __VLTVerifyVtablePointer. It also contains 92 debug versions __VLTRegisterPairDebug and 93 __VLTVerifyVtablePointerDebug, which have extra parameters in order 94 to make it easier to debug verification failures. 95 96 The final piece of functionality implemented in this file is symbol 97 resolution for multiple instances of the same vtable map variable. 98 If the same virtual class is used in two different compilation 99 units, then each compilation unit will create a vtable map variable 100 for the class. We need all instances of the same vtable map 101 variable to point to the same (single) set of valid vtable 102 pointers for the class, so we wrote our own hashtable-based symbol 103 resolution for vtable map variables (with a tiny optimization in 104 the case where there is only one instance of the variable). 105 106 There are two other important pieces to the runtime for vtable 107 verification besides the main pieces that go into libstdc++.so: two 108 special tiny shared libraries, libvtv_init.so and libvtv_stubs.so. 109 libvtv_init.so is built from vtv_init.cc. It is designed to help 110 minimize the calls made to mprotect (see the comments in 111 vtv_init.cc for more details). Anything compiled with 112 "-fvtable-verify=std" must be linked with libvtv_init.so (the gcc 113 driver has been modified to do this). vtv_stubs.so is built from 114 vtv_stubs.cc. It replaces the main runtime functions 115 (__VLTChangePermissino, __VLTRegisterPair and 116 __VLTVerifyVtablePointer) with stub functions that do nothing. If 117 a programmer has a library that was built with verification, but 118 wishes to not have verification turned on, the programmer can link 119 in the vtv_stubs.so library. */ 120 121#include <stdlib.h> 122#include <stdio.h> 123#include <string.h> 124#if defined (__CYGWIN__) || defined (__MINGW32__) 125#include <windows.h> 126#include <winternl.h> 127#include <psapi.h> 128#else 129#include <execinfo.h> 130#endif 131 132#include <unistd.h> 133#if !defined (__CYGWIN__) && !defined (__MINGW32__) 134#include <sys/mman.h> 135#include <link.h> 136#endif 137#include <errno.h> 138#include <fcntl.h> 139#include <limits.h> 140 141/* For gthreads suppport */ 142#include <bits/c++config.h> 143#include <ext/concurrence.h> 144 145#include "vtv_utils.h" 146#include "vtv_malloc.h" 147#include "vtv_set.h" 148#include "vtv_map.h" 149#include "vtv_rts.h" 150#include "vtv_fail.h" 151 152#include "vtv-change-permission.h" 153 154#if defined (__CYGWIN__) || defined (__MINGW32__) 155// porting: fix link error to libc 156void __fortify_fail (const char * msg){ 157 OutputDebugString(msg); 158 abort(); 159} 160#else 161extern "C" { 162 163 /* __fortify_fail is a function in glibc that calls __libc_message, 164 causing it to print out a program termination error message 165 (including the name of the binary being terminated), a stack 166 trace where the error occurred, and a memory map dump. Ideally 167 we would have called __libc_message directly, but that function 168 does not appear to be accessible to functions outside glibc, 169 whereas __fortify_fail is. We call __fortify_fail from 170 __vtv_really_fail. We looked at calling __libc_fatal, which is 171 externally accessible, but it does not do the back trace and 172 memory dump. */ 173 174 extern void __fortify_fail (const char *) __attribute__((noreturn)); 175 176} /* extern "C" */ 177#endif 178 179/* The following variables are used only for debugging and performance 180 tuning purposes. Therefore they do not need to be "protected". 181 They cannot be used to attack the vtable verification system and if 182 they become corrupted it will not affect the correctness or 183 security of any of the rest of the vtable verification feature. */ 184 185unsigned int num_calls_to_regset = 0; 186unsigned int num_calls_to_regpair = 0; 187unsigned int num_calls_to_verify_vtable = 0; 188unsigned long long regset_cycles = 0; 189unsigned long long regpair_cycles = 0; 190unsigned long long verify_vtable_cycles = 0; 191 192/* Be careful about initialization of statics in this file. Some of 193 the routines below are called before any runtime initialization for 194 statics in this file will be done. For example, dont try to 195 initialize any of these statics with a runtime call (for ex: 196 sysconf). The initialization will happen after calls to the routines 197 to protect/unprotec the vtabla_map variables */ 198 199/* No need to mark the following variables with VTV_PROTECTED_VAR. 200 These are either const or are only used for debugging/tracing. 201 debugging/tracing will not be ON on production environments */ 202 203static const bool debug_hash = HASHTABLE_STATS; 204static const int debug_functions = 0; 205static const int debug_init = 0; 206static const int debug_verify_vtable = 0; 207 208#ifdef VTV_DEBUG 209static const int debug_functions = 1; 210static const int debug_init = 1; 211static const int debug_verify_vtable = 1; 212#endif 213 214/* Global file descriptor variables for logging, tracing and debugging. */ 215 216static int init_log_fd = -1; 217static int verify_vtable_log_fd = -1; 218 219/* This holds a formatted error logging message, to be written to the 220 vtable verify failures log. */ 221static char debug_log_message[1024]; 222 223 224#ifdef __GTHREAD_MUTEX_INIT 225static __gthread_mutex_t change_permissions_lock = __GTHREAD_MUTEX_INIT; 226#else 227static __gthread_mutex_t change_permissions_lock; 228#endif 229 230 231#ifndef VTV_STATS 232#define VTV_STATS 0 233#endif 234 235#if VTV_STATS 236 237static inline unsigned long long 238get_cycle_count (void) 239{ 240 return rdtsc(); 241} 242 243static inline void 244accumulate_cycle_count (unsigned long long *sum, unsigned long long start) 245{ 246 unsigned long long end = rdtsc(); 247 *sum = *sum + (end - start); 248} 249 250static inline void 251increment_num_calls (unsigned int *num_calls) 252{ 253 *num_calls = *num_calls + 1; 254} 255 256#else 257 258static inline unsigned long long 259get_cycle_count (void) 260{ 261 return (unsigned long long) 0; 262} 263 264static inline void 265accumulate_cycle_count (unsigned long long *sum __attribute__((__unused__)), 266 unsigned long long start __attribute__((__unused__))) 267{ 268 /* Do nothing. */ 269} 270 271static inline void 272increment_num_calls (unsigned int *num_calls __attribute__((__unused__))) 273{ 274 /* Do nothing. */ 275} 276 277#endif 278 279/* Types needed by insert_only_hash_sets. */ 280typedef uintptr_t int_vptr; 281 282/* The set of valid vtable pointers for each virtual class is stored 283 in a hash table. This is the hashing function used for the hash 284 table. For more information on the implementation of the hash 285 table, see the class insert_only_hash_sets in vtv_set.h. */ 286 287struct vptr_hash 288 { 289 /* Hash function, used to convert vtable pointer, V, (a memory 290 address) into an index into the hash table. */ 291 size_t 292 operator() (int_vptr v) const 293 { 294 const uint32_t x = 0x7a35e4d9; 295 const int shift = (sizeof (v) == 8) ? 23 : 21; 296 v = x * v; 297 return v ^ (v >> shift); 298 } 299 }; 300 301/* This is the memory allocator used to create the hash table data 302 sets of valid vtable pointers. We use VTV_malloc in order to keep 303 track of which pages have been allocated, so we can update the 304 protections on those pages appropriately. See the class 305 insert_only_hash_sets in vtv_set.h for more information. */ 306 307struct vptr_set_alloc 308 { 309 /* Memory allocator operator. N is the number of bytes to be 310 allocated. */ 311 void * 312 operator() (size_t n) const 313 { 314 return __vtv_malloc (n); 315 } 316 }; 317 318/* Instantiate the template classes (in vtv_set.h) for our particular 319 hash table needs. */ 320typedef insert_only_hash_sets<int_vptr, vptr_hash, vptr_set_alloc> vtv_sets; 321typedef vtv_sets::insert_only_hash_set vtv_set; 322typedef vtv_set * vtv_set_handle; 323typedef vtv_set_handle * vtv_set_handle_handle; 324 325/* Records for caching the section header information that we have 326 read out of the file(s) on disk (in dl_iterate_phdr_callback), to 327 avoid having to re-open and re-read the same file multiple 328 times. */ 329 330struct sect_hdr_data 331{ 332#if defined (__CYGWIN__) || defined (__MINGW32__) 333 uintptr_t dlpi_addr; /* The header address in the INFO record, 334 passed in from dl_iterate_phdr. */ 335 uintptr_t mp_low; /* Start address of the .vtable_map_vars 336 section in memory. */ 337#else 338 ElfW (Addr) dlpi_addr; /* The header address in the INFO record, 339 passed in from dl_iterate_phdr. */ 340 ElfW (Addr) mp_low; /* Start address of the .vtable_map_vars 341 section in memory. */ 342#endif 343 size_t mp_size; /* Size of the .vtable_map_vars section in 344 memory. */ 345}; 346 347/* Array for caching the section header information, read from file, 348 to avoid re-opening and re-reading the same file over-and-over 349 again. */ 350 351#define MAX_ENTRIES 250 352static struct sect_hdr_data vtv_sect_info_cache[MAX_ENTRIES] VTV_PROTECTED_VAR; 353 354unsigned int num_cache_entries VTV_PROTECTED_VAR = 0; 355 356/* This function takes the LOAD_ADDR for an object opened by the 357 dynamic loader, and checks the array of cached file data to see if 358 there is an entry with the same addres. If it finds such an entry, 359 it returns the record for that entry; otherwise it returns 360 NULL. */ 361 362#if defined (__CYGWIN__) || defined (__MINGW32__) 363struct sect_hdr_data * 364search_cached_file_data (uintptr_t load_addr) 365#else 366struct sect_hdr_data * 367search_cached_file_data (ElfW (Addr) load_addr) 368#endif 369{ 370 unsigned int i; 371 for (i = 0; i < num_cache_entries; ++i) 372 { 373 if (vtv_sect_info_cache[i].dlpi_addr == load_addr) 374 return &(vtv_sect_info_cache[i]); 375 } 376 377 return NULL; 378} 379 380/* This function tries to read COUNT bytes out of the file referred to 381 by FD into the buffer BUF. It returns the actual number of bytes 382 it succeeded in reading. */ 383 384static size_t 385ReadPersistent (int fd, void *buf, size_t count) 386{ 387 char *buf0 = (char *) buf; 388 size_t num_bytes = 0; 389 while (num_bytes < count) 390 { 391 int len; 392 len = read (fd, buf0 + num_bytes, count - num_bytes); 393 if (len < 0) 394 return -1; 395 if (len == 0) 396 break; 397 num_bytes += len; 398 } 399 400 return num_bytes; 401} 402 403/* This function tries to read COUNT bytes, starting at OFFSET from 404 the file referred to by FD, and put them into BUF. It calls 405 ReadPersistent to help it do so. It returns the actual number of 406 bytes read, or -1 if it fails altogether. */ 407 408static size_t 409ReadFromOffset (int fd, void *buf, const size_t count, const off_t offset) 410{ 411 off_t off = lseek (fd, offset, SEEK_SET); 412 if (off != (off_t) -1) 413 return ReadPersistent (fd, buf, count); 414 return -1; 415} 416 417/* The function takes a MESSAGE and attempts to write it to the vtable 418 memory protection log (for debugging purposes). If the file is not 419 open, it attempts to open the file first. */ 420 421static void 422log_memory_protection_data (char *message) 423{ 424 static int log_fd = -1; 425 426 if (log_fd == -1) 427 log_fd = __vtv_open_log ("vtv_memory_protection_data.log"); 428 429 __vtv_add_to_log (log_fd, "%s", message); 430} 431 432#if defined (__CYGWIN__) || defined (__MINGW32__) 433static void 434read_section_offset_and_length (char *name, 435 uintptr_t addr, 436 const char *sect_name, 437 int mprotect_flags, 438 off_t *sect_offset, 439 WORD *sect_len) 440{ 441 bool found = false; 442 struct sect_hdr_data *cached_data = NULL; 443 444 /* Check to see if we already have the data for this file. */ 445 cached_data = search_cached_file_data (addr); 446 447 if (cached_data) 448 { 449 *sect_offset = cached_data->mp_low; 450 *sect_len = cached_data->mp_size; 451 return; 452 } 453 454 // check for DOS Header magic bytes 455 if (*(WORD *)addr == 0x5A4D) 456 { 457 int name_len = strlen (sect_name); 458 int fd = -1; 459 460 /* Attempt to open the binary file on disk. */ 461 if (strlen (name) == 0) 462 { 463 return; 464 } 465 else 466 fd = open (name, O_RDONLY | O_BINARY); 467 468 if (fd != -1) 469 { 470 /* Find the section header information in memory. */ 471 PIMAGE_DOS_HEADER pDosHeader = (PIMAGE_DOS_HEADER)addr; 472 PIMAGE_NT_HEADERS pNtHeaders = (PIMAGE_NT_HEADERS)((char *)addr 473 + pDosHeader->e_lfanew); 474 PIMAGE_FILE_HEADER pFileHeader = &pNtHeaders->FileHeader; 475 476 DWORD PointerToStringTable = pFileHeader->PointerToSymbolTable 477 + (pFileHeader->NumberOfSymbols*0x12); 478 479 PIMAGE_SECTION_HEADER sect_hdr = 480 (PIMAGE_SECTION_HEADER)((char *)&pNtHeaders->OptionalHeader 481 + pFileHeader->SizeOfOptionalHeader); 482 483 /* Loop through all the section headers, looking for one whose 484 name is ".vtable_map_vars". */ 485 486 for (int i = 0; i < pFileHeader->NumberOfSections && !found; ++i) 487 { 488 char header_name[64]; 489 490 /* Check if we have to get the section name from the COFF string 491 table. */ 492 if (sect_hdr[i].Name[0] == '/') 493 { 494 if (atoi((const char*)sect_hdr[i].Name+1) == 0) 495 { 496 continue; 497 } 498 499 off_t name_offset = PointerToStringTable 500 + atoi((const char*)sect_hdr[i].Name+1); 501 502 size_t bytes_read = ReadFromOffset (fd, &header_name, 64, 503 name_offset); 504 505 VTV_ASSERT (bytes_read > 0); 506 } 507 else 508 { 509 memcpy (&header_name, sect_hdr[i].Name, 510 sizeof (sect_hdr[i].Name)); 511 } 512 513 if (memcmp (header_name, sect_name, name_len) == 0) 514 { 515 /* We found the section; get its load offset and 516 size. */ 517 *sect_offset = sect_hdr[i].VirtualAddress; 518 if (sect_hdr[i].Misc.VirtualSize % VTV_PAGE_SIZE != 0) 519 *sect_len = sect_hdr[i].Misc.VirtualSize + VTV_PAGE_SIZE 520 - (sect_hdr[i].Misc.VirtualSize % VTV_PAGE_SIZE); 521 else 522 *sect_len = sect_hdr[i].Misc.VirtualSize; 523 found = true; 524 } 525 } 526 close (fd); 527 } 528 } 529 530 if (*sect_offset != 0 && *sect_len != 0) 531 { 532 /* Calculate the page location in memory, making sure the 533 address is page-aligned. */ 534 uintptr_t start_addr = addr + *sect_offset; 535 *sect_offset = start_addr & ~(VTV_PAGE_SIZE - 1); 536 *sect_len = *sect_len - 1; 537 538 /* Since we got this far, we must not have found these pages in 539 the cache, so add them to it. NOTE: We could get here either 540 while making everything read-only or while making everything 541 read-write. We will only update the cache if we get here on 542 a read-write (to make absolutely sure the cache is writable 543 -- also the read-write pass should come before the read-only 544 pass). */ 545 if ((mprotect_flags & PROT_WRITE) 546 && num_cache_entries < MAX_ENTRIES) 547 { 548 vtv_sect_info_cache[num_cache_entries].dlpi_addr = addr; 549 vtv_sect_info_cache[num_cache_entries].mp_low = *sect_offset; 550 vtv_sect_info_cache[num_cache_entries].mp_size = *sect_len; 551 num_cache_entries++; 552 } 553 } 554} 555#else 556static void 557read_section_offset_and_length (struct dl_phdr_info *info, 558 const char *sect_name, 559 int mprotect_flags, 560 off_t *sect_offset, 561 ElfW (Word) *sect_len) 562{ 563 char program_name[PATH_MAX]; 564 char *cptr; 565 bool found = false; 566 struct sect_hdr_data *cached_data = NULL; 567 const ElfW (Phdr) *phdr_info = info->dlpi_phdr; 568 const ElfW (Ehdr) *ehdr_info = 569 (const ElfW (Ehdr) *) (info->dlpi_addr + info->dlpi_phdr[0].p_vaddr 570 - info->dlpi_phdr[0].p_offset); 571 572 573 /* Get the name of the main executable. This may or may not include 574 arguments passed to the program. Find the first space, assume it 575 is the start of the argument list, and change it to a '\0'. */ 576 snprintf (program_name, sizeof (program_name), program_invocation_name); 577 578 /* Check to see if we already have the data for this file. */ 579 cached_data = search_cached_file_data (info->dlpi_addr); 580 581 if (cached_data) 582 { 583 *sect_offset = cached_data->mp_low; 584 *sect_len = cached_data->mp_size; 585 return; 586 } 587 588 /* Find the first non-escaped space in the program name and make it 589 the end of the string. */ 590 cptr = strchr (program_name, ' '); 591 if (cptr != NULL && cptr[-1] != '\\') 592 cptr[0] = '\0'; 593 594 if ((phdr_info->p_type == PT_PHDR || phdr_info->p_type == PT_LOAD) 595 && (ehdr_info->e_shoff && ehdr_info->e_shnum)) 596 { 597 int name_len = strlen (sect_name); 598 int fd = -1; 599 600 /* Attempt to open the binary file on disk. */ 601 if (strlen (info->dlpi_name) == 0) 602 { 603 /* If the constructor initialization function was put into 604 the preinit array, then this function will get called 605 while handling preinit array stuff, in which case 606 program_invocation_name has not been initialized. In 607 that case we can get the filename of the executable from 608 "/proc/self/exe". */ 609 if (strlen (program_name) > 0) 610 { 611 if (phdr_info->p_type == PT_PHDR) 612 fd = open (program_name, O_RDONLY); 613 } 614 else 615 fd = open ("/proc/self/exe", O_RDONLY); 616 } 617 else 618 fd = open (info->dlpi_name, O_RDONLY); 619 620 if (fd != -1) 621 { 622 /* Find the section header information in the file. */ 623 ElfW (Half) strtab_idx = ehdr_info->e_shstrndx; 624 ElfW (Shdr) shstrtab; 625 off_t shstrtab_offset = ehdr_info->e_shoff + 626 (ehdr_info->e_shentsize * strtab_idx); 627 size_t bytes_read = ReadFromOffset (fd, &shstrtab, sizeof (shstrtab), 628 shstrtab_offset); 629 VTV_ASSERT (bytes_read == sizeof (shstrtab)); 630 631 ElfW (Shdr) sect_hdr; 632 633 /* This code will be needed once we have crated libvtv.so. */ 634 bool is_libvtv = false; 635 636 /* 637 if (strstr (info->dlpi_name, "libvtv.so")) 638 is_libvtv = true; 639 */ 640 641 /* Loop through all the section headers, looking for one whose 642 name is ".vtable_map_vars". */ 643 644 for (int i = 0; i < ehdr_info->e_shnum && !found; ++i) 645 { 646 off_t offset = ehdr_info->e_shoff + (ehdr_info->e_shentsize * i); 647 648 bytes_read = ReadFromOffset (fd, §_hdr, sizeof (sect_hdr), 649 offset); 650 651 VTV_ASSERT (bytes_read == sizeof (sect_hdr)); 652 653 char header_name[64]; 654 off_t name_offset = shstrtab.sh_offset + sect_hdr.sh_name; 655 656 bytes_read = ReadFromOffset (fd, &header_name, 64, name_offset); 657 658 VTV_ASSERT (bytes_read > 0); 659 660 if (memcmp (header_name, sect_name, name_len) == 0) 661 { 662 /* We found the section; get its load offset and 663 size. */ 664 *sect_offset = sect_hdr.sh_addr; 665 if (!is_libvtv) 666 *sect_len = sect_hdr.sh_size - VTV_PAGE_SIZE; 667 else 668 *sect_len = sect_hdr.sh_size; 669 found = true; 670 } 671 } 672 close (fd); 673 } 674 } 675 676 if (*sect_offset != 0 && *sect_len != 0) 677 { 678 /* Calculate the page location in memory, making sure the 679 address is page-aligned. */ 680 ElfW (Addr) start_addr = (const ElfW (Addr)) info->dlpi_addr 681 + *sect_offset; 682 *sect_offset = start_addr & ~(VTV_PAGE_SIZE - 1); 683 *sect_len = *sect_len - 1; 684 685 /* Since we got this far, we must not have found these pages in 686 the cache, so add them to it. NOTE: We could get here either 687 while making everything read-only or while making everything 688 read-write. We will only update the cache if we get here on 689 a read-write (to make absolutely sure the cache is writable 690 -- also the read-write pass should come before the read-only 691 pass). */ 692 if ((mprotect_flags & PROT_WRITE) 693 && num_cache_entries < MAX_ENTRIES) 694 { 695 vtv_sect_info_cache[num_cache_entries].dlpi_addr = info->dlpi_addr; 696 vtv_sect_info_cache[num_cache_entries].mp_low = *sect_offset; 697 vtv_sect_info_cache[num_cache_entries].mp_size = *sect_len; 698 num_cache_entries++; 699 } 700 } 701} 702#endif 703 704#if defined (__CYGWIN__) || defined (__MINGW32__) 705/* This function is used to iterate over all loaded modules and searches 706 for a section called ".vtable_map_vars". The only interaction with 707 the binary file on disk of the module is to read section names in the 708 COFF string table. If the module contains a ".vtable_map_vars" section, 709 read section offset and size from the section header of the loaded module. 710 Call 'mprotect' on those pages, setting the protection either to 711 read-only or read-write, depending on what's in data. 712 The calls to change the protection occur in vtv_unprotect_vtable_vars 713 and vtv_protect_vtable_vars. */ 714 715static int 716iterate_modules (void *data) 717{ 718 int * mprotect_flags = (int *) data; 719 off_t map_sect_offset = 0; 720 WORD map_sect_len = 0; 721 char buffer[1024]; 722 const char *map_sect_name = VTV_PROTECTED_VARS_SECTION; 723 HMODULE hMods[1024]; 724 HANDLE hProcess; 725 DWORD cbNeeded; 726 727 hProcess = GetCurrentProcess (); 728 729 if (NULL == hProcess) 730 return 0; 731 732 if (EnumProcessModules (hProcess, hMods, sizeof (hMods), &cbNeeded)) 733 { 734 /* Iterate over all loaded modules. */ 735 for (unsigned int i = 0; i < (cbNeeded / sizeof (HMODULE)); i++) 736 { 737 char szModName[MAX_PATH]; 738 739 if (GetModuleFileNameExA (hProcess, hMods[i], szModName, 740 sizeof (szModName))) 741 { 742 map_sect_offset = 0; 743 map_sect_len = 0; 744 read_section_offset_and_length (szModName, 745 (uintptr_t) hMods[i], 746 map_sect_name, 747 *mprotect_flags, 748 &map_sect_offset, 749 &map_sect_len); 750 751 if (debug_functions) 752 { 753 snprintf (buffer, sizeof(buffer), 754 " Looking at load module %s to change permissions to %s\n", 755 szModName, 756 (*mprotect_flags & PROT_WRITE) ? "READ/WRITE" : "READ-ONLY"); 757 log_memory_protection_data (buffer); 758 } 759 760 /* See if we actually found the section. */ 761 if (map_sect_offset && map_sect_len) 762 { 763 unsigned long long start; 764 int result; 765 766 if (debug_functions) 767 { 768 snprintf (buffer, sizeof (buffer), 769 " (%s): Protecting %p to %p\n", 770 szModName, 771 (void *) map_sect_offset, 772 (void *) (map_sect_offset + map_sect_len)); 773 log_memory_protection_data (buffer); 774 } 775 776 /* Change the protections on the pages for the section. */ 777 778 start = get_cycle_count (); 779 result = mprotect ((void *) map_sect_offset, map_sect_len, 780 *mprotect_flags); 781 accumulate_cycle_count (&mprotect_cycles, start); 782 if (result == -1) 783 { 784 if (debug_functions) 785 { 786 snprintf (buffer, sizeof (buffer), 787 "Failed called to mprotect for %s error: ", 788 (*mprotect_flags & PROT_WRITE) ? 789 "READ/WRITE" : "READ-ONLY"); 790 log_memory_protection_data (buffer); 791 perror(NULL); 792 } 793 VTV_error(); 794 } 795 else 796 { 797 if (debug_functions) 798 { 799 snprintf (buffer, sizeof (buffer), 800 "mprotect'ed range [%p, %p]\n", 801 (void *) map_sect_offset, 802 (char *) map_sect_offset + map_sect_len); 803 log_memory_protection_data (buffer); 804 } 805 } 806 increment_num_calls (&num_calls_to_mprotect); 807 /* num_pages_protected += (map_sect_len + VTV_PAGE_SIZE - 1) 808 / VTV_PAGE_SIZE; */ 809 num_pages_protected += (map_sect_len + 4096 - 1) / 4096; 810 continue; 811 } 812 } 813 } 814 } 815 816 CloseHandle(hProcess); 817 818 return 0; 819} 820#else 821/* This is the callback function used by dl_iterate_phdr (which is 822 called from vtv_unprotect_vtable_vars and vtv_protect_vtable_vars). 823 It attempts to find the binary file on disk for the INFO record 824 that dl_iterate_phdr passes in; open the binary file, and read its 825 section header information. If the file contains a 826 ".vtable_map_vars" section, read the section offset and size. Use 827 the section offset and size, in conjunction with the data in INFO 828 to locate the pages in memory where the section is. Call 829 'mprotect' on those pages, setting the protection either to 830 read-only or read-write, depending on what's in DATA. */ 831 832static int 833dl_iterate_phdr_callback (struct dl_phdr_info *info, size_t, void *data) 834{ 835 int * mprotect_flags = (int *) data; 836 off_t map_sect_offset = 0; 837 ElfW (Word) map_sect_len = 0; 838 char buffer[1024]; 839 char program_name[1024]; 840 const char *map_sect_name = VTV_PROTECTED_VARS_SECTION; 841 842 /* Check to see if this is the record for the Linux Virtual Dynamic 843 Shared Object (linux-vdso.so.1), which exists only in memory (and 844 therefore cannot be read from disk). */ 845 846 if (strcmp (info->dlpi_name, "linux-vdso.so.1") == 0) 847 return 0; 848 849 if (strlen (info->dlpi_name) == 0 850 && info->dlpi_addr != 0) 851 return 0; 852 853 /* Get the name of the main executable. This may or may not include 854 arguments passed to the program. Find the first space, assume it 855 is the start of the argument list, and change it to a '\0'. */ 856 snprintf (program_name, sizeof (program_name), program_invocation_name); 857 858 read_section_offset_and_length (info, map_sect_name, *mprotect_flags, 859 &map_sect_offset, &map_sect_len); 860 861 if (debug_functions) 862 { 863 snprintf (buffer, sizeof(buffer), 864 " Looking at load module %s to change permissions to %s\n", 865 ((strlen (info->dlpi_name) == 0) ? program_name 866 : info->dlpi_name), 867 (*mprotect_flags & PROT_WRITE) ? "READ/WRITE" : "READ-ONLY"); 868 log_memory_protection_data (buffer); 869 } 870 871 /* See if we actually found the section. */ 872 if (map_sect_offset && map_sect_len) 873 { 874 unsigned long long start; 875 int result; 876 877 if (debug_functions) 878 { 879 snprintf (buffer, sizeof (buffer), 880 " (%s): Protecting %p to %p\n", 881 ((strlen (info->dlpi_name) == 0) ? program_name 882 : info->dlpi_name), 883 (void *) map_sect_offset, 884 (void *) (map_sect_offset + map_sect_len)); 885 log_memory_protection_data (buffer); 886 } 887 888 /* Change the protections on the pages for the section. */ 889 890 start = get_cycle_count (); 891 result = mprotect ((void *) map_sect_offset, map_sect_len, 892 *mprotect_flags); 893 accumulate_cycle_count (&mprotect_cycles, start); 894 if (result == -1) 895 { 896 if (debug_functions) 897 { 898 snprintf (buffer, sizeof (buffer), 899 "Failed called to mprotect for %s error: ", 900 (*mprotect_flags & PROT_WRITE) ? 901 "READ/WRITE" : "READ-ONLY"); 902 log_memory_protection_data (buffer); 903 perror(NULL); 904 } 905 VTV_error(); 906 } 907 else 908 { 909 if (debug_functions) 910 { 911 snprintf (buffer, sizeof (buffer), 912 "mprotect'ed range [%p, %p]\n", 913 (void *) map_sect_offset, 914 (char *) map_sect_offset + map_sect_len); 915 log_memory_protection_data (buffer); 916 } 917 } 918 increment_num_calls (&num_calls_to_mprotect); 919 /* num_pages_protected += (map_sect_len + VTV_PAGE_SIZE - 1) / VTV_PAGE_SIZE; */ 920 num_pages_protected += (map_sect_len + 4096 - 1) / 4096; 921 } 922 923 return 0; 924} 925#endif 926 927/* This function explicitly changes the protection (read-only or read-write) 928 on the vtv_sect_info_cache, which is used for speeding up look ups in the 929 function dl_iterate_phdr_callback. This data structure needs to be 930 explicitly made read-write before any calls to dl_iterate_phdr_callback, 931 because otherwise it may still be read-only when dl_iterate_phdr_callback 932 attempts to write to it. 933 934 More detailed explanation: dl_iterate_phdr_callback finds all the 935 .vtable_map_vars sections in all loaded objects (including the main program) 936 and (depending on where it was called from) either makes all the pages in the 937 sections read-write or read-only. The vtv_sect_info_cache should be in the 938 .vtable_map_vars section for libstdc++.so, which means that normally it would 939 be read-only until libstdc++.so is processed by dl_iterate_phdr_callback 940 (on the read-write pass), after which it will be writable. But if any loaded 941 object gets processed before libstdc++.so, it will attempt to update the 942 data cache, which will still be read-only, and cause a seg fault. Hence 943 we need a special function, called before dl_iterate_phdr_callback, that 944 will make the data cache writable. */ 945 946static void 947change_protections_on_phdr_cache (int protection_flag) 948{ 949 char * low_address = (char *) &(vtv_sect_info_cache); 950 size_t cache_size = MAX_ENTRIES * sizeof (struct sect_hdr_data); 951 952 low_address = (char *) ((uintptr_t) low_address & ~(VTV_PAGE_SIZE - 1)); 953 954 if (mprotect ((void *) low_address, cache_size, protection_flag) == -1) 955 VTV_error (); 956} 957 958/* Unprotect all the vtable map vars and other side data that is used 959 to keep the core hash_map data. All of these data have been put 960 into relro sections */ 961 962static void 963vtv_unprotect_vtable_vars (void) 964{ 965 int mprotect_flags; 966 967 mprotect_flags = PROT_READ | PROT_WRITE; 968 change_protections_on_phdr_cache (mprotect_flags); 969#if defined (__CYGWIN__) || defined (__MINGW32__) 970 iterate_modules ((void *) &mprotect_flags); 971#else 972 dl_iterate_phdr (dl_iterate_phdr_callback, (void *) &mprotect_flags); 973#endif 974} 975 976/* Protect all the vtable map vars and other side data that is used 977 to keep the core hash_map data. All of these data have been put 978 into relro sections */ 979 980static void 981vtv_protect_vtable_vars (void) 982{ 983 int mprotect_flags; 984 985 mprotect_flags = PROT_READ; 986#if defined (__CYGWIN__) || defined (__MINGW32__) 987 iterate_modules ((void *) &mprotect_flags); 988#else 989 dl_iterate_phdr (dl_iterate_phdr_callback, (void *) &mprotect_flags); 990#endif 991 change_protections_on_phdr_cache (mprotect_flags); 992} 993 994#ifndef __GTHREAD_MUTEX_INIT 995static void 996initialize_change_permissions_mutexes () 997{ 998 __GTHREAD_MUTEX_INIT_FUNCTION (&change_permissions_lock); 999} 1000#endif 1001 1002/* Variables needed for getting the statistics about the hashtable set. */ 1003#if HASHTABLE_STATS 1004_AtomicStatCounter stat_contains = 0; 1005_AtomicStatCounter stat_insert = 0; 1006_AtomicStatCounter stat_resize = 0; 1007_AtomicStatCounter stat_create = 0; 1008_AtomicStatCounter stat_probes_in_non_trivial_set = 0; 1009_AtomicStatCounter stat_contains_size0 = 0; 1010_AtomicStatCounter stat_contains_size1 = 0; 1011_AtomicStatCounter stat_contains_size2 = 0; 1012_AtomicStatCounter stat_contains_size3 = 0; 1013_AtomicStatCounter stat_contains_size4 = 0; 1014_AtomicStatCounter stat_contains_size5 = 0; 1015_AtomicStatCounter stat_contains_size6 = 0; 1016_AtomicStatCounter stat_contains_size7 = 0; 1017_AtomicStatCounter stat_contains_size8 = 0; 1018_AtomicStatCounter stat_contains_size9 = 0; 1019_AtomicStatCounter stat_contains_size10 = 0; 1020_AtomicStatCounter stat_contains_size11 = 0; 1021_AtomicStatCounter stat_contains_size12 = 0; 1022_AtomicStatCounter stat_contains_size13_or_more = 0; 1023_AtomicStatCounter stat_contains_sizes = 0; 1024_AtomicStatCounter stat_grow_from_size0_to_1 = 0; 1025_AtomicStatCounter stat_grow_from_size1_to_2 = 0; 1026_AtomicStatCounter stat_double_the_number_of_buckets = 0; 1027_AtomicStatCounter stat_insert_found_hash_collision = 0; 1028_AtomicStatCounter stat_contains_in_non_trivial_set = 0; 1029_AtomicStatCounter stat_insert_key_that_was_already_present = 0; 1030#endif 1031/* Record statistics about the hash table sets, for debugging. */ 1032 1033static void 1034log_set_stats (void) 1035{ 1036#if HASHTABLE_STATS 1037 if (set_log_fd == -1) 1038 set_log_fd = __vtv_open_log ("vtv_set_stats.log"); 1039 1040 __vtv_add_to_log (set_log_fd, "---\n%s\n", 1041 insert_only_hash_tables_stats().c_str()); 1042#endif 1043} 1044 1045/* Change the permissions on all the pages we have allocated for the 1046 data sets and all the ".vtable_map_var" sections in memory (which 1047 contain our vtable map variables). PERM indicates whether to make 1048 the permissions read-only or read-write. */ 1049 1050extern "C" /* This is only being applied to __VLTChangePermission*/ 1051void 1052__VLTChangePermission (int perm) 1053{ 1054 if (debug_functions) 1055 { 1056 if (perm == __VLTP_READ_WRITE) 1057 fprintf (stdout, "Changing VLT permisisons to Read-Write.\n"); 1058 else if (perm == __VLTP_READ_ONLY) 1059 fprintf (stdout, "Changing VLT permissions to Read-only.\n"); 1060 1061 else 1062 fprintf (stdout, "Unrecognized permissions value: %d\n", perm); 1063 } 1064 1065#ifndef __GTHREAD_MUTEX_INIT 1066 static __gthread_once_t mutex_once VTV_PROTECTED_VAR = __GTHREAD_ONCE_INIT; 1067 1068 __gthread_once (&mutex_once, initialize_change_permissions_mutexes); 1069#endif 1070 1071 /* Ordering of these unprotect/protect calls is very important. 1072 You first need to unprotect all the map vars and side 1073 structures before you do anything with the core data 1074 structures (hash_maps) */ 1075 1076 if (perm == __VLTP_READ_WRITE) 1077 { 1078 /* TODO: Need to revisit this code for dlopen. It most probably 1079 is not unlocking the protected vtable vars after for load 1080 module that is not the first load module. */ 1081 __gthread_mutex_lock (&change_permissions_lock); 1082 1083 vtv_unprotect_vtable_vars (); 1084 __vtv_malloc_init (); 1085 __vtv_malloc_unprotect (); 1086 1087 } 1088 else if (perm == __VLTP_READ_ONLY) 1089 { 1090 if (debug_hash) 1091 log_set_stats(); 1092 1093 __vtv_malloc_protect (); 1094 vtv_protect_vtable_vars (); 1095 1096 __gthread_mutex_unlock (&change_permissions_lock); 1097 } 1098} 1099 1100/* This is the memory allocator used to create the hash table that 1101 maps from vtable map variable name to the data set that vtable map 1102 variable should point to. This is part of our vtable map variable 1103 symbol resolution, which is necessary because the same vtable map 1104 variable may be created by multiple compilation units and we need a 1105 method to make sure that all vtable map variables for a particular 1106 class point to the same data set at runtime. */ 1107 1108struct insert_only_hash_map_allocator 1109 { 1110 /* N is the number of bytes to allocate. */ 1111 void * 1112 alloc (size_t n) const 1113 { 1114 return __vtv_malloc (n); 1115 } 1116 1117 /* P points to the memory to be deallocated; N is the number of 1118 bytes to deallocate. */ 1119 void 1120 dealloc (void *p, size_t) const 1121 { 1122 __vtv_free (p); 1123 } 1124 }; 1125 1126/* Explicitly instantiate this class since this file is compiled with 1127 -fno-implicit-templates. These are for the hash table that is used 1128 to do vtable map variable symbol resolution. */ 1129template class insert_only_hash_map <vtv_set_handle *, 1130 insert_only_hash_map_allocator >; 1131typedef insert_only_hash_map <vtv_set_handle *, 1132 insert_only_hash_map_allocator > s2s; 1133typedef const s2s::key_type vtv_symbol_key; 1134 1135static s2s * vtv_symbol_unification_map VTV_PROTECTED_VAR = NULL; 1136 1137const unsigned long SET_HANDLE_HANDLE_BIT = 0x2; 1138 1139/* In the case where a vtable map variable is the only instance of the 1140 variable we have seen, it points directly to the set of valid 1141 vtable pointers. All subsequent instances of the 'same' vtable map 1142 variable point to the first vtable map variable. This function, 1143 given a vtable map variable PTR, checks a bit to see whether it's 1144 pointing directly to the data set or to the first vtable map 1145 variable. */ 1146 1147static inline bool 1148is_set_handle_handle (void * ptr) 1149{ 1150 return ((uintptr_t) ptr & SET_HANDLE_HANDLE_BIT) 1151 == SET_HANDLE_HANDLE_BIT; 1152} 1153 1154/* Returns the actual pointer value of a vtable map variable, PTR (see 1155 comments for is_set_handle_handle for more details). */ 1156 1157static inline vtv_set_handle * 1158ptr_from_set_handle_handle (void * ptr) 1159{ 1160 return (vtv_set_handle *) ((uintptr_t) ptr & ~SET_HANDLE_HANDLE_BIT); 1161} 1162 1163/* Given a vtable map variable, PTR, this function sets the bit that 1164 says this is the second (or later) instance of a vtable map 1165 variable. */ 1166 1167static inline vtv_set_handle_handle 1168set_handle_handle (vtv_set_handle * ptr) 1169{ 1170 return (vtv_set_handle_handle) ((uintptr_t) ptr | SET_HANDLE_HANDLE_BIT); 1171} 1172 1173static inline void 1174register_set_common (void **set_handle_ptr, size_t num_args, 1175 void **vtable_ptr_array, bool debug) 1176{ 1177 /* Now figure out what pointer to use for the set pointer, for the 1178 inserts. */ 1179 vtv_set_handle *handle_ptr = (vtv_set_handle *) set_handle_ptr; 1180 1181 if (debug) 1182 VTV_DEBUG_ASSERT (vtv_symbol_unification_map != NULL); 1183 1184 if (!is_set_handle_handle (*set_handle_ptr)) 1185 handle_ptr = (vtv_set_handle *) set_handle_ptr; 1186 else 1187 handle_ptr = ptr_from_set_handle_handle (*set_handle_ptr); 1188 1189 /* Now we've got the set and it's initialized, add the vtable 1190 pointers. */ 1191 for (size_t index = 0; index < num_args; ++index) 1192 { 1193 int_vptr vtbl_ptr = (int_vptr) vtable_ptr_array[index]; 1194 vtv_sets::insert (vtbl_ptr, handle_ptr); 1195 } 1196} 1197 1198static inline void 1199register_pair_common (void **set_handle_ptr, const void *vtable_ptr, 1200 const char *set_symbol_name, const char *vtable_name, 1201 bool debug) 1202{ 1203 /* Now we've got the set and it's initialized, add the vtable 1204 pointer (assuming that it's not NULL...It may be NULL, as we may 1205 have called this function merely to initialize the set 1206 pointer). */ 1207 int_vptr vtbl_ptr = (int_vptr) vtable_ptr; 1208 if (vtbl_ptr) 1209 { 1210 vtv_set_handle *handle_ptr = (vtv_set_handle *) set_handle_ptr; 1211 if (debug) 1212 VTV_DEBUG_ASSERT (vtv_symbol_unification_map != NULL); 1213 if (!is_set_handle_handle (*set_handle_ptr)) 1214 handle_ptr = (vtv_set_handle *) set_handle_ptr; 1215 else 1216 handle_ptr = ptr_from_set_handle_handle (*set_handle_ptr); 1217 1218 vtv_sets::insert (vtbl_ptr, handle_ptr); 1219 } 1220 1221 if (debug && debug_init) 1222 { 1223 if (init_log_fd == -1) 1224 init_log_fd = __vtv_open_log("vtv_init.log"); 1225 1226 __vtv_add_to_log(init_log_fd, 1227 "Registered %s : %s (%p) 2 level deref = %s\n", 1228 set_symbol_name, vtable_name, vtbl_ptr, 1229 is_set_handle_handle(*set_handle_ptr) ? "yes" : "no" ); 1230 } 1231} 1232 1233/* This routine initializes a set handle to a vtable set. It makes 1234 sure that there is only one set handle for a particular set by 1235 using a map from set name to pointer to set handle. Since there 1236 will be multiple copies of the pointer to the set handle (one per 1237 compilation unit that uses it), it makes sure to initialize all the 1238 pointers to the set handle so that the set handle is unique. To 1239 make this a little more efficient and avoid a level of indirection 1240 in some cases, the first pointer to handle for a particular handle 1241 becomes the handle itself and the other pointers will point to the 1242 set handle. This is the debug version of this function, so it 1243 outputs extra debugging messages and logging. SET_HANDLE_PTR is 1244 the address of the vtable map variable, SET_SYMBOL_KEY is the hash 1245 table key (containing the name of the map variable and the hash 1246 value) and SIZE_HINT is a guess for the best initial size for the 1247 set of vtable pointers that SET_HANDLE_POINTER will point to. */ 1248 1249static inline void 1250init_set_symbol_debug (void **set_handle_ptr, const void *set_symbol_key, 1251 size_t size_hint) 1252{ 1253 VTV_DEBUG_ASSERT (set_handle_ptr); 1254 1255 if (vtv_symbol_unification_map == NULL) 1256 { 1257 /* TODO: For now we have chosen 1024, but we need to come up with a 1258 better initial size for this. */ 1259 vtv_symbol_unification_map = s2s::create (1024); 1260 VTV_DEBUG_ASSERT(vtv_symbol_unification_map); 1261 } 1262 1263 vtv_set_handle *handle_ptr = (vtv_set_handle *) set_handle_ptr; 1264 vtv_symbol_key *symbol_key_ptr = (vtv_symbol_key *) set_symbol_key; 1265 1266 const s2s::value_type * map_value_ptr = 1267 vtv_symbol_unification_map->get (symbol_key_ptr); 1268 char buffer[200]; 1269 if (map_value_ptr == NULL) 1270 { 1271 if (*handle_ptr != NULL) 1272 { 1273 snprintf (buffer, sizeof (buffer), 1274 "*** Found non-NULL local set ptr %p missing for symbol" 1275 " %.*s", 1276 *handle_ptr, symbol_key_ptr->n, symbol_key_ptr->bytes); 1277 __vtv_log_verification_failure (buffer, true); 1278 VTV_DEBUG_ASSERT (0); 1279 } 1280 } 1281 else if (*handle_ptr != NULL && 1282 (handle_ptr != *map_value_ptr && 1283 ptr_from_set_handle_handle (*handle_ptr) != *map_value_ptr)) 1284 { 1285 VTV_DEBUG_ASSERT (*map_value_ptr != NULL); 1286 snprintf (buffer, sizeof(buffer), 1287 "*** Found diffence between local set ptr %p and set ptr %p" 1288 "for symbol %.*s", 1289 *handle_ptr, *map_value_ptr, 1290 symbol_key_ptr->n, symbol_key_ptr->bytes); 1291 __vtv_log_verification_failure (buffer, true); 1292 VTV_DEBUG_ASSERT (0); 1293 } 1294 else if (*handle_ptr == NULL) 1295 { 1296 /* Execution should not reach this point. */ 1297 } 1298 1299 if (*handle_ptr != NULL) 1300 { 1301 if (!is_set_handle_handle (*set_handle_ptr)) 1302 handle_ptr = (vtv_set_handle *) set_handle_ptr; 1303 else 1304 handle_ptr = ptr_from_set_handle_handle (*set_handle_ptr); 1305 vtv_sets::resize (size_hint, handle_ptr); 1306 return; 1307 } 1308 1309 VTV_DEBUG_ASSERT (*handle_ptr == NULL); 1310 if (map_value_ptr != NULL) 1311 { 1312 if (*map_value_ptr == handle_ptr) 1313 vtv_sets::resize (size_hint, *map_value_ptr); 1314 else 1315 { 1316 /* The one level handle to the set already exists. So, we 1317 are adding one level of indirection here and we will 1318 store a pointer to the one level handle here. */ 1319 1320 vtv_set_handle_handle * handle_handle_ptr = 1321 (vtv_set_handle_handle *)handle_ptr; 1322 *handle_handle_ptr = set_handle_handle(*map_value_ptr); 1323 VTV_DEBUG_ASSERT(*handle_handle_ptr != NULL); 1324 1325 /* The handle can itself be NULL if the set has only 1326 been initiazlied with size hint == 1. */ 1327 vtv_sets::resize (size_hint, *map_value_ptr); 1328 } 1329 } 1330 else 1331 { 1332 /* We will create a new set. So, in this case handle_ptr is the 1333 one level pointer to the set handle. Create copy of map name 1334 in case the memory where this comes from gets unmapped by 1335 dlclose. */ 1336 size_t map_key_len = symbol_key_ptr->n + sizeof (vtv_symbol_key); 1337 void *map_key = __vtv_malloc (map_key_len); 1338 1339 memcpy (map_key, symbol_key_ptr, map_key_len); 1340 1341 s2s::value_type *value_ptr; 1342 vtv_symbol_unification_map = 1343 vtv_symbol_unification_map->find_or_add_key ((vtv_symbol_key *)map_key, 1344 &value_ptr); 1345 *value_ptr = handle_ptr; 1346 1347 /* TODO: We should verify the return value. */ 1348 vtv_sets::create (size_hint, handle_ptr); 1349 VTV_DEBUG_ASSERT (size_hint <= 1 || *handle_ptr != NULL); 1350 } 1351 1352 if (debug_init) 1353 { 1354 if (init_log_fd == -1) 1355 init_log_fd = __vtv_open_log ("vtv_init.log"); 1356 1357 __vtv_add_to_log (init_log_fd, 1358 "Init handle:%p for symbol:%.*s hash:%u size_hint:%lu" 1359 "number of symbols:%lu \n", 1360 set_handle_ptr, symbol_key_ptr->n, 1361 symbol_key_ptr->bytes, symbol_key_ptr->hash, size_hint, 1362 vtv_symbol_unification_map->size ()); 1363 } 1364} 1365 1366 1367/* This routine initializes a set handle to a vtable set. It makes 1368 sure that there is only one set handle for a particular set by 1369 using a map from set name to pointer to set handle. Since there 1370 will be multiple copies of the pointer to the set handle (one per 1371 compilation unit that uses it), it makes sure to initialize all the 1372 pointers to the set handle so that the set handle is unique. To 1373 make this a little more efficient and avoid a level of indirection 1374 in some cases, the first pointer to handle for a particular handle 1375 becomes the handle itself and the other pointers will point to the 1376 set handle. This is the debug version of this function, so it 1377 outputs extra debugging messages and logging. SET_HANDLE_PTR is 1378 the address of the vtable map variable, SET_SYMBOL_KEY is the hash 1379 table key (containing the name of the map variable and the hash 1380 value) and SIZE_HINT is a guess for the best initial size for the 1381 set of vtable pointers that SET_HANDLE_POINTER will point to. */ 1382 1383void 1384__VLTRegisterSetDebug (void **set_handle_ptr, const void *set_symbol_key, 1385 size_t size_hint, size_t num_args, 1386 void **vtable_ptr_array) 1387{ 1388 unsigned long long start = get_cycle_count (); 1389 increment_num_calls (&num_calls_to_regset); 1390 1391 VTV_DEBUG_ASSERT(set_handle_ptr != NULL); 1392 init_set_symbol_debug (set_handle_ptr, set_symbol_key, size_hint); 1393 1394 register_set_common (set_handle_ptr, num_args, vtable_ptr_array, true); 1395 1396 accumulate_cycle_count (®set_cycles, start); 1397} 1398 1399/* This function takes a the address of a vtable map variable 1400 (SET_HANDLE_PTR), a VTABLE_PTR to add to the data set, the name of 1401 the vtable map variable (SET_SYMBOL_NAME) and the name of the 1402 vtable (VTABLE_NAME) being pointed to. If the vtable map variable 1403 is NULL it creates a new data set and initializes the variable, 1404 otherwise it uses our symbol unification to find the right data 1405 set; in either case it then adds the vtable pointer to the set. 1406 The other two parameters are used for debugging information. */ 1407 1408void 1409__VLTRegisterPairDebug (void **set_handle_ptr, const void *set_symbol_key, 1410 size_t size_hint, const void *vtable_ptr, 1411 const char *set_symbol_name, const char *vtable_name) 1412{ 1413 unsigned long long start = get_cycle_count (); 1414 increment_num_calls (&num_calls_to_regpair); 1415 1416 VTV_DEBUG_ASSERT(set_handle_ptr != NULL); 1417 init_set_symbol_debug (set_handle_ptr, set_symbol_key, size_hint); 1418 1419 register_pair_common (set_handle_ptr, vtable_ptr, set_symbol_name, vtable_name, 1420 true); 1421 1422 accumulate_cycle_count (®pair_cycles, start); 1423} 1424 1425 1426/* This is the debug version of the verification function. It takes 1427 the address of a vtable map variable (SET_HANDLE_PTR) and a 1428 VTABLE_PTR to validate, as well as the name of the vtable map 1429 variable (SET_SYMBOL_NAME) and VTABLE_NAME, which are used for 1430 debugging messages. It checks to see if VTABLE_PTR is in the set 1431 pointed to by SET_HANDLE_PTR. If so, it returns VTABLE_PTR, 1432 otherwise it calls __vtv_verify_fail, which usually logs error 1433 messages and calls abort. */ 1434 1435const void * 1436__VLTVerifyVtablePointerDebug (void **set_handle_ptr, const void *vtable_ptr, 1437 const char *set_symbol_name, 1438 const char *vtable_name) 1439{ 1440 unsigned long long start = get_cycle_count (); 1441 VTV_DEBUG_ASSERT (set_handle_ptr != NULL && *set_handle_ptr != NULL); 1442 int_vptr vtbl_ptr = (int_vptr) vtable_ptr; 1443 1444 increment_num_calls (&num_calls_to_verify_vtable); 1445 vtv_set_handle *handle_ptr; 1446 if (!is_set_handle_handle (*set_handle_ptr)) 1447 handle_ptr = (vtv_set_handle *) set_handle_ptr; 1448 else 1449 handle_ptr = ptr_from_set_handle_handle (*set_handle_ptr); 1450 1451 if (vtv_sets::contains (vtbl_ptr, handle_ptr)) 1452 { 1453 if (debug_verify_vtable) 1454 { 1455 if (verify_vtable_log_fd == -1) 1456 __vtv_open_log ("vtv_verify_vtable.log"); 1457 __vtv_add_to_log (verify_vtable_log_fd, 1458 "Verified %s %s value = %p\n", 1459 set_symbol_name, vtable_name, vtable_ptr); 1460 } 1461 } 1462 else 1463 { 1464 /* We failed to find the vtable pointer in the set of valid 1465 pointers. Log the error data and call the failure 1466 function. */ 1467 snprintf (debug_log_message, sizeof (debug_log_message), 1468 "Looking for %s in %s\n", vtable_name, set_symbol_name); 1469 __vtv_verify_fail_debug (set_handle_ptr, vtable_ptr, debug_log_message); 1470 1471 /* Normally __vtv_verify_fail_debug will call abort, so we won't 1472 execute the return below. If we get this far, the assumption 1473 is that the programmer has replaced __vtv_verify_fail_debug 1474 with some kind of secondary verification AND this secondary 1475 verification succeeded, so the vtable pointer is valid. */ 1476 } 1477 accumulate_cycle_count (&verify_vtable_cycles, start); 1478 1479 return vtable_ptr; 1480} 1481 1482/* This routine initializes a set handle to a vtable set. It makes 1483 sure that there is only one set handle for a particular set by 1484 using a map from set name to pointer to set handle. Since there 1485 will be multiple copies of the pointer to the set handle (one per 1486 compilation unit that uses it), it makes sure to initialize all the 1487 pointers to the set handle so that the set handle is unique. To 1488 make this a little more efficient and avoid a level of indirection 1489 in some cases, the first pointer to handle for a particular handle 1490 becomes the handle itself and the other pointers will point to the 1491 set handle. SET_HANDLE_PTR is the address of the vtable map 1492 variable, SET_SYMBOL_KEY is the hash table key (containing the name 1493 of the map variable and the hash value) and SIZE_HINT is a guess 1494 for the best initial size for the set of vtable pointers that 1495 SET_HANDLE_POINTER will point to.*/ 1496 1497static inline void 1498init_set_symbol (void **set_handle_ptr, const void *set_symbol_key, 1499 size_t size_hint) 1500{ 1501 vtv_set_handle *handle_ptr = (vtv_set_handle *) set_handle_ptr; 1502 1503 if (*handle_ptr != NULL) 1504 { 1505 if (!is_set_handle_handle (*set_handle_ptr)) 1506 handle_ptr = (vtv_set_handle *) set_handle_ptr; 1507 else 1508 handle_ptr = ptr_from_set_handle_handle (*set_handle_ptr); 1509 vtv_sets::resize (size_hint, handle_ptr); 1510 return; 1511 } 1512 1513 if (vtv_symbol_unification_map == NULL) 1514 vtv_symbol_unification_map = s2s::create (1024); 1515 1516 vtv_symbol_key *symbol_key_ptr = (vtv_symbol_key *) set_symbol_key; 1517 const s2s::value_type *map_value_ptr = 1518 vtv_symbol_unification_map->get (symbol_key_ptr); 1519 1520 if (map_value_ptr != NULL) 1521 { 1522 if (*map_value_ptr == handle_ptr) 1523 vtv_sets::resize (size_hint, *map_value_ptr); 1524 else 1525 { 1526 /* The one level handle to the set already exists. So, we 1527 are adding one level of indirection here and we will 1528 store a pointer to the one level pointer here. */ 1529 vtv_set_handle_handle *handle_handle_ptr = 1530 (vtv_set_handle_handle *) handle_ptr; 1531 *handle_handle_ptr = set_handle_handle (*map_value_ptr); 1532 vtv_sets::resize (size_hint, *map_value_ptr); 1533 } 1534 } 1535 else 1536 { 1537 /* We will create a new set. So, in this case handle_ptr is the 1538 one level pointer to the set handle. Create copy of map name 1539 in case the memory where this comes from gets unmapped by 1540 dlclose. */ 1541 size_t map_key_len = symbol_key_ptr->n + sizeof (vtv_symbol_key); 1542 void * map_key = __vtv_malloc (map_key_len); 1543 memcpy (map_key, symbol_key_ptr, map_key_len); 1544 1545 s2s::value_type * value_ptr; 1546 vtv_symbol_unification_map = 1547 vtv_symbol_unification_map->find_or_add_key ((vtv_symbol_key *)map_key, 1548 &value_ptr); 1549 1550 *value_ptr = handle_ptr; 1551 1552 /* TODO: We should verify the return value. */ 1553 vtv_sets::create (size_hint, handle_ptr); 1554 } 1555} 1556 1557/* This routine initializes a set handle to a vtable set. It makes 1558 sure that there is only one set handle for a particular set by 1559 using a map from set name to pointer to set handle. Since there 1560 will be multiple copies of the pointer to the set handle (one per 1561 compilation unit that uses it), it makes sure to initialize all the 1562 pointers to the set handle so that the set handle is unique. To 1563 make this a little more efficient and avoid a level of indirection 1564 in some cases, the first pointer to handle for a particular handle 1565 becomes the handle itself and the other pointers will point to the 1566 set handle. SET_HANDLE_PTR is the address of the vtable map 1567 variable, SET_SYMBOL_KEY is the hash table key (containing the name 1568 of the map variable and the hash value) and SIZE_HINT is a guess 1569 for the best initial size for the set of vtable pointers that 1570 SET_HANDLE_POINTER will point to.*/ 1571 1572 1573void 1574__VLTRegisterSet (void **set_handle_ptr, const void *set_symbol_key, 1575 size_t size_hint, size_t num_args, void **vtable_ptr_array) 1576{ 1577 unsigned long long start = get_cycle_count (); 1578 increment_num_calls (&num_calls_to_regset); 1579 1580 init_set_symbol (set_handle_ptr, set_symbol_key, size_hint); 1581 register_set_common (set_handle_ptr, num_args, vtable_ptr_array, false); 1582 1583 accumulate_cycle_count (®set_cycles, start); 1584} 1585 1586 1587 1588/* This function takes a the address of a vtable map variable 1589 (SET_HANDLE_PTR) and a VTABLE_PTR. If the vtable map variable is 1590 NULL it creates a new data set and initializes the variable, 1591 otherwise it uses our symbol unification to find the right data 1592 set; in either case it then adds the vtable pointer to the set. */ 1593 1594void 1595__VLTRegisterPair (void **set_handle_ptr, const void *set_symbol_key, 1596 size_t size_hint, const void *vtable_ptr) 1597{ 1598 unsigned long long start = get_cycle_count (); 1599 increment_num_calls (&num_calls_to_regpair); 1600 1601 init_set_symbol (set_handle_ptr, set_symbol_key, size_hint); 1602 register_pair_common (set_handle_ptr, vtable_ptr, NULL, NULL, false); 1603 1604 accumulate_cycle_count (®pair_cycles, start); 1605} 1606 1607/* This is the main verification function. It takes the address of a 1608 vtable map variable (SET_HANDLE_PTR) and a VTABLE_PTR to validate. 1609 It checks to see if VTABLE_PTR is in the set pointed to by 1610 SET_HANDLE_PTR. If so, it returns VTABLE_PTR, otherwise it calls 1611 __vtv_verify_fail, which usually logs error messages and calls 1612 abort. Since this function gets called VERY frequently, it is 1613 important for it to be as efficient as possible. */ 1614 1615const void * 1616__VLTVerifyVtablePointer (void ** set_handle_ptr, const void * vtable_ptr) 1617{ 1618 unsigned long long start = get_cycle_count (); 1619 int_vptr vtbl_ptr = (int_vptr) vtable_ptr; 1620 1621 vtv_set_handle *handle_ptr; 1622 increment_num_calls (&num_calls_to_verify_vtable); 1623 if (!is_set_handle_handle (*set_handle_ptr)) 1624 handle_ptr = (vtv_set_handle *) set_handle_ptr; 1625 else 1626 handle_ptr = ptr_from_set_handle_handle (*set_handle_ptr); 1627 1628 if (!vtv_sets::contains (vtbl_ptr, handle_ptr)) 1629 { 1630 __vtv_verify_fail ((void **) handle_ptr, vtable_ptr); 1631 /* Normally __vtv_verify_fail will call abort, so we won't 1632 execute the return below. If we get this far, the assumption 1633 is that the programmer has replaced __vtv_verify_fail with 1634 some kind of secondary verification AND this secondary 1635 verification succeeded, so the vtable pointer is valid. */ 1636 } 1637 accumulate_cycle_count (&verify_vtable_cycles, start); 1638 1639 return vtable_ptr; 1640} 1641 1642static int page_count_2 = 0; 1643 1644#if !defined (__CYGWIN__) && !defined (__MINGW32__) 1645static int 1646dl_iterate_phdr_count_pages (struct dl_phdr_info *info, 1647 size_t unused __attribute__ ((__unused__)), 1648 void *data) 1649{ 1650 int *mprotect_flags = (int *) data; 1651 off_t map_sect_offset = 0; 1652 ElfW (Word) map_sect_len = 0; 1653 const char *map_sect_name = VTV_PROTECTED_VARS_SECTION; 1654 1655 /* Check to see if this is the record for the Linux Virtual Dynamic 1656 Shared Object (linux-vdso.so.1), which exists only in memory (and 1657 therefore cannot be read from disk). */ 1658 1659 if (strcmp (info->dlpi_name, "linux-vdso.so.1") == 0) 1660 return 0; 1661 1662 if (strlen (info->dlpi_name) == 0 1663 && info->dlpi_addr != 0) 1664 return 0; 1665 1666 read_section_offset_and_length (info, map_sect_name, *mprotect_flags, 1667 &map_sect_offset, &map_sect_len); 1668 1669 /* See if we actually found the section. */ 1670 if (map_sect_len) 1671 page_count_2 += (map_sect_len + VTV_PAGE_SIZE - 1) / VTV_PAGE_SIZE; 1672 1673 return 0; 1674} 1675#endif 1676 1677static void 1678count_all_pages (void) 1679{ 1680 int mprotect_flags; 1681 1682 mprotect_flags = PROT_READ; 1683 page_count_2 = 0; 1684 1685#if defined (__CYGWIN__) || defined (__MINGW32__) 1686 iterate_modules ((void *) &mprotect_flags); 1687#else 1688 dl_iterate_phdr (dl_iterate_phdr_count_pages, (void *) &mprotect_flags); 1689#endif 1690 page_count_2 += __vtv_count_mmapped_pages (); 1691} 1692 1693void 1694__VLTDumpStats (void) 1695{ 1696 int log_fd = __vtv_open_log ("vtv-runtime-stats.log"); 1697 1698 if (log_fd != -1) 1699 { 1700 count_all_pages (); 1701 __vtv_add_to_log (log_fd, 1702 "Calls: mprotect (%d) regset (%d) regpair (%d)" 1703 " verify_vtable (%d)\n", 1704 num_calls_to_mprotect, num_calls_to_regset, 1705 num_calls_to_regpair, num_calls_to_verify_vtable); 1706 __vtv_add_to_log (log_fd, 1707 "Cycles: mprotect (%lld) regset (%lld) " 1708 "regpair (%lld) verify_vtable (%lld)\n", 1709 mprotect_cycles, regset_cycles, regpair_cycles, 1710 verify_vtable_cycles); 1711 __vtv_add_to_log (log_fd, 1712 "Pages protected (1): %d\n", num_pages_protected); 1713 __vtv_add_to_log (log_fd, "Pages protected (2): %d\n", page_count_2); 1714 1715 close (log_fd); 1716 } 1717} 1718 1719/* This function is called from __VLTVerifyVtablePointerDebug; it 1720 sends as much debugging information as it can to the error log 1721 file, then calls __vtv_verify_fail. SET_HANDLE_PTR is the pointer 1722 to the set of valid vtable pointers, VTBL_PTR is the pointer that 1723 was not found in the set, and DEBUG_MSG is the message to be 1724 written to the log file before failing. n */ 1725 1726void 1727__vtv_verify_fail_debug (void **set_handle_ptr, const void *vtbl_ptr, 1728 const char *debug_msg) 1729{ 1730 __vtv_log_verification_failure (debug_msg, false); 1731 1732 /* Call the public interface in case it has been overwritten by 1733 user. */ 1734 __vtv_verify_fail (set_handle_ptr, vtbl_ptr); 1735 1736 __vtv_log_verification_failure ("Returned from __vtv_verify_fail." 1737 " Secondary verification succeeded.\n", false); 1738} 1739 1740/* This function calls __fortify_fail with a FAILURE_MSG and then 1741 calls abort. */ 1742 1743void 1744__vtv_really_fail (const char *failure_msg) 1745{ 1746 __fortify_fail (failure_msg); 1747 1748 /* We should never get this far; __fortify_fail calls __libc_message 1749 which prints out a back trace and a memory dump and then is 1750 supposed to call abort, but let's play it safe anyway and call abort 1751 ourselves. */ 1752 abort (); 1753} 1754 1755/* This function takes an error MSG, a vtable map variable 1756 (DATA_SET_PTR) and a vtable pointer (VTBL_PTR). It is called when 1757 an attempt to verify VTBL_PTR with the set pointed to by 1758 DATA_SET_PTR failed. It outputs a failure message with the 1759 addresses involved, and calls __vtv_really_fail. */ 1760 1761static void 1762vtv_fail (const char *msg, void **data_set_ptr, const void *vtbl_ptr) 1763{ 1764 char buffer[128]; 1765 int buf_len; 1766 const char *format_str = 1767 "*** Unable to verify vtable pointer (%p) in set (%p) *** \n"; 1768 1769 snprintf (buffer, sizeof (buffer), format_str, vtbl_ptr, 1770 is_set_handle_handle(*data_set_ptr) ? 1771 ptr_from_set_handle_handle (*data_set_ptr) : 1772 *data_set_ptr); 1773 buf_len = strlen (buffer); 1774 /* Send this to to stderr. */ 1775 write (2, buffer, buf_len); 1776 1777#ifndef VTV_NO_ABORT 1778 __vtv_really_fail (msg); 1779#endif 1780} 1781 1782/* Send information about what we were trying to do when verification 1783 failed to the error log, then call vtv_fail. This function can be 1784 overwritten/replaced by the user, to implement a secondary 1785 verification function instead. DATA_SET_PTR is the vtable map 1786 variable used for the failed verification, and VTBL_PTR is the 1787 vtable pointer that was not found in the set. */ 1788 1789void 1790__vtv_verify_fail (void **data_set_ptr, const void *vtbl_ptr) 1791{ 1792 char log_msg[256]; 1793 snprintf (log_msg, sizeof (log_msg), "Looking for vtable %p in set %p.\n", 1794 vtbl_ptr, 1795 is_set_handle_handle (*data_set_ptr) ? 1796 ptr_from_set_handle_handle (*data_set_ptr) : 1797 *data_set_ptr); 1798 __vtv_log_verification_failure (log_msg, false); 1799 1800 const char *format_str = 1801 "*** Unable to verify vtable pointer (%p) in set (%p) *** \n"; 1802 snprintf (log_msg, sizeof (log_msg), format_str, vtbl_ptr, *data_set_ptr); 1803 __vtv_log_verification_failure (log_msg, false); 1804 __vtv_log_verification_failure (" Backtrace: \n", true); 1805 1806 const char *fail_msg = "Potential vtable pointer corruption detected!!\n"; 1807 vtv_fail (fail_msg, data_set_ptr, vtbl_ptr); 1808} 1809