Python / Django: 2016

JIT Just-in-time compilation динамическая компиляция. Технология увеличения производительности программ, использующих байт-код, путём компиляции байт-кода в машинный код непосредственно во время работы. Так достигается высокая скорость выполнения по сравнению с интерпретируемым байт-кодом (сравнимая с компилируемыми языками) за счёт увеличения потребления памяти (для хранения результатов компиляции) и затрат времени на компиляцию. Из-за того что компиляция произходит во время исполнения можно проводить различные оптимизации, например, компиляция может осуществляться непосредственно для целевого CPU и операционной системы (SSE, MMX), profile-guided optimizations - cреда может собирать статистику о работающей программе и производить оптимизации с учётом этой информации; cреда может делать глобальные оптимизации кода (встраивание библиотечных функций в код, pseudo-constant propagation или indirect/virtual function inlining); перестраивание кода для лучшего использования кэша.

Tracing JIT в отличие от классического JIT записывает линейную последовательность наиболее часто используемых операций, компилирует их в нативные машинные инструкции, а затем выполняет. Этим он отличается от классического JIT, который компилирует методы целиком.

AOT Ahead-of-time compilation. Процесс компиляции полностью выполняется перед выполнением программы. АОТ не требует выделения дополнительной памяти. АОТ компиляция проходит с минимальной нагрузкой на систему. Хорошо подходит для встроенных систем или mobile устройств.

Примеры

Python, Ruby - интерпретируемый байт-кодом (минимальные оптимизации)

PyPy (via RPython), HOPE, Dalvik (Android), Java, .Net - JIT

ART (заменит Dalvik), Mono AOT, GNU Compiler for Java - AOT

Lua, Pyston, JavaScript V8 - Tracing JIT

Misc

Just-In-Time Compilation Using GCC: Video, Slides, Wiki

Contents

GDB
- Controlling Execution
ASM
Misc

GDB

Controlling Execution

run

set args test1 test2

show args

next

step

finish - takes you out of the function call, if you are already inside one

return - returns to the caller of the current frame in the stack. This means that you can return from a function without actually completing the function code execution.

continue

quit

kill - stops debugging but does not quit the debugger

Break Points

break function_name

break file.c:n

info break

disa/enable *m* - disable/enable break

del *m*

!!!!! condition bp_number [expression]

condition 2 *p1 != *p2

condition 2 - deletes condition

!!!!! watch expression - The debugger stops the program when the value of expression changes.

!!!!! rwatch expression - The debugger stops the program whenever the program reads the value of any object involved in the evaluation of expression.

!!!!! awatch expression - The debugger stops the program whenever the program reads or modifies the value of any object involved in the evaluation of expression.

info locals - print all local variables

list n - lists lines in the source code file

list n, m

print expr

p 2*circularArea($2) - $i - refer to previous output

!!! p main::radius - access variable in other stack frame

ptype var - prints structure or union

display expr - (disp)

undisplay

info display

enable/disable display

show path

pwd

Modifying Variables

print var1

set var1=22

print var1

show environment - displays environment variables

set/unset env

Stack

frame - shows the current frame of execution for the program

info frame

info locals

info reg

info all-reg - including math registers

backtrace

up - takes you one level up in the stack

down

Files and Shared Libraries

info files

info share

Macroses

Compile with options gcc -gdwarf-4 -g3 sample.c -o sample

info macro ADD

macro expand ADD(x)

TUI

TUI-Keys

C-x C-a - enter or leave the TUI mode

C-x 2 - change layout (1 or 2 windows)

C-x o - change active window

C-x s - switch TUI SingleKey mode

C-L - refresh

info win (i win) - current window info

focus winname - (fs) - set focus to "SRC", "CMD", "ASM", or "REG" or by position "next" or "prev"

layout type - set layout "src", "asm", "split", or "reg"

tui reg type - set the register window layout "general", "float", "system", or "next"

winheight val - (wh) - set the window height (either an absolute value, or a relative value prefaced with "+" or "-")

C-p - previous command in history

C-n - next command in history

C-f - move cursor forward

C-b - move cursor backward

M-f - move cursor forward word

M-b - move cursor backward word

C-a - move to the start of the line

C-e - move to the end of the line

C-d - delete the character underneath the cursor

C-_ or C-x C-u - undo the last editing command

C-l - clear the screen

C-k - kill to the end of the line

M-d - kill to the end of the current word

M-<DEL> - kill from the cursor the start of the current word

C-w - kill from the cursor to the previous whitespace

C-y - yank the most recently killed text back

M-y - rotate the kill-ring, and yank the new top. You can only do this if the prior command is C-y or M-y

Disassembly

set disassembly-flavor flavor - set the look-and-feel of the disassembly. On Intel machines, valid flavors are intel and att

set disassemble-next-line on

ASM

(gdb) break *label+offset
(gdb) break *_start
(gdb) break *_start+1

(gdb) x/d 0x100001018
0x100001018 <natural_generator.b>:  -1
(gdb) x/d &b
0x100001018 <natural_generator.b>:  -1

Stack

(gdb) x/20x $rsp-20
0x7fffffffe800:     0x00000000      0x00000000      0x00000000      0x00000000
0x7fffffffe810:     0x00000000      0x00000000      0x00000000      0x00000000
0x7fffffffe820:     0x439d1463      0x00000000      0x00000000      0x00000000
0x7fffffffe830:     0x004000c2      0x00000000      0x0000000a      0x00000000
0x7fffffffe840:     0x00000001      0x00000000      0xffffeb0f      0x00007fff

Machine instructions

(gdb) x/10i $rip
=> 0x40011f <area+27>:      pop    %rbp
   0x400120 <area+28>:      retq
   0x400121:        jg     0x400123
   0x400123:        add    %al,(%rcx)

Data

Output formats:

d - decimal
x - hexadecimal
t - binary. The letter `t' stands for "two"
u - unsigned
o - octal
f - floating point
i - instruction
c - character
s - string
a - address. You can use this format used to discover where (in what function) an unknown address is located:

(gdb) p/a 0x54320
$1 = 0x54320 <_initialize_vx+396>
(gdb) p/a &h
$2 = 0x7ffff7dd7820 <h>

# same as
(gdb) info symbol 0x54320
(gdb) info symbol  &h
h in section .bss of /lib/x86_64-linux-gnu/libc.so.6

(gdb) p (int)$rax
$3 = -1

Memmory

Examining Memory:

b - byte
h - halfword (2 bytes)
w - word (4 bytes)
g - giant (8 bytes)

l for a 32-bit long word value
w for a 16-bit word value
b for an 8-bit byte value

x/nfu addr

n, the repeat count
f, the display format is one of the formats used by print (‘x’, ‘d’, ‘u’, ‘o’, ‘t’, ‘a’, ‘c’, ‘f’, ‘s’), and in addition ‘i’ (for machine instructions). The default is ‘x’ (hexadecimal) initially. The default changes each time you use either x or print.
u, the unit size

(gdb) x/42cb &output
0x80490ac <output>:84 ‘T’ 104 ‘h’ 101 ‘e’ 32 ‘ ‘ 112 ‘p’ 114 ‘r’ 111 ‘o’99 ‘c’

Misc

Tricks

info locals - print all local variables

p *data@10 - print array

n

<enter> - repeat last command

call PyObject_Print(0x7ffff7f64ea0, stderr, 1)

info threads

thread 2

Run program reverse

(gdb) target record-full
(gdb) next
(gdb) reverse-next

https://sourceware.org/gdb/current/onlinedocs/gdb/Reverse-Execution.html

http://stackoverflow.com/questions/1206872/go-to-previous-line-in-gdb

GDB command options

-cd Specify the working directory

-d Specify a directory to search for source files

-nx Do not execute commands from .gdbinit file

Install from source

./configure --with-python --prefix=/home/user/opt/gdb
make
make install

Links

8 gdb tricks you should know

	import string

	# In [1]: string.punctuation
	# Out[1]: '!"#$%&\'()*+,-./:;<=>?@[\\]^_`{\|}~'

	# In [2]: string.uppercase
	# Out[2]: 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'

	# In [3]: " " * len(string.punctuation) + string.lowercase
	# Out[3]: ' abcdefghijklmnopqrstuvwxyz'

	# translation table maps upper case to lower case and punctuation to spaces
	translation_table = string.maketrans(string.punctuation+string.uppercase,
	" " * len(string.punctuation) + string.lowercase)

	line = ""
	line = line.translate(translation_table)

	# In [4]: line
	# Out[4]: 'foo bar bizz'

	word_list = line.split()

	# In [5]: word_list
	# Out[5]: ['foo', 'bar', 'bizz']

	/*
	https://en.wikipedia.org/wiki/Time_Stamp_Counter
	https://ru.wikipedia.org/wiki/Rdtsc
	*/

	#include <stdio.h>

	typedef unsigned long long uint64;

	int main() {
	uint64 val;
	unsigned int h, l;
	for (int i=0; i<=10; i++) {
	__asm__ __volatile__("rdtsc" : "=a" (l), "=d" (h));
	val = ((uint64)l) \| (((uint64)h) << 32);

	printf("%llu \n", val);
	}
	}

	// http://developers.redhat.com/blog/2016/03/11/practical-micro-benchmarking-with-ltrace-and-sched/

	/* One drawback of the RDTSC instruction is that the CPU is allowed to reorder
	it relative to other instructions, which causes noise in our results. Fortunately,
	Intel has provided an RDTSCP instruction that’s more deterministic. We’ll pair
	that with a CPUID instruction which acts as a memory barrier, resulting in this: */

	static __inline__ int64_t rdtsc_s(void)
	{
	unsigned a, d;
	asm volatile("cpuid" ::: "%rax", "%rbx", "%rcx", "%rdx");
	asm volatile("rdtsc" : "=a" (a), "=d" (d));
	return ((unsigned long)a) \| (((unsigned long)d) << 32);
	}

	static __inline__ int64_t rdtsc_e(void)
	{
	unsigned a, d;
	asm volatile("rdtscp" : "=a" (a), "=d" (d));
	asm volatile("cpuid" ::: "%rax", "%rbx", "%rcx", "%rdx");
	return ((unsigned long)a) \| (((unsigned long)d) << 32);
	}

	. . .

	clocks_before = rdtsc_s ();
	p = malloc (i); /* Test goes here */
	clocks_after = rdtsc_e ();
	clocks_per_malloc = clocks_after - clocks_before;

	// let the OS use CPU #0
	// boot options:
	// linux . . . isolcpus=1,2,3,4,5,6,7

	// check:
	// taskset -p $$

	// Interrupt affinity:
	// cd /proc/irq
	// for i in */smp_affinity; do echo 1 > $i; done

	define pyp
	set \$\_unused\_void = PyObject\_Print($arg0, stderr, 1)
	printf "\n"
	end

Dec 13, 2016

Oct 25, 2016

Compile and run Postgres with debug symbols

Apr 16, 2016

String translation table

Mar 12, 2016

Using RDTSC instruction that returns CPU TSC (Time Stamp Counter)

Feb 4, 2016

JIT vs AOT

Примеры

Misc

Feb 2, 2016

Gdb commands cheatsheet

GDB

Controlling Execution

Break Points

Modifying Variables

Stack

Files and Shared Libraries

Macroses

TUI

Disassembly

ASM

Stack

Machine instructions

Data

Memmory

Misc

Tricks

Run program reverse

GDB command options

Install from source

Links