echo | tee set_*
echo 0 | tee tracing_enabled tracing_max_latency
trace-cmd record -g '*fsync*' -p function_graph -s 10000 -b 10000
Makes a nice small log only of the fsyncs on your system. Good to check what's waking up your hard disk. The low frequency (-s) effects your system less and the large buffer (-b) makes sure no tracing data is lost.
echo '*somemodulename*' > set_ftrace_filter
e.g. echo '*ext4*' > set_ftrace_filter
or echo :mod:e1000e > set_ftrace_filter
Switch tracing on/off when a certain function is called:
echo bad_inode_create:traceoff > set_ftrace_filter
Check IRQ Latencies:
echo do_IRQ > set_ftrace_filter
echo function_graph > current_tracer
echo 1 > events/irq/irq_handler_entry/enable
Check for code areas where IRQs are disabled
echo irqsoff > current_tracer
Trance a certain function, e.g. sys_read:
echo sys_read > set_graph_function
echo 1 > events/enable -> all echo
or echo 1 > events/sched/enable -> just some,
sched > set_event -> the same command
"available_events shows what events are available"
Further Filtering on an by event basis:
echo "prev_state == 0" > events/sched/sched_switch/filter
echo 1 > /proc/sys/kernel/stack_tracer_enabled
Don't trace locks:
echo '*lock*' > set_ftrace_notrace
Ftrace on oops:
echo 1 > /proc/sys/kernel/ftrace_dump_on_oops
sysrq-z -> get the dump
Trace a command:
trace-cmd record -o fgraph-events.dat -e sched -p function_graph \
ls -ltr /usr > /dev/null
"-f : filter
-O : option
-g : same as echoing into set_graph_function
-l : same as echoing into set_ftrace_filter
-n : same as echoing into set_ftrace_notrace"
Trace via network:
trace-cmd listen -p 5678 -d
trace-cmd record -N host:5678 -e all
Disable all tracing:
And if that's all unclear or not enough, try to catch one of Steve's presentations and check his slides.
And if you wonder why in his slides he likes to use 'cat | head' instead of just 'head', here's his answer to that: Most of these tools are not efficient with non-linear reads. So cat is slower to type, but works faster as it forces a linear read.
And feel free to post your favorites!