This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Poor scaling
- From: Avi Kivity <avi at scylladb dot com>
- To: systemtap at sourceware dot org
- Date: Sun, 7 Aug 2016 15:47:54 +0300
- Subject: Poor scaling
- Authentication-results: sourceware.org; auth=none
Running a process with 12 active threads, each bound to its own core,
and touching a systemtap array fairly frequently, I see:
72.91% scylla [kernel.kallsyms] [k] _raw_spin_lock
3.49% scylla [kernel.kallsyms] [k]
start_callback.isra.62
1.77% scylla scylla [.]
_ZSt13__adjust_heapIN9__gnu_cxx17__normal_iteratorIPPN8logalloc11region_implESt6vectorIS4_SaIS4_EEEElS4_NS0_5__ops15_Iter_comp_iterIZNS2_7tracker4impl15compact_on_idleERKSt8functionIFbvEEEUlS4_S4_E_EEEvT_T0_SM_T1_T2_.isra.437.constprop.582
1.54% scylla [kernel.kallsyms] [k] _raw_spin_unlock
1.39% scylla scylla [.]
_ZN7reactor10smp_pollfn4pollEv
1.29% scylla libc-2.19.so [.] epoll_pwait
1.21% swapper [kernel.kallsyms] [k] poll_idle
1.06% scylla [kernel.kallsyms] [k]
_raw_read_lock_irqsave
1.01% scylla scylla [.]
_ZSt11__push_heapIN9__gnu_cxx17__normal_iteratorIPPN8logalloc11region_implESt6vectorIS4_SaIS4_EEEElS4_NS0_5__ops14_Iter_comp_valIZNS2_7tracker4impl15compact_on_idleERKSt8functionIFbvEEEUlS4_S4_E_EEEvT_T0_SM_T1_T2_.isra.436.constprop.583
0.80% scylla [kernel.kallsyms] [k]
_raw_read_unlock_irqrestore
0.78% scylla scylla [.]
_ZN17smp_message_queue20flush_response_batchEv
0.75% scylla [kernel.kallsyms] [k]
utrace_report_syscall_entry
0.63% scylla [kernel.kallsyms] [k] task_utrace_struct
0.50% scylla [kernel.kallsyms] [k] native_read_tsc
0.37% scylla [kernel.kallsyms] [k]
utrace_report_syscall_exit
0.32% scylla [kernel.kallsyms] [k] delay_tsc
0.28% scylla scylla [.]
_ZN17smp_message_queue13process_queueILm2EZNS_16process_incomingEvEUlPNS_9work_itemEE_EEmRNS_8lf_queueET0_.isra.2371
0.26% scylla scylla [.]
_ZN17smp_message_queue13process_queueILm4EZNS_19process_completionsEvEUlPNS_9work_itemEE_EEmRNS_8lf_queueET0_.isra.2278
0.20% scylla [kernel.kallsyms] [k] _raw_spin_lock_irq
0.18% scylla scylla [.]
_ZN6memory9cpu_pages4freeEPv
0.18% scylla liblz4.so.1.0.0 [.] LZ4_compress
0.17% scylla scylla [.]
_ZN7reactor9run_tasksER15circular_bufferISt10unique_ptrI4taskSt14default_deleteIS2_EESaIS5_EE
Now this is not something I can run to monitor a production load.
Is there any way I can improve scaling?