OpenSolaris Community: Observability
View the leaders for this community
Community Observers
OpenSolaris Observability
Over the course of history, we've done a lot of debugging in Solaris. Whether
it's processes misbehaving, performance degredations, system panics, or hardware
failure, there is one key element of any approach:
If you cannot observe the problem, you cannot fix it.
To this end, we have spent a great deal of time on robust tools to observe all
aspects of system behavior. Some of these have been around forever; others are
relatively new. They all make the lives of administrators and developers much
easier. This community will serve to explore existing Solaris tools and
features, as well as plan the next generation of OpenSolaris observability
tools.
We've divided this rather large cross section of Solaris functionality into
several specific areas, described here. Most of these just link to the relevant
code. Over time, those that are sufficiently complex will link to more detailed
documentation.
Process Observability
| DTrace |
A fantastic tool that can observe process and system behavior and tie it all
together. Check out the DTrace community for more information. |
| truss |
A tool for examining system calls as well library calls made by an
application. |
| ptools |
A suite of process observability tools. |
| libproc |
A common library used to simplify examining and manipulating
processes. |
| procfs |
The /proc filesystem, upon which libproc and all process tools are
built. |
| MDB |
The modular debugger, the standard debugger for Solaris. See the MDB
community for more information |
| ps |
Basic process monitoring tool. Displays a snapshot of current process state
on the machine. |
| prstat |
The standard process monitoring tool for Solaris, similar to
top. |
| plockstat |
The userland equiavlent of lockstat(1M), used to monitor lock
events and aid in MT scalability. Built on top of DTrace |
System Observability
| DTrace |
Once again, the indispensible system monitoring tool. |
|
kstat
|
A kernel statistics framework used by a variety of tools to examine system
wide staistics. |
|
NUMA
|
Tools for examining NUMA (Non Uniform Memory Architecture) characteristics
on Solaris. |
| ctfs |
The contract filesystem, used to observe and manipulate process contracts. |
| objfs |
The object filesystem, used to examine kernel module symbols and CTF data from
userland. |
mpstat
iostat
vmstat |
Tools to report on basic system statistics. |
| lockstat |
Tools to examine kernel locking statistic, as well as basic profiling data. Implemented using
DTrace |
| intrstat |
Tools to examine kernel interrupt statistics. Implemented using DTrace |
| trapstat |
SPARC-only tool to examine trap statistics. |
Network Observability
| snoop |
The standard tool for observing network packets. |
| netstat |
Tool for observing network connection status. |
| nfsstat |
Basic tool for monitoring NFS statistics. |
Hardware Observability
| FMA |
Fault Management Architecture, a unified system of hardware fault diagnosis, repair, and
reporting. |
| CPC |
A system to gather and report CPU performance counter information. Consists of kernel
components, libcpc,
cpustat, and
cputrack |
| psrinfo |
Report on current processor status and configuration |
| prtconf |
Display the system device tree, optionally in excrutiating detail. |
| prtpicl |
Similar to prtconf, prints out the state of the device nodes maintained by the PICL daemon. |
| prtdiag |
Display summary of attached hardware components. |
| prtfru |
SPARC only - Display FRUID information for system or domain. |
Post Mortem Observability
| MDB |
The standard debugger. See the MDB community for more information. |
| dumpadm |
Configures kernel crash dump generation. |
| coreadm |
Configures process core file generation. |
| CTF |
Compact C Type Format, the format used to store type information in the kernel and userland.
Consumed by MDB and DTrace, among others. |
| process core
generation |
The elfcore() routine, which is responsible for the nuts and bolts of core file
generation. |
| kernel panic
code |
The entry point to the kernel panic code. |
| gcore |
The gcore(1) utility for taking a core dump of a live running process without perturbing the process. |
Unbundled Tools
More to come...
Blogs
Jul 1, 3:49 PM
As I mentioned in my previous post , I wrote an article about the hybrid storage pool (HSP); that article appears in the recently released July issue of Communications of the ACM . You can find it ...
Jun 30, 11:35 PM
The interest in DTrace on Linux is heating up again -- this time in an inferno on the Linux 2008 Kernel Summit discussion list . Under discussion is SystemTap, the Linux-born DTrace-knockoff , with ...
Jun 11, 2:24 AM
Jonathan had a terrific post yesterday that does an excellent job of presenting Sun's strategy for flash for the next few years. With my colleagues at Fishworks, an advanced product development team, ...
Jun 7, 3:29 PM
Awhile ago I posted about a problem with Apple's port of DTrace to Mac OS X. The heart of the issue is that their port would silently drop data such that certain experiments would be quietly invalid. ...
May 31, 11:58 PM
Like several hundred others, I spent today at Berkeley at a tribute to honor Jim Gray . While many there today had collaborated with Jim in some capacity, my own connection to him is tertiary at ...
|