Instrumental Server Monitoring Tools
Instrumental is an application monitoring platform built for developers who want a better understanding of their production software. Powerful tools, like the Instrumental Query Language, combined with an exploration-focused interface allow you to get real answers to complex questions, in real-time.
This tool suite supports server monitoring through the instrumental_server
daemon. It provides high-data reliability at high scale.
Installation
instrumental_tools
is currently officially supported on 32-bit and 64-bit Linux, Windows systems and Mac OS X. There are prebuilt packages available for Debian, Ubuntu, RHEL and Win32 systems.
Installation instructions for supported platforms is available in INSTALL.md. The recommended installation method is to use a prebuilt package, which will automatically install the application as a service in your operating system's startup list.
Once you've installed the package, you will want to edit the /etc/instrumental.yml
file with your Instrumental project token. Example /etc/instrumental.yml
:
api_key: YOUR_PROJECT_API_TOKEN
Metrics
By default, Instrumental Tools will collect metrics on the following server data:
- CPU (
user
,nice
,system
,idle
,iowait
andtotal in use
) - Load (at 1 minute, 5 minute and 15 minute intervals)
- Memory (
used
,free
,buffers
,cached
,free_percent
) - Swap (
used
,free
,free_percent
) - Disk Capacity (
total
,used
,available
,available percent
for all mounted disks) - Disk Usage (
percent_utilization
for all mounted disks) - Filesystem stats (
open_files
,max_open_files
)
Monitoring Services & Other Processes
Instrumental Tools monitors other processes through a powerful plugin system built on binary and shell scripts. Plugin installation and development instructions are listed in PLUGIN_SCRIPTS.md. Existing plugins include:
Command Line Usage
Basic usage:
instrument_server -k <API_KEY>
To start instrument_server
as a daemon:
instrument_server -k <API_KEY> start
The API key can also be provided by setting the INSTRUMENTAL_TOKEN environment variable, which eliminates the need to supply the key via command line option.
By default, instrument_server will use the hostname of the current host when reporting metrics, e.g. 'hostname.cpu.in_use'. To specify a different hostname:
instrument_server -k <API_KEY> -H <HOSTNAME>
The start
command will start and detach the process. You may issue additional commands to the process like:
-
stop
- stop the currently runninginstrument_server
process -
restart
- restart the currently runninginstrument_server
process -
foreground
- run the process in the foreground instead of detaching -
status
- display daemon status (running, stopped) -
clean
- remove any files created by the daemon -
kill
- forcibly halt the daemon and remove its pid file
Capistrano Integration
Add require "instrumental_tools/capistrano"
to your capistrano
configuration and instrument_server
will be restarted after your
deploy is finished. Additionally, you will need to add a new variable
to your capistrano file.
set :instrumental_key, "API_KEY"
The following configuration will be added:
after "deploy", "instrumental:restart_instrument_server"
after "deploy:migrations", "instrumental:restart_instrument_server"
By default, this will attempt to restart the instrument_server
command
on all the servers specified in your configuration. If you need to
limit the servers on which you restart the server, you can do
something like this in your capistrano configuration:
namespaces[:instrumental].tasks[:restart_instrument_server].options[:roles] = [:web, :worker]
Troubleshooting & Help
We are here to help! Email us at support@instrumentalapp.com.