Installing Graphite on aws linux

Etsy has followed the pattern of yahoo and flickr in developing a philosophy of measurement. Click here to see their post.

They created a node.js application called StatsD which makes capturing the measurements easier.  It’s now open source.

Before you can use StatsD, you will want to install Graphite.  StatsD can use multiple backends for graphing, it was originally designed for use with graphite.

Here are the steps to getting graphite installed on aws.  I chose to use the Amazon linux ami.

  1. Open the graphite installation page.
  2. Login to the aws console.
  3. Create an ec2 instance (type Amazon linux which is based on centos).
  4. Be sure your security group allows http traffic through on port 80 and ssh so you can access it and UDP on port 8125.
  5. Login to the new instance (using ssh, keys and default username ec2-user)
  6. Create a new user to run the service (e.g., newuser)
  7. sodu -i (change to root for a while)
  8. yum update (updates all installed packages)
  9. sudo yum install git (will be needed to install graphite)
  10. Following instructions from graphite page:
  11. yum install python (ensure python is installed.  You’ll need the version later.)
  12. Install pre-requisites
  13. yum install pycairo
  14. yum install mod_python
  15. easy_install pip (needed to install django)
  16. pip install django
  17. yum install -y gcc make (needed to install packages later)
  18. skipping ldap for now
  19. yum install python-memcached
  20. skip sql lite for now
  21. yum install mod_wsgi (referenced in the graphite http config file.  be sure it’s installed).
  22. cd /opt
  23. mkdir graphite
  24. chown newuser:newuser graphite
  25. mkdir carbon
  26. chown newuser:newuser carbon
  27. mkdir whisper
  28. chown newuser:newuser whisper
  29. su newuser (change to the newser for now)
  30. cd /opt (if necessary)
  31. git clone graphite
  32. git clone carbon
  33. git clone whisper
  34. Follow directions from graphite install page to change branch to 0.9.x for all three repos
  35. Follow direction to install whisper & carbon
  36. Follow directions to configure carbon except get info for configuration files from StatsD page.
  37. Follow directions on graphite install page to configure graphite webapp
  38. Configure apache.  (The apache configuration file is located at /etc/httpd/conf/httpd.conf and explicitly include /etc/http/conf.d/*.conf).  Copy the example config file to that location:
    cp /opt/graphite/examples/example-graphite-vhost.conf graphite.conf /etc/httpd/conf.d/graphite.conf
  39. service httpd start (not started by default.  Need to set it for auto start)
  40. /etc/init.d/httpd reload (not necessary but for clarity with the graphite install instructions and to make sure the config file works.)
  41. exit (return to working as the “newuser”)
  42. Initial Database Creation (from graphite install page)
  43. cd /opt/graphite/webapp/graphite
  44. You need to create a file.
    Edit  Find the datbase section and uncomment:
    ‘default’: {
    ‘NAME’: ‘/opt/graphite/storage/graphite.db’,
    ‘ENGINE’: ‘django.db.backends.sqlite3’,
    ‘USER’: ”,
    ‘PASSWORD’: ”,
    ‘HOST’: ”,
    ‘PORT’: ”
  45. Create a root database user when prompted.
  46. changed owner of graphite.db to apache:apache
  47. Had an apache error in the logs about incorrect python version.  Removed all referencces to mod_python as described in this post.  Basically if you’re using mod_wsdi you shouldn’t be using mod_python.
  48. Restarted apache:  sudo service https restart
  49. Received a 404 error at this point.  Found the error in the graphite logs /opt/graphite/storage/logs/webapp/error.log
  50. cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
  51. Now I’m getting a 500 internal server error.  Progress 😉
  52. There is an error in the graphite webapp error.log file:
    Permission denied: ‘/opt/graphite/storage/log/webapp/info.log’
  53. chown apache:apache /opt/graphite/storage
  54. sudo service httpd restart
  55. chown apache:apache /opt/graphite/storage
  56. chown apache:apache /opt/graphite/storage/log/
  57. chown apache:apache /opt/graphite/storage/log/webapp/
  58. cd /opt/graphite/storage/log/webapp/
  59. chown apache:apache *
  60. service httpd restart

It works!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s