Latest graphite on Amazon Linux at AWS

NOTE: someone broke master branch, so reverting to 0.9.x

1. yum update -y;reboot
2. yum -y groupinstall "Development Tools"
3. yum -y install git-core python-pip mlocate
4. yum -y install python-rrdtool pycairo-devel
5. mkdir -p /root/graphite-install
6. cd /root/graphite-install
7. git clone https://github.com/graphite-project/carbon.git
8. git clone https://github.com/graphite-project/whisper.git
9. git clone https://github.com/graphite-project/ceres
10. git clone https://github.com/graphite-project/graphite-web.git
11. cd /root/graphite-install/whisper
12. git checkout 0.9.x
13. python setup.py install
14. cd /root/graphite-install/ceres
15. # only has master branch, but it works with others
16. pip install -r requirements.txt
17. python setup.py install
18. cd /root/graphite-install/carbon
19. git checkout 0.9.x
20. pip install -r requirements.txt
21. python setup.py install
22. cd /root/graphite-install/graphite-web
23. git checkout 0.9.x
24. yum install libffi-devel -y # some developer broke things, add this
25. pip install -r requirements.txt
26. python check-dependencies.py
27. python setup.py install 
28. cp -v /opt/graphite/conf/carbon.conf.example /opt/graphite/conf/carbon.conf
29. cp -v /opt/graphite/conf/storage-schemas.conf.example /opt/graphite/conf/storage-schemas.conf
30. cp -v /opt/graphite/conf/storage-aggregation.conf.example /opt/graphite/conf/storage-aggregation.conf
31. cp -v /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py
32. cp -v /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
33. vi /opt/graphite/webapp/graphite/local_settings.py
1. set SECRET_KEY param
2. uncomment DATABASE section
34. add these lines to /root/.bash_profile
1. export GRAPHITE_ROOT=/opt/graphite
2. export PYTHONPATH=$GRAPHITE_ROOT/webapp:$GRAPHITE_ROOT/whisper
35. source /root/.bash_profile
36. django-admin.py syncdb --settings=graphite.settings
37. yum -y install httpd24 mod24_wsgi
38. mkdir /etc/httpd/vhosts.d
39. add to end of /etc/httpd/conf/httpd.conf
1. IncludeOptional vhosts.d/*.conf
40. create /etc/httpd/vhosts.d/graphite.conf with below
41. mkdir /var/run/wsgi;chmod -v 777 /var/run/wsgi
42. apachectl -S
1. check for errors
43. service httpd start
44. /opt/graphite/bin/carbon-cache.py start
45. find /opt/graphite/storage -type d -exec chmod -v 777 {} \;
46. find /opt/graphite/storage -type f -exec chmod -v 666 {} \;
47. echo '127.0.1.3 graphite' >> /etc/hosts
48. curl -I http://graphite
49. on local machine
1. make same /etc/hosts entry
2. port forward 127.0.1.3:8081 to remote-server:80
3. curl -I http://graphite:8081
50. END

WSGISocketPrefix /var/run/wsgi

<VirtualHost *:80>
    ServerName graphite
    DocumentRoot "/opt/graphite/webapp"
    ErrorLog logs/webapp_error.log
    CustomLog logs/webapp_access.log common

    WSGIDaemonProcess graphite processes=5 threads=5 display-name='%{GROUP}' inactivity-timeout=120
    WSGIProcessGroup graphite
    WSGIApplicationGroup %{GLOBAL}
    WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite application-group=%{GLOBAL}
    WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi

    Alias /content/ /opt/graphite/webapp/content/
    <Location "/content/">
        SetHandler None
        Require all granted
    </Location>
  
    Alias /media/ "@DJANGO_ROOT@/contrib/admin/media/"
    <Location "/media/">
        SetHandler None
        Require all granted
    </Location>
 
    <Directory /opt/graphite/conf/>
        Require all granted
    </Directory>
</VirtualHost>

Puppet 3.1.1 using Ruby 1.9.3 on Amazon Linux

UPDATE: Amazon Linux is now on Ruby 2.x, so below is DEPRECATED for new Amazon Linux images. But parts may be useful.

1. yum -y remove ruby
2. yum -y install ruby19
3. gem install --no-rdoc --no-ri puppet --version=3.1.1 
4.  /usr/local/bin/puppet -V
1. add /usr/local/bin to $PATH of users that need it
5.  vi /usr/local/share/gems1.9/gems/facter-1.7.5/lib/facter/ec2.rb
1. change line 28 to:
1. if (Facter::Util::EC2.can_connect?)
2. Reference: http://projects.puppetlabs.com/issues/7559
3. existing line 28 is much longer

 If you manage user passwords with Puppet

1. yum -y install ruby19-devel
2. yum -y groupinstall "Developer Tools"
3. gem install --no-rdoc --no-ri ruby-shadow

Building latest collectd on Amazon Linux server in AWS

1. yum install byacc flex automake libtool libgcrypt-devel glib2-devel libtool-ltdl-devel perl-ExtUtils-MakeMaker
1. comments below imply this may be necessary: apt-get install bison
2. git clone https://github.com/collectd/collectd.git
3. cd collectd
4. ./build.sh
5. ./configure
6. make 

 Some helpful libs, start over to use them

1. yum install lvm2-devel net-snmp-devel liboping-devel libpcap-devel libesmtp-devel libcurl-devel libmnl-devel

Dated: Nov, 2013

Zenoss on Amazon Linux

Someone saved my life tonight, Sugarbear: http://binarynature.blogspot.com/2012/11/zenoss-core-4-installation.html

Also taken: http://magazine.redhat.com/2007/12/04/hacking-rpms-with-rpmrebuild/

WARNING: USE AT OWN RISK

1. remove non-critical, unmet dependencies in rpm
1. get good version of rpmrebuild
1. wget ftp://fr2.rpmfind.net/linux/opensuse/ports/update/12.3/noarch/rpmrebuild-2.9-7.4.1.noarch.rpm
2. yum localinstall rpmrebuild-2.9-7.4.1.noarch.rpm
2. wget http://downloads.sourceforge.net/project/zenoss/zenoss-4.2/zenoss-4.2.4/4.2.4-1897/zenoss_core-4.2.4-1897.el6.x86_64.rpm
3. rpmrebuild -e -n -p zenoss_core-4.2.4-1897.el6.x86_64.rpm
1. remove lines
1. Requires:      libgcj
2. %dir %attr(0755, root, root) "/etc/sudoers.d"
3. # ensure that the system uses the /etc/sudoers.d directory
4. SUDOERSD_TOKEN="#includedir /etc/sudoers.d"
5. SUDOERSD_FOUND=`/bin/egrep "^$SUDOERSD_TOKEN" /etc/sudoers`
6. if [ -z "$SUDOERSD_FOUND" ]; then
7.    echo "# zenoss rpm, ensure that /etc/sudoers.d loads" >> /etc/sudoers
8.    echo $SUDOERSD_TOKEN >> /etc/sudoers
9. fi
2. save file and "continue"
3. note place resultant file is created, copy to local working directory
1. if you can't find it where it is supposed to be
1. yum -y install mlocate
2. updatedb
3. locate zenoss
2. ntp
1. ntpq -pn
2. make sure ntp is installed and running
3. java
1. remove openjdk and install oracle java
1. yum -y remove java jdk
2. wget -N -O jre-6u31-linux-x64-rpm.bin http://javadl.sun.com/webapps/download/AutoDL?BundleId=59622
3. chmod +x jre-6u31-linux-x64-rpm.bin
4. ./jre-6u31-linux-x64-rpm.bin
2. verify
1. java -version
2. echo $JAVA_HOME
3. cover missing libgcj, not strictly necessary
1. yum -y install ecj
2. only for running java classes natively instead of using JVM
1. about as obscure as it gets
4. Add extra repos
1. rpm --import http://dev.zenoss.org/yum/RPM-GPG-KEY-zenoss
2. yum-config-manager --enable epel
3. yum --nogpgcheck -y localinstall http://apt.sw.be/redhat/el6/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
5. install some deps
1. yum -y install memcached net-snmp
6. start some dep services
1. service memcached start
2. service snmpd start
3. chkconfig memcached on
4. chkconfig snmpd on
7. mysql
1. yum -y install mysql-server
2. service mysqld start
3. chkconfig mysqld on
8. install amazon linux compatible zenoss rpm you created above
1. yum --nogpgcheck localinstall zenoss-4.2.4-1897.el6.x86_64.rpm
2. echo > /opt/zenoss/var/zenpack_actions.txt
1. you REALLY don't want these right now, install by-hand later
9. start more dep services
1. service rabbitmq-server start
2. chkconfig rabbitmq-server on
10. start and stop zenoss core, dbs and files are created
1. service zenoss start;sleep 5;service zenoss stop
11. post zenoss updates as zenoss user, then back to root user
1. su -l zenoss
2. wget --no-check-certificate https://raw.github.com/osu-sig/zenoss-autodeploy-4.2.3/master/secure_zenoss.sh
3. sh secure_zenoss.sh
4. egrep 'user|password' $ZENHOME/etc/global.conf | grep -v admin
5. zodbpw=$(grep zodb-password $ZENHOME/etc/global.conf | awk '{print $2}')
6. sed -i.orig "5s/zenoss/$zodbpw/" $ZENHOME/etc/zodb_db_{main,session}.conf
7. tail -n +1 $ZENHOME/etc/zodb_db_{main,session}.conf
8. exit
12. set mysql password for zenoss user using password generated above
1. mysql -u root -p
2. SET PASSWORD FOR 'zenoss'@'localhost' = PASSWORD('18zmcTgYsA+AjczljwQd');
1. sub your password in instead
13. create zenoss user for rabbitmq
1. vim set-rabbitmq-perms.sh
1. grab code below and put in here
2. sh set-rabbitmq-perms.sh
3. service rabbitmq-server restart
14. start zenoss final time
1. service zenoss restart
2. chkconfig zenoss on
3. verify
1. su -l zenoss -c 'zenoss status'
2. rabbitmqctl -p /zenoss list_queues
15. Add this line to the end of /etc/sudo.conf
1. #includedir /etc/sudoers.d
2. was removed during rpm tweak

NOTE: this is probably not perfect, but very close, please, send corrections

###################################################

## code for set-rabbitmq-perms.sh

#!/usr/bin/env bash

set -e
ZENHOME="/opt/zenoss"

VHOSTS="/zenoss"

USER="zenoss"

PASS="grep amqppassword \$ZENHOME/etc/global.conf | awk '{print \$2}'"

if [ $(id -u) -eq 0 ]

then

RABBITMQCTL=$(which rabbitmqctl)

$RABBITMQCTL stop_app

$RABBITMQCTL reset

$RABBITMQCTL start_app

$RABBITMQCTL add_user "$USER" "$(su -l zenoss -c "$PASS")"

for vhost in $VHOSTS; do

$RABBITMQCTL add_vhost "$vhost"

$RABBITMQCTL set_permissions -p "$vhost" "$USER" '.*' '.*' '.*'

done

exit 0

else

echo "Error: Run this script as the root user." >&2

exit 1

fi


REFERENCE

1. http://pastebin.com/sTRU0gki
2. http://pastebin.com/CLcq3xNb 

vncserver on Amazon Linux on Amazon's AWS

NOTE: this first step may be outdated in new versions of Amazon Linux, which provides libjpeg-turbo.

First, get a good version of libjpeg-turbo:

1. yum clean all
2. yum --enablerepo=amzn-preview install libjpeg-turbo
3. see: https://forums.aws.amazon.com/thread.jspa?threadID=121128

Necessary packages:

1. gnutls-2.8.5-10.el6.x86_64.rpm
2. libfontenc-1.0.5-2.el6.x86_64.rpm
3. libtasn1-2.3-3.el6.x86_64.rpm
4. libxdmcp-1.1.1-7.ram1.x86_64.rpm
5. libXfont-1.4.5-2.el6.x86_64.rpm
6. libxkbfile-1.0.6-1.1.el6.x86_64.rpm
7. libXmu-1.1.1-2.el6.x86_64.rpm
8. mesa-dri-drivers-9.0-0.7.el6.x86_64.rpm
9. pixman-0.26.2-5.el6_4.x86_64.rpm
10. tigervnc-license-1.3.0-16.el6.noarch.rpm
11. tigervnc-server-1.3.0-16.el6.x86_64.rpm
12. tigervnc-server-minimal-1.3.0-16.el6.x86_64.rpm
13. xkeyboard-config-2.6-6.el6.noarch.rpm
14. xorg-x11-proto-devel-7.6-25.el6.noarch.rpm
15. xorg-x11-xauth-1.0.2-7.1.el6.x86_64.rpm
16. xorg-x11-xkb-utils-7.7-4.el6.x86_64.rpm

Necessary packages for fluxbox:

1. pyxdg-0.18-1.el6.noarch.rpm

Here's a link to all the packages needed: rpms

Start by trying to install tigervnc-server. That will fail. Follow the dependencies down, installing packages one by one as they come up. If they fail, install their dependencies first.

These packages were special cases

1. mesa-dri-drivers-9.0-0.7.el6.x86_64.rpm
1. just "--nodeps" install this one
2. the libraries of the dependencies are never called from my experience
2. libXtst-1.2.1-2.el6.x86_64.rpm
1. there is a more recent version of this already in Amazon Linux, so just skip this one and "--nodeps" the parent package, which is libXmu-1.1.1-2.el6.x86_64.rpm

If not found, add location of libXdmcp.so.6 to /etc/ld.so.conf and run "ldconfig", e.g. locations is /usr/lib/libXdmcp.so.6 so add the line "/usr/lib".

Once I got the vncserver up with

  vncserver :66 -localhost

bizarrely, it was listening on port 5966 and some 6066 port. The 5966 port is the one you want.

See previous post on Redhat VNC for details, click here.