Showing posts with label aws. Show all posts
Showing posts with label aws. Show all posts

Thursday, 4 February 2016

SSH config: command for bastion server proxying


Host 51.21.21.11
  IdentityFile ~/.ssh/mykey.pem
  User ec2-user
  ForwardAgent yes

Host 10.*
  ProxyCommand ssh ec2-user@51.21.21.11 nc %h 22
  User ec2-user
  IdentityFile ~/.ssh/mykey.pem
  ForwardAgent yes

Put your ssh public key in ~/.ssh/authorized_keys along the path.

Friday, 17 July 2015

Slow loading mysql dump into RDS

It's the PIOPS, stupid. Try cranking them up, e.g. 300GB volume at 3000 IOPS on your RDS instance when you create it.

See if that helps.

I just went from load taking days to load taking only hours.

This is an official AWS collection point for data, so please leave a comment on your experience. jk.

Friday, 1 August 2014

AWS cli: rework EBS volume on AMI launch: switch to SSD, "delete on termination" to true

aws
    ec2
        run-instances
            --image-id
                ami-aaaaaa
            --instance-type
                hi1.4xlarge
            --security-group-ids
                sg-eeeeeeee
            --subnet-id
                subnet-cccccccc
            --block-device-mappings
                '[
                    {
                        "DeviceName":"/dev/sdb",
                        "VirtualName":"ephemeral0"
                    },
                    {
                        "DeviceName":"/dev/sdc",
                        "VirtualName":"ephemeral1"
                    },
                    {
                        "DeviceName":"/dev/sdd",
                        "Ebs":
                        {
                            "SnapshotId":"snap-6",
                            "VolumeType":"gp2",
                            "DeleteOnTermination":"true"
                        }
                    }
                ]'
            --region
                us-east-1


Switch to high IOPS
aws
    ec2
        run-instances
            --image-id
                ami-aaaaaa
            --instance-type
                hi1.4xlarge
            --security-group-ids
                sg-eeeeeeee
            --subnet-id
                subnet-cccccccc
            --block-device-mappings
                '[
                    {
                        "DeviceName":"/dev/sdb",
                        "VirtualName":"ephemeral0"
                    },
                    {
                        "DeviceName":"/dev/sdc",
                        "VirtualName":"ephemeral1"
                    },
                    {
                        "DeviceName":"/dev/sdd",
                        "Ebs":
                        {
                            "SnapshotId":"snap-6",
                            "VolumeType":"io1",
                            "Iops":4000,
                            "DeleteOnTermination":"true"
                        }
                    }
                ]'
            --region
                us-east-1

Tuesday, 8 July 2014

Latest graphite on Amazon Linux at AWS

NOTE: someone broke master branch, so reverting to 0.9.x
  1. yum update -y;reboot
  2. yum -y groupinstall "Development Tools"
  3. yum -y install git-core python-pip mlocate
  4. yum -y install python-rrdtool pycairo-devel
  5. mkdir -p /root/graphite-install
  6. cd /root/graphite-install
  7. git clone https://github.com/graphite-project/carbon.git
  8. git clone https://github.com/graphite-project/whisper.git
  9. git clone https://github.com/graphite-project/ceres
  10. git clone https://github.com/graphite-project/graphite-web.git
  11. cd /root/graphite-install/whisper
  12. git checkout 0.9.x
  13. python setup.py install
  14. cd /root/graphite-install/ceres
  15. # only has master branch, but it works with others
  16. pip install -r requirements.txt
  17. python setup.py install
  18. cd /root/graphite-install/carbon
  19. git checkout 0.9.x
  20. pip install -r requirements.txt
  21. python setup.py install
  22. cd /root/graphite-install/graphite-web
  23. git checkout 0.9.x
  24. yum install libffi-devel -y # some developer broke things, add this
  25. pip install -r requirements.txt
  26. python check-dependencies.py
  27. python setup.py install 
  28. cp -v /opt/graphite/conf/carbon.conf.example /opt/graphite/conf/carbon.conf
  29. cp -v /opt/graphite/conf/storage-schemas.conf.example /opt/graphite/conf/storage-schemas.conf
  30. cp -v /opt/graphite/conf/storage-aggregation.conf.example /opt/graphite/conf/storage-aggregation.conf
  31. cp -v /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py
  32. cp -v /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi
  33. vi /opt/graphite/webapp/graphite/local_settings.py
    1. set SECRET_KEY param
    2. uncomment DATABASE section
  34. add these lines to /root/.bash_profile
    1. export GRAPHITE_ROOT=/opt/graphite
    2. export PYTHONPATH=$GRAPHITE_ROOT/webapp:$GRAPHITE_ROOT/whisper
  35. source /root/.bash_profile
  36. django-admin.py syncdb --settings=graphite.settings
  37. yum -y install httpd24 mod24_wsgi
  38. mkdir /etc/httpd/vhosts.d
  39. add to end of /etc/httpd/conf/httpd.conf
    1. IncludeOptional vhosts.d/*.conf
  40. create /etc/httpd/vhosts.d/graphite.conf with below
  41. mkdir /var/run/wsgi;chmod -v 777 /var/run/wsgi
  42. apachectl -S
    1. check for errors
  43. service httpd start
  44. /opt/graphite/bin/carbon-cache.py start
  45. find /opt/graphite/storage -type d -exec chmod -v 777 {} \;
  46. find /opt/graphite/storage -type f -exec chmod -v 666 {} \;
  47. echo '127.0.1.3 graphite' >> /etc/hosts
  48. curl -I http://graphite
  49. on local machine
    1. make same /etc/hosts entry
    2. port forward 127.0.1.3:8081 to remote-server:80
    3. curl -I http://graphite:8081
  50. END

WSGISocketPrefix /var/run/wsgi

<VirtualHost *:80>
    ServerName graphite
    DocumentRoot "/opt/graphite/webapp"
    ErrorLog logs/webapp_error.log
    CustomLog logs/webapp_access.log common

    WSGIDaemonProcess graphite processes=5 threads=5 display-name='%{GROUP}' inactivity-timeout=120
    WSGIProcessGroup graphite
    WSGIApplicationGroup %{GLOBAL}
    WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite application-group=%{GLOBAL}
    WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi

    Alias /content/ /opt/graphite/webapp/content/
    <Location "/content/">
        SetHandler None
        Require all granted
    </Location>
  
    Alias /media/ "@DJANGO_ROOT@/contrib/admin/media/"
    <Location "/media/">
        SetHandler None
        Require all granted
    </Location>
 
    <Directory /opt/graphite/conf/>
        Require all granted
    </Directory>
</VirtualHost>


Friday, 6 December 2013

aws ec2 cli filter by tag name and value

  1. aws ec2 describe-instances --filter Name=tag:Name,Values=ADS-prod-ads
The horrible syntax of "and" filters:
  1.  aws ec2 describe-instances 
    1. --filter 
      1. '{"Name":"tag:backup","Values":["yes"]}' 
      2. '{"Name":"instance-state-name","Values":["running"]}' 
    2. non-breakout view below
aws ec2 describe-instances --filter '{"Name":"tag:backup","Values":["yes"]}' '{"Name":"instance-state-name","Values":["running"]}'

Friday, 8 November 2013

Building latest collectd on Amazon Linux server in AWS

  1. yum install byacc flex automake libtool libgcrypt-devel glib2-devel libtool-ltdl-devel perl-ExtUtils-MakeMaker
    1. comments below imply this may be necessary: apt-get install bison
  2. git clone https://github.com/collectd/collectd.git
  3. cd collectd
  4. ./build.sh
  5. ./configure
  6. make 
 Some helpful libs, start over to use them
  1. yum install lvm2-devel net-snmp-devel liboping-devel libpcap-devel libesmtp-devel libcurl-devel libmnl-devel
Dated: Nov, 2013

Saturday, 2 November 2013

Zenoss on Amazon Linux

Someone saved my life tonight, Sugarbear: http://binarynature.blogspot.com/2012/11/zenoss-core-4-installation.html

Also taken: http://magazine.redhat.com/2007/12/04/hacking-rpms-with-rpmrebuild/

WARNING: USE AT OWN RISK
  1. remove non-critical, unmet dependencies in rpm
    1. get good version of rpmrebuild
      1. wget ftp://fr2.rpmfind.net/linux/opensuse/ports/update/12.3/noarch/rpmrebuild-2.9-7.4.1.noarch.rpm
      2. yum localinstall rpmrebuild-2.9-7.4.1.noarch.rpm
    2. wget http://downloads.sourceforge.net/project/zenoss/zenoss-4.2/zenoss-4.2.4/4.2.4-1897/zenoss_core-4.2.4-1897.el6.x86_64.rpm
    3. rpmrebuild -e -n -p zenoss_core-4.2.4-1897.el6.x86_64.rpm
      1. remove lines
        1. Requires:      libgcj
        2. %dir %attr(0755, root, root) "/etc/sudoers.d"
        3. # ensure that the system uses the /etc/sudoers.d directory
        4. SUDOERSD_TOKEN="#includedir /etc/sudoers.d"
        5. SUDOERSD_FOUND=`/bin/egrep "^$SUDOERSD_TOKEN" /etc/sudoers`
        6. if [ -z "$SUDOERSD_FOUND" ]; then
        7.    echo "# zenoss rpm, ensure that /etc/sudoers.d loads" >> /etc/sudoers
        8.    echo $SUDOERSD_TOKEN >> /etc/sudoers
        9. fi
      2. save file and "continue"
      3. note place resultant file is created, copy to local working directory
        1. if you can't find it where it is supposed to be
          1. yum -y install mlocate
          2. updatedb
          3. locate zenoss
  2. ntp
    1. ntpq -pn
    2. make sure ntp is installed and running
  3. java
    1. remove openjdk and install oracle java
      1. yum -y remove java jdk
      2. wget -N -O jre-6u31-linux-x64-rpm.bin http://javadl.sun.com/webapps/download/AutoDL?BundleId=59622
      3. chmod +x jre-6u31-linux-x64-rpm.bin
      4. ./jre-6u31-linux-x64-rpm.bin
    2. verify
      1. java -version
      2. echo $JAVA_HOME
    3. cover missing libgcj, not strictly necessary
      1. yum -y install ecj
      2. only for running java classes natively instead of using JVM
        1. about as obscure as it gets
  4. Add extra repos
    1. rpm --import http://dev.zenoss.org/yum/RPM-GPG-KEY-zenoss
    2. yum-config-manager --enable epel
    3. yum --nogpgcheck -y localinstall http://apt.sw.be/redhat/el6/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el6.rf.x86_64.rpm
  5. install some deps
    1. yum -y install memcached net-snmp
  6. start some dep services
    1. service memcached start
    2. service snmpd start
    3. chkconfig memcached on
    4. chkconfig snmpd on
  7. mysql
    1. yum -y install mysql-server
    2. service mysqld start
    3. chkconfig mysqld on
  8. install amazon linux compatible zenoss rpm you created above
    1. yum --nogpgcheck localinstall zenoss-4.2.4-1897.el6.x86_64.rpm
    2. echo > /opt/zenoss/var/zenpack_actions.txt
      1. you REALLY don't want these right now, install by-hand later
  9. start more dep services
    1. service rabbitmq-server start
    2. chkconfig rabbitmq-server on
  10. start and stop zenoss core, dbs and files are created
    1. service zenoss start;sleep 5;service zenoss stop
  11. post zenoss updates as zenoss user, then back to root user
    1. su -l zenoss
    2. wget --no-check-certificate https://raw.github.com/osu-sig/zenoss-autodeploy-4.2.3/master/secure_zenoss.sh
    3. sh secure_zenoss.sh
    4. egrep 'user|password' $ZENHOME/etc/global.conf | grep -v admin
    5. zodbpw=$(grep zodb-password $ZENHOME/etc/global.conf | awk '{print $2}')
    6. sed -i.orig "5s/zenoss/$zodbpw/" $ZENHOME/etc/zodb_db_{main,session}.conf
    7. tail -n +1 $ZENHOME/etc/zodb_db_{main,session}.conf
    8. exit
  12. set mysql password for zenoss user using password generated above
    1. mysql -u root -p
    2. SET PASSWORD FOR 'zenoss'@'localhost' = PASSWORD('18zmcTgYsA+AjczljwQd');
      1. sub your password in instead
  13. create zenoss user for rabbitmq
    1. vim set-rabbitmq-perms.sh
      1. grab code below and put in here
    2. sh set-rabbitmq-perms.sh
    3. service rabbitmq-server restart
  14. start zenoss final time
    1. service zenoss restart
    2. chkconfig zenoss on
    3. verify
      1. su -l zenoss -c 'zenoss status'
      2. rabbitmqctl -p /zenoss list_queues
  15. Add this line to the end of /etc/sudo.conf
    1. #includedir /etc/sudoers.d
    2. was removed during rpm tweak
NOTE: this is probably not perfect, but very close, please, send corrections

###################################################
## code for set-rabbitmq-perms.sh
#!/usr/bin/env bash
set -e
ZENHOME="/opt/zenoss"
VHOSTS="/zenoss"
USER="zenoss"
PASS="grep amqppassword \$ZENHOME/etc/global.conf | awk '{print \$2}'"
if [ $(id -u) -eq 0 ]
then
RABBITMQCTL=$(which rabbitmqctl)
$RABBITMQCTL stop_app
$RABBITMQCTL reset
$RABBITMQCTL start_app
$RABBITMQCTL add_user "$USER" "$(su -l zenoss -c "$PASS")"
for vhost in $VHOSTS; do
$RABBITMQCTL add_vhost "$vhost"
$RABBITMQCTL set_permissions -p "$vhost" "$USER" '.*' '.*' '.*'
done
exit 0
else
echo "Error: Run this script as the root user." >&2
exit 1

Wednesday, 30 October 2013

Latest aws cli tools on Redhat

  1. wget https://s3.amazonaws.com/aws-cli/awscli-bundle.zip
  2. unzip awscli-bundle.zip
  3. sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
 Result is in /usr/local/bin, so set up your PATH accordingly.

Wednesday, 23 October 2013

aws cli run-instances block-device-mappings ephemeral encrypted

aws --version => aws-cli/1.1.1 Python/2.6.8 Linux/3.4.43-43.43.amzn1.x86_64
  1. aws ec2 run-instances 
    1. --image-id 
      1. ami-eeff1122 
    2. --instance-type 
      1. m2.2xlarge 
    3. --security-group-ids 
      1. sg-eeff1122
    4. --subnet-id 
      1. subnet-eeff1122
    5. --private-ip-address 
      1. 10.0.0.2
    6. --user-data 
      1. file://meta_myserver.txt 
    7. --block-device-mappings 
      1. '[{ "DeviceName":"/dev/sdb", "VirtualName":"ephemeral0" }]'
For 50G EBS attached on boot (auto-deleted on terminate unless you override), block device mapping becomes:
  1.  '[{ "DeviceName":"/dev/sdb", "VirtualName":"ephemeral0" },{"DeviceName":"/dev/sdc","Ebs":{"VolumeSize":50}}]'
WARNING: "Ebs" is very case sensitive here.

To encrypt the Ebs volume, add "Encrypted": true to the device params like so:
  1.  {"DeviceName":"/dev/sdc","Ebs":{"VolumeSize":50,"Encrypted": true}}


Tuesday, 1 October 2013

Snapshot AWS instance store as AMI

  1. install api-tools
  2. install ami-tools
  3. generate key / cert
  4. create IAM user 
  5. upload cert
  6. java install / export JAVA_HOME
  7. export key and secret
  8. ec2-bundle-vol 
    1. --user  <AWS acct #> 
    2. --privatekey /myhome/my-key.pem
    3. --cert /myhome/my-cert.pem
    4. --arch x86_64
    5. --destination /var/tmp
    6. --exclude
      1. /backup,
      2. /mnt,
      3. /swapfile
  9. ec2-upload-bundle
    1. --manifest /var/tmp/image.manifest.xml
    2. --bucket mybucket/hostname
    3. --access-key <AWS Key>
    4. --secret-key <AWS Secret>
    5. --location EU
  10. ec2-register
    1. --region eu-west-1
    2. --name "myaminame"
    3. --description "Backing up hostname"
    4. mybucket/hostname/image.manifest.xml
Taken:
  1. http://www.dowdandassociates.com/content/howto-create-an-instance-store-backed-amazon-ec2-ami/
  2. http://www.dowdandassociates.com/content/howto-install-aws-cli-amazon-elastic-compute-cloud-ec2-ami-tools/
  3. http://www.dowdandassociates.com/content/howto-install-aws-cli-amazon-elastic-compute-cloud-ec2-api-tools/
NOTE: the above link's content has typos in very essential parts, proof all steps

Tuesday, 6 August 2013

vncserver on Amazon Linux on Amazon's AWS

NOTE: this first step may be outdated in new versions of Amazon Linux, which provides libjpeg-turbo.

First, get a good version of libjpeg-turbo:
  1. yum clean all
  2. yum --enablerepo=amzn-preview install libjpeg-turbo
  3. see: https://forums.aws.amazon.com/thread.jspa?threadID=121128
Necessary packages:
  1. gnutls-2.8.5-10.el6.x86_64.rpm
  2. libfontenc-1.0.5-2.el6.x86_64.rpm
  3. libtasn1-2.3-3.el6.x86_64.rpm
  4. libxdmcp-1.1.1-7.ram1.x86_64.rpm
  5. libXfont-1.4.5-2.el6.x86_64.rpm
  6. libxkbfile-1.0.6-1.1.el6.x86_64.rpm
  7. libXmu-1.1.1-2.el6.x86_64.rpm
  8. mesa-dri-drivers-9.0-0.7.el6.x86_64.rpm
  9. pixman-0.26.2-5.el6_4.x86_64.rpm
  10. tigervnc-license-1.3.0-16.el6.noarch.rpm
  11. tigervnc-server-1.3.0-16.el6.x86_64.rpm
  12. tigervnc-server-minimal-1.3.0-16.el6.x86_64.rpm
  13. xkeyboard-config-2.6-6.el6.noarch.rpm
  14. xorg-x11-proto-devel-7.6-25.el6.noarch.rpm
  15. xorg-x11-xauth-1.0.2-7.1.el6.x86_64.rpm
  16. xorg-x11-xkb-utils-7.7-4.el6.x86_64.rpm
Necessary packages for fluxbox:
  1. pyxdg-0.18-1.el6.noarch.rpm
Here's a link to all the packages needed: rpms

Start by trying to install tigervnc-server. That will fail. Follow the dependencies down, installing packages one by one as they come up. If they fail, install their dependencies first.

These packages were special cases
  1. mesa-dri-drivers-9.0-0.7.el6.x86_64.rpm
    1. just "--nodeps" install this one
    2. the libraries of the dependencies are never called from my experience
  2. libXtst-1.2.1-2.el6.x86_64.rpm
    1. there is a more recent version of this already in Amazon Linux, so just skip this one and "--nodeps" the parent package, which is libXmu-1.1.1-2.el6.x86_64.rpm
If not found, add location of libXdmcp.so.6 to /etc/ld.so.conf and run "ldconfig", e.g. locations is /usr/lib/libXdmcp.so.6 so add the line "/usr/lib".

Once I got the vncserver up with

  vncserver :66 -localhost

bizarrely, it was listening on port 5966 and some 6066 port. The 5966 port is the one you want.

See previous post on Redhat VNC for details, click here.

Wednesday, 22 May 2013

LVM crypt disks on Linux/AWS



  1. dd if=/dev/urandom of=/keys/xvdm.key bs=1024 count=4
  2. dd if=/dev/urandom of=/keys/xvdn.key bs=1024 count=4
  3. cryptsetup --verbose -y luksFormat /dev/xvdm /keys/xvdm.key
  4. cryptsetup --verbose -y luksFormat /dev/xvdn /keys/xvdn.key
  5. cryptsetup luksOpen /dev/xvdm cryptm --key-file /etc/xvdm.key
  6. cryptsetup luksOpen /dev/xvdn cryptn --key-file /etc/xvdn.key
  7. pvcreate /dev/mapper/cryptm /dev/mapper/cryptn
  8. Add entries to /etc/crypttab for reboots and test somehow
    1. cryptm /dev/xvdm /etc/xvdm.key luks
    2. cryptn /dev/xvdn /etc/xvdn.key luks
Complete LVM setup and add entries to /etc/fstab.

Hint: don't make one, single typo...ever.

Tuesday, 2 April 2013

Set up private, internal DNS for your VPC using Route 53 and unbound

CRITICAL: AWS now offers internal VPC DNS! Below is no longer necessary AFAIK. Woo hoo!

http://aws.amazon.com/about-aws/whats-new/2014/11/05/amazon-route-53-now-supports-private-dns-with-amazon-vpc/

BELOW IS DEPRECATED!
  1. create a Hosted Zone, something like "mydomain.internal"
  2. get the IP addresses of the name servers assigned to your new zone
    1. STRIP OFF '.' at the end of the name servers or BOOM!
  3.  create a new DHCP Options Set
    1. add the IP addresses you gathered above to the domain-name-servers field
  4. Change DHCP Options Set of your VPC by right-clicking it
  5. run sudo dhclient on any already-running instance in the VPC to pick up changes
  6. debug changes have taken place on an instance: cat /etc/resolv.conf

RECOMMEND ALTERNATE SOLUTION: here's a sample unbound.conf I ended up using for a DNS forwarding server within my VPC -- see comments below. I adjusted the "options set" to point at this DNS server instead, 10.0.0.254 in my case.

NOTE: Btw, unbound is available under "epel" yum repo.

server:
        verbosity: 1
        interface: 0.0.0.0
        access-control: 10.0.0.0/16 allow
forward-zone:
       name: "mydomain.internal"
       forward-host: ns-123.awsdns-12.com
       forward-host: ns-234.awsdns-34.biz
       forward-host: ns-567.awsdns-56.net
       forward-host: ns-890.awsdns-78.org
       forward-first: no 
 

See also:

unbound, custom records:  http://sysadminandnetworking.blogspot.com/2014/05/unbound-custom-records.html
unbound, default to google: http://sysadminandnetworking.blogspot.com/2014/05/unbound-default-to-googles-dns.html

Friday, 15 March 2013

Generate IAM certs for users on AWS


  1. openssl genrsa 1024 > username-env-pk.pem
    1. pk stands for private key
  2. openssl req -new -x509 -nodes -sha1 -days 365 -key username-env-pk.pem -outform PEM > username-env-cert.pem
    1. lasts for 365
  3. Paste username-env-cert.pem in to the AWS Signing Certificates area for that user
  4. Give user both username-env-pk.pem and username-env-cert.pem, and wish them luck

vagrant on aws


  1. vagrant plugin install vagrant-aws
  2. vagrant box add aws001 https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
  3. vagrant init
  4. Adapt below and put in the "Vagrantfile" file
  5. vagrant up --provider=aws
  6. vagrant ssh
  7. vagrant destroy
Vagrant.configure("2") do |config|
  config.vm.box = "aws001"

  config.vm.provider :aws do |aws|
    aws.access_key_id = "<your_aws_key_id>"
    aws.secret_access_key = "<your_aws_secret>"
    aws.keypair_name = "<your_keypair_name>"
    aws.ssh_private_key_path = "/home/<you>/.ssh/<your_keypair_name>.pem"

    aws.region = "eu-west-1"
    aws.ami = "ami-01080b75"
    aws.ssh_username = "ubuntu"
  end
end

Wednesday, 13 March 2013

Get provisioned public key for AWS EC2 instance via curl

curl http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key

Thursday, 28 February 2013

Exim: rewrite "From" field


Amazon's AWS SES service requires email be addressed from a particular user.
  1. Add to begin rewrite section
    1. *  root@abc.com  Ffrs
  2. Reload exim

Wednesday, 27 February 2013

AWS SES with exim4 on debian-based Linux



  1. apt-get install exim4
  2. dpkg-reconfigure exim4-config
    1. Select: internet site; mail is sent and received directly using SMTP
    2. IP-addresses to listen on for incoming SMTP connections:
      1. 127.0.0.1 ; ::1 (it's the default anyways)
    3. Take most defaults
    4. Split configuration into small files?
      1. NO!
  3. lsof -nP -i :25
    1. Make sure you aren't allowing the world to connect!
      1. 127.0.0.1:25 is good
  4. AWS -> SES
    1. Verified Senders
      1. Verify one of your existing email addresses
    2. SMTP Settings
      1. Create My SMTP Credentials
      2. Use downloaded credentials.csv file contents for below steps
  5. Edit /etc/exim4/exim4.conf.template
    1. Find ALREADY existing line "public_name = LOGIN"
      1. change to "public_name = OLD_LOGIN"
    2. Add below sections to existing sections in the file
    3. Use info from credentials.csv in place of pointy brackets, e.g. <aws_id> 
  6. service exim4 stop
  7. service exim4 start
  8. tail -F /var/log/exim4/mainlog
    1. Keep this running in another terminal while you do the below
  9. echo test001 | mail -r <email_you_verified_with_aws> -s "Test subject" <a_test_email_address>
    1. e.g. echo test001 | mail -r bob@myawsverified.com -s test bob@testaccount.com
  10. See this to set the "From:" field to your verified user for every email sent from your system
    1. http://sysadminandnetworking.blogspot.com/2013/02/exim-rewrite-from-field.html


begin routers

send_via_ses:
  driver = manualroute
  domains = ! +local_domains
  transport = ses_smtp
  route_list = * email-smtp.us-east-1.amazonaws.com


begin transports

ses_smtp:
  driver = smtp
  port = 25
  hosts_require_auth = $host_address
  hosts_require_tls = $host_address


begin authenticators

ses_login:
  driver = plaintext
  public_name = LOGIN
  client_send = : <aws_id> : <aws_secret>

Saturday, 15 December 2012

OpenVPN on EC2/AWS


EC2 Instance
  1. Allow UDP to port 1194 under the instance's security group
    1. to the world 
    2. or just your IP network
Install and configure OpenVPN instance/server
  1. apt-get install openvpn
  2. cd /etc/openvpn
  3. openvpn --genkey --secret my.key
  4. Put code below into /etc/openvpn/server.conf
  5. Run: openvpn --config /etc/openvpn/server.conf
    1. Leave running while taking remaining steps
  6. Allow NAT
    1. modprobe iptable_nat
    2. echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward
    3. iptables -t nat -A POSTROUTING -s 192.168.2.0/24 -o eth0 -j MASQUERADE
  7. Checks for the paranoid
    1. lsof -nP -i
    2. lsof -c openvpn

port 1194
proto udp
dev tun
secret /etc/openvpn/my.key
ifconfig 192.168.2.1 192.168.2.2
keepalive 10 120
comp-lzo
persist-key
persist-tun
status server-tcp.log
verb 3

  1. If on Mac, use tunnelblick to open a file like myopenvpn.ovpn with below code
    1. Replace hostname with yours
    2. You'll need the my.key file in the same directory as the myopenvpn.ovpn file
  2. On Linux
    1. Put below in /etc/openvpn/client.conf
    2. Put my.key in /etc/openvpn as well
    3. Run: openvpn --config /etc/openvpn/client.conf
    4. Leave running
  3. For all operating systems
    1. When/if you see this in the client connection logs/messages, you've made it
      1. "Initialization Sequence Completed"
  4. Confirm
    1. You can ping 192.168.2.1, the openvpn server/instance via the ssh tunnel
    2. You can ssh into the openvpn server/instance via 192.168.2.1
    3. You have a new network interface with 192.168.2.2 defined on it
    4. Goto: http://ipaddr.me
      1. Does it look like your IP?
      2. If uncertain, use "whois" command to find out
dev tun
proto udp
remote ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com 1194
resolv-retry infinite
nobind
secret my.key
ifconfig 192.168.2.2 192.168.2.1
comp-lzo
verb 3
dhcp-option DNS 172.16.0.23
redirect-gateway def1

Interview questions: 2020-12

Terraform provider vs provisioner Load balancing Network Load Balancer vs Application Load Balancer  Networking Layer 1 vs Layer 4 haproxy u...