Sysctl Linux server for performance enhancement.

Here are my  tips and suggestions, that I apply to a server for performance enhancement.
I find these values to be safe to run any and everywhere.

sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.core.rmem_default=16777216
sysctl -w net.core.wmem_default=16777216
sysctl -w net.core.optmem_max=40960
sysctl -w net.ipv4.tcp_rmem='4096 87380 16777216'
sysctl -w net.ipv4.tcp_wmem='4096 65536 16777216'
sysctl -w net.ipv4.tcp_window_scaling=1
sysctl -w net.ipv4.tcp_fin_timeout=30
sysctl -w net.ipv4.tcp_keepalive_time=1800
sysctl -w net.ipv4.ip_local_port_range='1024 65000'

For those that use puppet, for your hiera configs:

 sysctl:
  net.core.rmem_max:
    value: "16777216"
  net.core.wmem_max:
    value: "16777216"
  net.core.rmem_default:
    value: "16777216"
  net.core.wmem_default:
    value: "16777216"
  net.core.optmem_max:
    value: "40960"
  net.ipv4.tcp_rmem:
    value: "4096 87380 16777216"
  net.ipv4.tcp_wmem:
    value: "4096 87380 16777216"
  net.ipv4.tcp_window_scaling:
    value: '1'
  net.ipv4.tcp_fin_timeout:
    value: '30'
  net.ipv4.tcp_keepalive_time:
    value: '1800'
 net.ipv4.ip_local_port_range:
     value: '1024 65000'

HTH
Brent

Issues importing with myloader on galera cluster.

I had a very interesting situation and problem whereby,  I was loading data into a Galera cluster using `myloader` (http://www.mydumper.org/).

Our 3 node cluster was correctly replicate writes when we insert
rows from the console. When I loaded our SQL dump, the table
definitions (all InnoDB) got created on all the nodes.

However, the rows from the import only showed up on the single node, I was loading onto, and did not make their way to the other nodes.

Whats was interesting, nothing was showing up in the error logs for the other nodes during the import. I even manually inserted a row into one of the new tables after the big import is done and only that single row showed up on the other nodes.

I eventually found the option ' -enable-binlog', via the man page (default is off). This specifically means that events will not be replicated.

Long story short.

On a galera cluster you need to add the option (--enable-binlog)

time myloader --database=$DATABASE --directory=/RESTORE/PATH --queries-per-transaction=50000 --threads=6  --verbose=3 -o --enable-binlog

HTH
Brent

Ansible 2 On Debian Wheezy.

Today was an interesting day.

I needed ansible ( >= 1.9.2) for a quick research project that resided on Debian Wheezy.

The current version on backports is 1.7.2.

Here is my steps to compiling a Debian package from git.


export DEBFULLNAME="Brent Clark"
export DEBEMAIL="brentgclark@gmail.com"
sudo apt-get install cdbs debhelper dpkg-dev git-core reprepro  python-setuptools devscripts build-essential asciidoc -y
cd /tmp
git clone git://github.com/ansible/ansible.git
cd /tmp/ansible 
git submodule update --init --recursive
make deb

Ansible will too need these packages.

sudo apt-get install python-crypto python-httplib2 python-jinja2 python-markupsafe python-paramiko python-six python-yaml sshpass -y

And then, to install.
dpkg -i /tmp/ansible/deb-build/unstable/ansible_2.0.0-0.git201508210336.9bb95b5.devel~unstable_all.deb

And then to ensure all is working, run:

 ansible localhost -m setup

HTH

Brent

Command line set timezone

In my environment I use vagrant, and I find the timezone that comes with the boxes is not my current location.

Therefore a quick and dirty hack to ensure the timezone reflects your location is:

cp /usr/share/zoneinfo/Africa/Johannesburg /etc/localtime

HTH
Brent
:)

Change IO Scheduler/elevator on a linux server.

Here is a quick how to to change IO Scheduler/elevator on a Linux server and to get some performance enhancement (courtesy from Linbit).

cat /sys/block/sd*/queue/scheduler
echo deadline|tee /sys/block/sd*/queue/scheduler

 echo 0       | tee /sys/block/sd*/queue/iosched/front_merges
 echo 150   | tee /sys/block/sd*/queue/iosched/read_expire
 echo 1500 | tee /sys/block/sd*/queue/iosched/write_expire

More can be read here and here to find out more why I choose deadline.

HTH
Brent

IP lookup

I know there is geoip and you can use its commandline tool geoiplookup to find out some information about an IP.

But if you need a quick lookup, look no further than running

 curl ipinfo.io/74.125.233.37

What you get is:

 bclark@bclark:~$ curl ipinfo.io/74.125.233.37
{
  "ip": "74.125.233.37",
  "hostname": "jnb01s01-in-f5.1e100.net",
  "city": "Mountain View",
  "region": "California",
  "country": "US",
  "loc": "37.4192,-122.0574",
  "org": "AS15169 Google Inc.",
  "postal": "94043"
}

Speed up exim

I received an interesting challenge / request to speed up a clients server that runs the email marketing tool, Active Campaign.

One of the first things I did was move the Exim queue to SSDs.

Thereafter I pushed the following settings, to Exim.

root@mailserver ~ # cat /etc/exim4/conf.d/main/01_exim4-deliver_speedup
# THIS FILE IS AUTOMATICALLY DISTRIBUTED BY PUPPET.  ANY CHANGES WILL BE
# OVERWRITTEN.

deliver_queue_load_max = 20
queue_only_load = 20
split_spool_directory
queue_run_max = 150
remote_max_parallel = 200
smtp_connect_backlog = 50
smtp_accept_max = 500
smtp_accept_queue_per_connection = 2000

On going live, we have identified that the client can deliver 90 000 newletters in under two and a half hours.

HTH
Brent