Blog.Project13.pl » centos http://www.blog.project13.pl The Blog of a Coder Sun, 05 Feb 2012 02:28:52 +0000 en hourly 1 http://wordpress.org/?v=3.2.1 Keeping processes alive using God’s power http://www.blog.project13.pl/index.php/coding/1338/keeping-processes-alive-using-gods-power/ http://www.blog.project13.pl/index.php/coding/1338/keeping-processes-alive-using-gods-power/#comments Thu, 04 Aug 2011 17:48:21 +0000 Ktoso http://www.blog.project13.pl/?p=1338 God is a nice “framework” and daemon to watch Linux processes and restart them if necessary. As I’ve configured it today to work on an ubuntu machine here’s a quick howto:

First install ruby, gem and god.

apt-get install ruby, rubygems
gem install god

Then paste this initscript into /etc/init.d/god: got init script. Configure it to use /etc/god/config.rb for example, and there do some minor setup (using ruby, yay) so we can get emails if something happenes:

  1. require File.dirname(__FILE__) + '/watches.rb'
  2.  
  3. # mailer configuration
  4. # …
  5.  
  6. God::Contacts::Email.delivery_method = :sendmail
  7.  
  8. God.contact(:email) do |c|
  9.   c.name = 'ktoso'
  10.   c.to_email = 'konrad.malawski@llp.pl'
  11. end
  12.  
  13. # include all watches
  14. God.load "/etc/god/watches/*.rb"

And then all that’s left to implement is a watch, so create an /etc/god/watches/xvbf.rb. It’s using a nice DSL to configure how God should act when a process dies etc:

  1. God.watch do |w|
  2.   w.uid = "jenkins"
  3.   w.gid = "jenkins"
  4.   w.name = "xvfb"
  5.   w.interval = 30.seconds # default
  6.   w.env = { "DISPLAY" => ":99" }
  7.   w.start = "Xvfb -ac :99 &"
  8.   w.start_grace = 10.seconds
  9.   w.restart_grace = 10.seconds
  10.   w.behavior(:clean_pid_file)
  11.   w.start_if do |start|
  12.     start.condition(:process_running) do |c|
  13.       c.running = false
  14.       c.notify = ['konrad']
  15.     end
  16.   end
  17. end

The code is pretty much self explanatory. We watch if the process Xvfb is running, and if not, after the grace period we restart it. Optional info email could also be sent using the notify property.

That’s it, have fun automating your build environments!

]]> http://www.blog.project13.pl/index.php/coding/1338/keeping-processes-alive-using-gods-power/feed/ 0 Terminal Heroes 8: Memory usage scripts http://www.blog.project13.pl/index.php/coding/442/memory-usage-scripts/ http://www.blog.project13.pl/index.php/coding/442/memory-usage-scripts/#comments Wed, 24 Feb 2010 10:54:21 +0000 Ktoso http://www.blog.project13.pl/?p=442 I have a low-memory VPS running online and thus always have to worry about what uses how much memory… Of course top and my favorite htop are very good tools to check this, but sometimes I just want to get a super simple report if I’m overusing memory or not yet. The bellow scripts (most of them was found online or hacked up from multiple scripts to fit my needs) print simple yet very readable memory usage information.

memused

#!/bin/bash
  1.  
  2. #if [ $1 = '']; then
  3. #    ps auxf | awk '{sum=sum+$6}; END {print sum/1024}';
  4. #else
  5. #    ps auxf | grep $1 | awk '{sum=sum+$6}; END {print sum/1024}';
  6. #fi
  7. bean=`cat /proc/user_beancounters` guar=`echo "$bean" | grep vmguar | awk '{ print $4;}'` burst=`echo "$bean" | grep privvm | awk '{ print $5;}'` priv=`echo "$bean" | grep privvm | awk '{ print $2;}'`
  8. let total=guar/256 let used=priv/256 let burst=burst/256
  9. echo "VPS memory usage:"
  10. echo "Used: $used MB"
  11. echo "Total: $total MB"
  12. echo "Burstable to: $burst MB"

Sample output:
VPS memory usage:
Used: 296 MB
Total: 256 MB
Burstable to: 512 MB

memfree

#!/bin/bash
  1. #
  2. # Revised 02-Feb-2007: include kernel memory (kmemsize) in 'used' calculation
  3. # and show percentages in output.
  4. #
  5.  
  6. BEAN=`cat /proc/user_beancounters`
  7. GUAR=`echo "$BEAN" | grep vmguar | awk '{ print $4;}'`
  8. PRIV=`echo "$BEAN" | grep privvm | awk '{ print $2;}'`
  9. KMEM=`echo "$BEAN" | grep kmem | awk '{ print $3;}'`
  10.  
  11. let TOTL=$GUAR/256
  12.  
  13. let KMMB=$KMEM/1048576
  14. let PVMB=$PRIV/256
  15. let USED=$KMMB+$PVMB
  16. let FREE=$TOTL-$USED
  17.  
  18. if [ "$FREE" -gt "0" ]; then
  19.     let UPER=$USED*100/$TOTL
  20.     let FPER=100-$UPER
  21. else
  22.     let UPER="100"
  23.     let FPER="0"
  24. fi
  25.  
  26. echo "VPS Memory:"
  27. echo " Total: $TOTL mb Used: $USED mb (${UPER}%) Free: $FREE mb (${FPER}%)"

Sample output:
VPS Memory:
Total: 256 mb Used: 302 mb (100%) Free: -46 mb (0%)

memheld

#!/bin/bash
  1. grep oomguarpages /proc/user_beancounters | awk '{s=$2;t=$3;u=$4; {print "VPS Memory Usage\nCurrent Held: " s/256 "MB\nMax Held: " t/256 "MB\nBarrier: "u/256"MB" }}'

Sample output:
VPS Memory Usage
Current Held: 160.832MB
Max Held: 294.605MB
Barrier: 256MB

memps

#!/bin/bash
  1.  
  2. echo "ps axo pmem,pcpu,comm | sort"
  3. ps axo pmem,pcpu,comm | sort

Sample output:
ps axo pmem,pcpu,comm | sort
0.0 0.0 script
0.0 0.0 script
0.1 0.0 init
#...
2.2 0.0 httpd
4.0 0.0 mysqld
7.3 0.0 httpd
8.0 0.0 httpd
%MEM %CPU COMMAND

memprivmpages

#!/bin/bash
  1.  
  2. beans=`cat /proc/user_beancounters | grep priv`
  3. max=`echo $beans | awk '{ print $4;}'`
  4. use=`echo $beans | awk '{ print $2;}'`
  5. let "per=$use*100/$max"
  6. let "mb=$use/256"
  7. let "mmb=$max/256"
  8.  
  9. echo "privvmpages usage: $mb MB ($per% of $mmb)"

Sample output:
privvmpages usage: 296 MB (57% of 512)
For those wondering, yes I do have memory usage above “100%” in the output of some of these scripts, and it’s partially true. A VPS does usually have some guaranteed memory, and some “burstable” – so you can use more than just the guaranteed memory for some period of time. On the mashine I ran this I have 256 guaranteed and a “total” of 512MB – including the burstable memory.

If you like any of these, feel free to use them. I use them by creating a scripts directory and adding it to the PATH, so I can simply invoke them from anywhere.

PS: For more detailed memory usage information (as graphs etc), I’m using Munin and really recommend it. The graphs it draws are really nice, and it doesn’t inflict heavy load on the server, and it can monitor remote hosts if needed!

]]> http://www.blog.project13.pl/index.php/coding/442/memory-usage-scripts/feed/ 0 MySQLIntegra: dbfiller & wiki http://www.blog.project13.pl/index.php/null/401/mysqlintegra-dbfiller-wiki/ http://www.blog.project13.pl/index.php/null/401/mysqlintegra-dbfiller-wiki/#comments Tue, 09 Feb 2010 23:06:21 +0000 Ktoso http://www.blog.project13.pl/?p=401 I passed all my exams at my first attempt and should have now some time to relax, right? Well something like that, I’m actually doing quite a lot these days. We’re on the roll with MySQL Integra.

Our server “Kopiec” is somewhat online (running CentOS off course) – we’re able to start/stop if thanks to a tool that’s called “remote insight”, and I’ve put 3 databases on there. MariaDB, InfiniDB and a clean MySQL instalation. We’re somewhat having problems with TokuDB, but that’s going to be our 4th database of interest. If you want to keep track of our test’s you can take a look at: http://mysqlintegra.project13.pl/.

The first step in getting things tested is filling those db’s with some data, and I mean a LOT of data. So we hacked up a more or less quick and dirty python script called dbfiller (source code avaiable on github) that would simulate a very simple table generation scheme (people working on shifts and checking in when they’re finished or start some activity). If you’re interested, our task is specyfied in the /doc/Opis.pdf file (sorry, only in polish). After a few days it’s up and running and we’re moving on to filling up those databases with some gigabytes of data.

Next we’re going to write some SQL Views/Procedures and then we’ll use Apache JMeter to run some performance benchmarks of some interesting data storage engines.

PS: If you’re wondering about my GWT-Crossword project – yeah, it’s still actual, and I’ll get right on it when done with some of the stuff here.
PPS: Yes, an Linux powered phone AT LAST in my hands! The HTC HERO is comming to me really soon – never again shall I use a ShitOS like WindowsMobile 6.5.

]]> http://www.blog.project13.pl/index.php/null/401/mysqlintegra-dbfiller-wiki/feed/ 0 Aiming Higher… http://www.blog.project13.pl/index.php/coding/363/aiming-higher/ http://www.blog.project13.pl/index.php/coding/363/aiming-higher/#comments Mon, 21 Dec 2009 21:54:47 +0000 Ktoso http://www.blog.project13.pl/?p=363 …is exactly what I’m currently trying to do. Thus, I’m out of Mediaframe, and got involved in a fascinating and very cool project. Under the lead of Dr Marek Długosz me and 2 of my friends will be tackling the topic of “MySQL Data Storage Engine Comprehension – with focus on Automatics use cases”. It’s really interesting since MySQL is so popular, yet very little people know about the dozens of 3rd party storage engines – some of them are really interesting, just to name a few:

A simple comprehension will be the first thing we’ll do for this project… Later on we’ll do some simulated “real life” tests using JMeter. It’s possible that we’ll write some app based on this database later on – but that’s somehow far off at the moment. That said, I’m really happy to be involved in an interesting and open (it’s all open source & free software after all) topic.

Tomorrow I’m going to the university to setup our very own (awesome ;-)) HP ProLiant DL380 server. It has 6 hard drives, thus it should be very interesting to run HEAVY database LOAD tests – can’t wait. We’ll install CentOS 5.4 and think about it’s future then…

server

]]> http://www.blog.project13.pl/index.php/coding/363/aiming-higher/feed/ 0