NTP scripts

http://www.david-taylor.myby.co.uk/ntp/NTPandMRTG.txt

People have asked how I get the NTP statistics plotted by
MRTG.  Well I got the idea from Piotr Trojanek who had the 
 following code for Unix:

#!/bin/sh

ntpq -c rv \
	|tail -n 1 \
	|sed 's/[a-z=,]//g' \
	|mawk '{printf "%d\n%d\n",
	int(($1 > 0) ? $1 : -$1),
	int(($2 > 0) ? $2 : -$2)}'
 
ntpdc -c sysstats \
	|mawk '{TOTSEC = $3;
		H = int(TOTSEC/3600);
		TOTSEC -= H*3600;
		M = int(TOTSEC/60);
		TOTSEC -= M*60;
		printf "%02d:%02d:%02d\n", H, M, TOTSEC;
		exit 0}'
 echo "ntp_host"

as the command to be run from MRTG, but as I am running 
Windows, I substituted a Perl script for the Unix script.
The first step is to get MRTG to take its data from a 
command line rather than an SNMP query.  You do this by 
 placing the command to run in back-quotes as the argument
to an MRTG "Target" statement.  Then you need to write the
command or script to write the four values MRTG is expecting 
to standard output.
 ---------------------------------------------------------------


Extract from mrtg.cfg
---------------------------------------------------------------

Target[odin_ntp]: `perl GetNTP.pl odin`
MaxBytes[odin_ntp]: 200
 MaxBytes2[odin_ntp]: 200
Unscaled[odin_ntp]: dwmy
Timezone[odin_ntp]: GMT
Title[odin_ntp]: NTP statistics for Odin - offset from NTP
Options[odin_ntp]: integer, gauge, nopercent, growright
YLegend[odin_ntp]: offset+100 ms
 ShortLegend[odin_ntp]: ms
LegendI[odin_ntp]: 
LegendO[odin_ntp]: offset: 
Legend1[odin_ntp]: n/a
Legend2[odin_ntp]: time offset in ms, with 100ms offset added to ensure it's positive!
PageTop[odin_ntp]: <H1>NTP -- PC Odin</H1>
 

Perl script run to manipulate NTPQ output - file: GetNTP.pl
-----------------------------------------------------------

$ntp_str = `ntpq -c rv $ARGV[0]`;
$val = (split(/\,/,$ntp_str))[20];
$val =~ s/offset=//i;
 $val = int ($val + 100);
if ($val < 0) {
  $val = 0;
}
print "0\n";
print "$val\n";
print "0\n";
print "0\n";


-----------------------------------------------------------
 Update 2004-Mar-13:

Thanks to Jim O'Boyle, who mentioned that I could use parameter-passing from 
the mrtg.cfg to avoid hard-coding the node name into GetNTP.pl.  The version 
above includes his update.
 
Thanks, Jim!


-----------------------------------------------------------
Update 2006-Jul-10:

In February, I added a simple stratum 1 server, and added a different version
of the Perl script to cover the more limited range of +/-20 microseconds 
 (displayed as 0..40us).  By July, the GPS was failing more often (tree leaf 
growth?), so I modified the script to limit on both positive and negative
excursions (as without the GPS the server could be hundreds of microseconds
 out).

$ntp_str = `ntpq -c rv $ARGV[0]`;
$val = (split(/\,/,$ntp_str))[20];
$val =~ s/offset=//i;
$val = 1000.0 * $val;	# convert to microseconds
$report = int ($val + 20);
if ($report < 0) {
  $report = 0;
 }
if ($report > 40) {
  $report = 40;
}
print "0\n";
print "$report\n";
print "0\n";
print "0\n";


Thanks, Jim!

Itrade - system written in python

http://itrade.sourceforge.net/snaps.htm

ProfitPy is a set of libraries and tools for the development, testing, and execution of automated trading systems.

Python - Generating Sparkline Graphs For Stock Pricing

http://coreygoldberg.blogspot.com/2008/06/python-generating-sparkline-graphs-for.html

Thursday, June 5, 2008

Python - Generating Sparkline Graphs For Stock Pricing

In a previous post, I introduced the new historical stock pricing function in the ystockquote.py module.

Here is an interesting use of the module to create sparklines of historical stock pricing for a given stock.

To generate the sparklines, I use Grig Gheorghiu's Sparkplot Python module (which uses Matplotlib for graphing).

This module expects a file named data.txt, which contains lines of data to be graphed. All you need to do is generate a data file with the pricing data, and the Sparkplot module will take care of the rest.

To generate the data, I use a script like this:

 import ystockquote

ticker = 'GOOG'
start = '20080101'
end = '20080523'

data = ystockquote.get_historical_prices(ticker, start, end)

closing_prices = [x[4] for x in data][1:]
 closing_prices.reverse()

fh = open('data.txt', 'w')
for closing_price in closing_prices:
    fh.write(closing_price + '\n')
fh.close()

Now that you have the data.txt file setup, you can call the Sparkplot script via the command line to generate the sparkline image:

 python sparkplot.py --label_first --lable_last

*Note: The Sparkplot module needs Matplotlib installed.


The sparkline output is an image (png) like this:

Posted by Corey Goldberg at 9:32 AM

Fully Automated Nagios Linux distribution

FAN (Fully Automated Nagios)

FAN (Fully Automated Nagios) aims to provide a CD based on CentOS in order to simplify installation of Nagios and other Nagios tools. Tools installed by FAN are: Linux, MySQL, Nagios, Nagios Plugins, NaReTo, NagVis, Centreon, Net-SNMP and NDOUtils. Version 0.3 was recently released.

Big databases

Welcome to HBase!

HBase is the Hadoop database. Its an open-source, distributed, column-oriented store modeled after the Google paper, Bigtable: A Distributed Storeage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, HBase provides Bigtable-like capabilities on top of Hadoop.

HBase's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop clusters of commodity hardward. Try it if your plans for a data store run to big.

http://tomictech.com/blog/?p=9

What is sharding?

While working at Auction Watch, Dathan got the idea to solve their scaling problems by creating a database server for a group of users and running those servers on cheap Linux boxes. In this scheme the data for User A is stored on one server and the data for User B is stored on another server. It's a federated model. Groups of 500K users are stored together in what are called shards.

Goosh google shell and Meow - Twitter on Google App. Engine

 goosh.org - the unofficial google shell.

This google-interface behaves similar to a unix-shell.
You type commands and the results are shown on this page.

goosh is written by Stefan Grothkopp it is NOT an official google product!

appengine vs twitter

Eric Moritz

So Glenn Franxman was opining this afternoon about how twitter is going to reach a point where they're going to grow to big and need to make money somehow. He said that they're either going to have to throw ads everywhere or hope to be aquired.

He thinks that they're like every startup in web 2.0 where they're hoping to be aquired by Google.

I said to him, "Why would Google aquire them? I could write twitter on google app engine in like 30 minutes."

Well it took me two hours to shoehorn Django into GAE and about an hour to get a working prototype. It's far from complete, but it works. I still have to integrate textmarks to enable SMS i/o, add following and replies, but it works.

You can view the app at http://meow.appspot.com/ and you can checkout the code at https://code.launchpad.net/~ericmoritz/+junk/meow

C++ website toolkit

Wt (pronounced 'witty') is a C++ library and application server for developing and deploying web applications. It is not a 'framework', which enforces a way of programming, but a library.
A web application developed with Wt is written in only one compiled language (C++), from which the library generates the necessary HTML/XHTML, Javascript, CGI, SVG/VML/Canvas and AJAX code. The responsibility of writing secure and browser-portable web applications is handled by Wt. For example, if available, Wt will maximally use JavaScript and AJAX, but applications developed using Wt will also function correctly when AJAX is not available, or when JavaScript is disabled, reverting to a plain HTML/CGI mechanism for communication between browser and server.

Uses

Ext JS 2.0

Ext JS is a cross-browser JavaScript library for building rich internet applications.

Wt: A Web Toolkit

Writing web applications using a C++ GUI programming style

http://www.webtoolkit.eu/wt/hello.C