All of the interesting technological, artistic or just plain fun subjects I'd investigate if I had an infinite number of lifetimes. In other words, a dumping ground...

Tuesday 23 November 2010

what should a developer know before building a public web site


Thursday 11 November 2010

Python, the colour blue, flash mobs

http://nfggames.com/games/ntsc/visual.shtm
http://ashitvora.info/flash-mob-gone-wrong
http://stackoverflow.com/questions/2573135/python-progression-path-from-apprentice-to-guru

Friday 13 August 2010

Using sqlite3 - C programming example

void select_stack_shuffle(const char * database, const char * blk_lbl, int blk_lbl_len)
{
    sqlite3 *db;
    char *zErrMsg = 0;
    int rc;

    rc = sqlite3_open(database, &db);
    if ( rc ) {
        fprintf(stderr, "sqlite4: Can't open database: %s\n", sqlite3_errmsg(db));
        sqlite3_close(db);
        return;
    }
    const unsigned int sql_str_len = 1024;
    char sql_str[sql_str_len];
    strcpy(sql_str, "SELECT ctr_key, deadline FROM stack_shuffle WHERE block = ?1 AND still_there = 1;");
    sqlite3_busy_timeout(db, 500);
    sqlite3_stmt * pStmt;
    const char * pzTail;
    rc = sqlite3_prepare(db, sql_str, -1, &pStmt, &pzTail);
    // rc = sqlite3_prepare_v2(db, sql_str, -1, &pStmt, &pzTail);
    if ( rc != SQLITE_OK ) {
        fprintf(stderr, "sqlite4: Can't prepare statement: %s\n", sqlite3_errmsg(db));
        sqlite3_close(db);
        return;
    }
    rc = sqlite3_bind_text(pStmt, 1, blk_lbl, blk_lbl_len, SQLITE_STATIC);
    if (rc != SQLITE_OK) {
        fprintf(stderr, "sqlite4: bind failed: %s\n", sqlite3_errmsg(db));
        sqlite3_close(db);
        return;
    }
    do {
        rc = sqlite3_step(pStmt);
        if (rc == SQLITE_ROW) {
            int cols = sqlite3_column_count(pStmt);
            int col_1_type = sqlite3_column_type(pStmt, 0);
            int col_2_type = sqlite3_column_type(pStmt, 1);
            printf("sqlite4: %s: %d columns, type col 1 %d, type col 2 %d\n", __FUNCTION__, cols, col_1_type, col_2_type);
            int ctr_key = sqlite3_column_int(pStmt, 0);
            const unsigned char * deadline = sqlite3_column_text(pStmt, 1);
            printf("sqlite4: %s: ctr_key %d, deadline %s\n", __FUNCTION__, ctr_key, deadline);

        } else {
            if (rc != SQLITE_DONE) {
                printf("sqlite4: %s: received a %d, %s\n", __FUNCTION__, rc, sqlite3_errmsg(db));
            }
        }
    } while (rc == SQLITE_ROW);

}

Monday 19 July 2010

Approximate profiling with gprof.

A work colleague (AB) discovered a little trick with pg profiling that everyone should know about if they don't already,

Profiling is often avoided in live apps because it is expensive and slows down apps that are already running slow (why else profile?)

But there are two things going on when profiling. If you COMPILE with a -pg, every function that gets called stores information about itself. This info is used to determine how many times particular functions are called, and which function gets called by which function. Unfortunately this is expensive for your application, especially for properly written c++ apps as functions are called lots and lots.

When you LINK with a -pg, the program will sample your application 100 times a second and record which function is active at that point in time. This information is used to estimate what proportion of time your app spends in each function. This is often the most sought after information when profiling and is actually very cheap, computationally.

To come to these conclusions I wrote a test application that had a handful of functions that get called a lot, one of them was a recursive function. My results with different profiling methods were as follows:

(1) No profiling:                                         application took 13 seconds to execute.
(2) Linked with -pg:                                  application took 13 seconds to execute.
(3) Compiled and Linked with -pg:     application took 59 seconds to execute.

When linking with -pg the call graph was absent from the profiling results, but the percentage of time spent in each function was retained with accurate results.

Android CyanogenMod soon for HTC Desire

http://www.cyanogenmod.com/

IPv6 internode openwrt

http://ipv6.internode.on.net/access/adsl/
http://superuser.com/questions/104859/native-ipv6-on-openwrt
http://www.andy.id.au/Home/openwrt/custom-builds
http://kamikaze.openwrt.org/docs/openwrt.html

Monday 12 July 2010

Using zlib in C programming

g++ -Wall -Wextra comp.cpp -lz -o comp

#include <iostream>
#include <cstring>
#include <zlib.h>

using namespace std;

/*
ZEXTERN int ZEXPORT compress OF((Bytef *dest,   uLongf *destLen,
                                 const Bytef *source, uLong sourceLen));
     Compresses the source buffer into the destination buffer.  sourceLen is
   the byte length of the source buffer. Upon entry, destLen is the total
   size of the destination buffer, which must be at least the value returned
   by compressBound(sourceLen). Upon exit, destLen is the actual size of the
   compressed buffer.
     This function can be used to compress a whole file at once if the
   input file is mmap'ed.
     compress returns Z_OK if success, Z_MEM_ERROR if there was not
   enough memory, Z_BUF_ERROR if there was not enough room in the output
   buffer.

ZEXTERN int ZEXPORT uncompress OF((Bytef *dest,   uLongf *destLen,
                                   const Bytef *source, uLong sourceLen));
     Decompresses the source buffer into the destination buffer.  sourceLen is
   the byte length of the source buffer. Upon entry, destLen is the total
   size of the destination buffer, which must be large enough to hold the
   entire uncompressed data. (The size of the uncompressed data must have
   been saved previously by the compressor and transmitted to the decompressor
   by some mechanism outside the scope of this compression library.)
   Upon exit, destLen is the actual size of the compressed buffer.
     This function can be used to decompress a whole file at once if the
   input file is mmap'ed.

     uncompress returns Z_OK if success, Z_MEM_ERROR if there was not
   enough memory, Z_BUF_ERROR if there was not enough room in the output
   buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete.

*/

int main()
{
    char d[] =
    "QWERTYUIOPASDFGHJKLZXCVBNMqwertyuiopasdfghjklzxcvbnmQWERTYUIOPASDFGHJKLZXCVBNMqwertyuiopasdfghjklzxcvbnm";
    cout << "sizeof(d): " << sizeof(d) << endl;
    const unsigned int DESTLEN = 256;
    char dest[DESTLEN];
    if (sizeof(d) > DESTLEN) {
        cerr << "Dest size " << DESTLEN << " too small : " << sizeof(d) << endl;
        return 0;
    }
    uLongf destlen = DESTLEN;
    unsigned int compress_bound = compressBound(sizeof(d));
    cout << "Source len: " << sizeof(d) << ", size of compress bound: " << compress_bound << ", size of destlen: " <<
    DESTLEN << endl;
    if (compress_bound > DESTLEN) {
        cerr << "Dest size " << DESTLEN << " too small for compress_bound: " << compress_bound << endl;
        return 0;
    }
    int rtn = compress((Bytef*)dest, &destlen, (Bytef*)d, sizeof(d));
    if (rtn != Z_OK) {
        cerr << "Compress failed" << rtn << endl;
    } else {
        cout << "Compress suceeded: destlen: " << destlen << endl;
        //cout << "Compressed: " << dest << endl;
    }

    char dest2[DESTLEN] = {0};
    uLongf destlen2 = DESTLEN;
    rtn = uncompress((Bytef*)dest2, &destlen2, (Bytef*)dest, destlen);
    if (rtn != Z_OK) {
        cerr << "Decompress failed" << rtn << endl;
    } else {
        cout << "Decompress suceeded: destlen: " << destlen2 << endl;
        cout << "Decompressed: " << dest2 << endl;
    }
    rtn = memcmp(d, dest2, sizeof(d));
    cout << "Comparison: " << rtn << endl;
    return 0;
}

Tuesday 29 June 2010

Setting the core file location in Linux

$ cat /proc/sys/kernel/core_pattern
core
$ ls /var/crash/

root# echo "/var/crash/core.%s.%e.%p" > /proc/sys/kernel/core_pattern

Saturday 29 May 2010

Saving the FBI Radio stream to mp3 on Linux (Ubuntu Lucid)

crontab
0 18  * * mon-fri  /home/tohare/fbi.sh >> /home/tohare/mimms.log 2>&1
30 20  * * mon-fri  /home/tohare/convert_fbi.sh >> /home/tohare/mimms.log 2>&1

save the stream (fbi.sh)
#!/bin/bash
/usr/bin/mimms -v -t 120 mms://wm9.streaming.telstra.com/UCS-wh_fbi0live1  /home/tohare/web-music/fbi-radio-sunsets-`date +%F`.asf > /home/tohare/mimms.log 2>&1


convert to mp3 (convert_fbi.sh)
#!/bin/bash
cd /home/tohare/web-music
/usr/bin/ffmpeg  -i /home/tohare/web-music/fbi-radio-sunsets-`date +%F`.asf /home/tohare/web-music/fbi-radio-sunsets-`date +%F`.mp3 >> /home/tohare/mimms.log 2>&1
/usr/bin/mp3info -t "fbi-radio-sunsets-`date +%F`" -a "FBI Radio" -g "Dance" -y "`date +%Y`" /home/tohare/web-music/fbi-radio-sunsets-`date +%F`.mp3 >> /home/tohare/mimms.log 2>&1
/usr/bin/mp3splt -s /home/tohare/web-music/fbi-radio-sunsets-`date +%F`.mp3 -d fbi-radio-sunsets-`date +%F`  >> /home/tohare/mimms.log 2>&1

Monday 24 May 2010

PyPy/Codespeed - benchmarking code performance

Codespeed is a web application to monitor and analyze the performance of your code.

It is built using Django for the backend, and jQuery and jqPlot for the frontend.

http://wiki.github.com/tobami/codespeed/

http://github.com/tobami/codespeed#readme

Wednesday 19 May 2010

England predicted to win World Cup

http://ftalphaville.ft.com/blog/2010/05/18/233991/england-to-win-world-cup-says-jpm-quant-model/


Ultimately our Model indicates Brazil as being the strongest team taking part in the tournament. However, due to the fixture schedule our Model predicts the following final outcome:

- 3rd: Netherlands

- 2nd: Spain

- World Cup Winners: England

• Alternatively, we point out that the 3 favourite teams (from market prices recorded on 30 April of 3.9-to-1 for Spain, 5-to-1 for Brazil and 5.4-to-1 for England) represent a 52.5% probability of winning the World Cup.


Penetration testing 

http://www.backtrack-linux.org/

Friday 5 February 2010

Linux glibc Heap Consistency Checking

Heap Consistency Checking
Another possibility to check for and guard against bugs in the use of malloc, realloc and free is to set the environment variable MALLOC_CHECK_. When MALLOC_CHECK_ is set, a special (less efficient) implementation is used which is designed to be tolerant against simple errors, such as double calls of free with the same argument, or overruns of a single byte (off-by-one bugs). Not all such errors can be protected against, however, and memory leaks can result. If MALLOC_CHECK_ is set to 0, any detected heap corruption is silently ignored; if set to 1, a diagnostic is printed on stderr; if set to 2, abort is called immediately. This can be useful because otherwise a crash may happen much later, and the true cause for the problem is then very hard to track down.     

Thursday 4 February 2010

Changing shared memory size on Linux

This is useful for Postgresql and Oracle.

Look in to sysctl.
Here is a sample /etc/sysctl.conf file.

[@ ~]# cat /etc/sysctl.conf 
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 1

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1

# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65536

# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 4294967295

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 268435456

Friday 22 January 2010

Open source imageworks

Pystring is a collection of C++ functions which match the interface and behavior of python's string class methods using std::string. Implemented in C++, it does not require or make use of a python interpreter. It provides convenience and familiarity for common string operations not included in the standard C++ library. It's also useful in environments where both C++ and python are used.

http://code.google.com/p/pystring/

Scala Migrations is a library to manage upgrades and rollbacks to database schemas. Migrations allow a source control system to manage together the database schema and the code using the schema. It is designed to allow multiple developers working on a project with a database backend to design schema modifications independently, apply the migrations to their local database for debugging and when complete, check them into a source control system to manage as one manages normal source code. Other developers then check out the new migrations and apply them to their local database. Finally, the migrations are used to migrate the production databases to the latest schema version.

The package is based off Ruby on Rails Migrations and in fact shares the exact same schema_migrations table to manage the list of installed migrations. The Scala Migrations library is written in Scala and makes use of the clean Scala language to write easy to understand migrations, which are also written in Scala. Scala Migrations provides a database abstraction layer that allows migrations to target any supported database vendor.

http://code.google.com/p/scala-migrations/

Tuesday 19 January 2010

ilog license manager

ilog ILM Error 4: Views: cannot connect to token server

[-@- ilm]$ su -
[root@- ~]# cd /usr/ilog/
[root@- /usr/ilog]# cd ilm
[root@- /usr/ilog/ilm]# ./ilmd &
[root@- /usr/ilog/ilm]# Jan 18 22:12:22 UTC 0 ILOG License Manager v2.7.0 (rev. 'r').
Jan 18 22:12:22 UTC 8 Running on linux [pid 19655]
Jan 18 22:12:24 UTC V Using license file "/usr/ilog/ilm/access.ilm"
Jan 18 22:12:26 UTC o TOKEN reservation period = 0 sec
Jan 18 22:12:28 UTC D Licensed to ""
Jan 18 22:12:30 UTC G Tokens for Scheduler on -: 1 linux 
Jan 18 22:12:30 UTC T Tokens for Solver on -: 1 linux 
Jan 18 22:12:30 UTC 7 Tokens for Views: 2 linux , server: - , license: -
Jan 18 22:12:32 UTC x * Warning! Views was used by -, pid 5353, usage=

Wednesday 13 January 2010

OpenGL text display using FTGL
Making the demo work on Ubuntu




tohare@quantum-solace:~/projects/ftgl-demo$ cat Makefile
FTGL_CPPFLAGS := $(shell pkg-config --cflags ftgl)
FTGL_LDFLAGS := $(shell pkg-config --libs-only-L ftgl)
FTGL_LIBS := $(shell pkg-config --libs-only-l ftgl)
OBJDIR := obj

CXX := g++
CXXFLAGS := -MMD -g -Wall -Wextra -Wno-unused-parameter -march=pentium4 -O2 $(FTGL_CPPFLAGS)

#INCLUDE :=
LDFLAGS := $(FTGL_LDFLAGS)
LDLIBS := $(FTGL_LIBS) -lglut

SRCScc := demo.cc
OBJS = $(patsubst %.cc,$(OBJDIR)/%.o,$(SRCScc))

# PHONY is used when the target is not a file
.PHONY: all clean

all: demo

demo: $(OBJS)
$(CXX) $(LDFLAGS) $^ $(LDLIBS) -o $@


$(OBJDIR)/%.o: %.cc
$(CXX) $(CXXFLAGS) $(INCLUDE) -c $< -o $@

-include $(OBJS:.o=.d)



demo.cc





tohare@quantum-solace:~/projects/ftgl-demo$ cat demo.cc
#include // exit()
#include

#include
#include

#include //"FTGLOutlineFont.h"
#include "FTGLPolygonFont.h"
#include "FTGLBitmapFont.h"
#include "FTGLTextureFont.h"
#include "FTGLPixmapFont.h"

static FTFont* fonts[5];
static int width;
static int height;

using namespace std;
void
my_init( const char* font_filename )
{
glClearColor(0.0, 0.0, 0.0, 0.0);
glPolygonMode(GL_FRONT_AND_BACK, GL_FILL);

fonts[0] = new FTGLOutlineFont(font_filename);
fonts[1] = new FTGLPolygonFont(font_filename);
fonts[2] = new FTGLTextureFont(font_filename);
fonts[3] = new FTGLBitmapFont(font_filename);
fonts[4] = new FTGLPixmapFont(font_filename); // looks the best
for (int i=0; i< 5; i++) {
//if (!fonts[i]->Open(font_filename)) {
// cerr << "ERROR: Unable to open file " << font_filename << "\n";
//}
//else {
int point_size = 24;
if (!fonts[i]->FaceSize(point_size)) {
cerr << "ERROR: Unable to set font face size " << point_size << "\n";
}
//}
}
}

static void
do_ortho()
{
int w;
int h;
GLdouble size;
GLdouble aspect;

w = width;
h = height;
aspect = (GLdouble)w / (GLdouble)h;

// Use the whole window.
glViewport(0, 0, w, h);

// We are going to do some 2-D orthographic drawing.
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
size = (GLdouble)((w >= h) ? w : h) / 2.0;
if (w <= h) {
aspect = (GLdouble)h/(GLdouble)w;
glOrtho(-size, size, -size*aspect, size*aspect,
-100000.0, 100000.0);
}
else {
aspect = (GLdouble)w/(GLdouble)h;
glOrtho(-size*aspect, size*aspect, -size, size,
-100000.0, 100000.0);
}

// Make the world and window coordinates coincide so that 1.0 in
// model space equals one pixel in window space.
glScaled(aspect, aspect, 1.0);

// Now determine where to draw things.
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
}


void
my_reshape(int w, int h)
{
width = w;
height = h;
}

void
my_handle_key(unsigned char key, int x, int y)
{
switch (key) {

case 27: // Esc - Quits the program.
for (int i=0; i<5; i++) {
if (fonts[i]) {
delete fonts[i];
fonts[i] = 0;
}
}
exit(1);
break;

default:
break;
}
}

void
draw_scene()
{
/* Set up some strings with the characters to draw. */
unsigned int count = 0;
char string[8][256];
int i;
for (i=1; i < 32; i++) { /* Skip zero - it's the null terminator! */
string[0][count] = i;
count++;
}
string[0][count] = '\0';

count = 0;
for (i=32; i < 64; i++) {
string[1][count] = i;
count++;
}
string[1][count] = '\0';

count = 0;
for (i=64; i < 96; i++) {
string[2][count] = i;
count++;
}
string[2][count] = '\0';

count = 0;
for (i=96; i < 128; i++) {
string[3][count] = i;
count++;
}
string[3][count] = '\0';

count = 0;
for (i=128; i < 160; i++) {
string[4][count] = i;
count++;
}
string[4][count] = '\0';

count = 0;
for (i=160; i < 192; i++) {
string[5][count] = i;
count++;
}
string[5][count] = '\0';

count = 0;
for (i=192; i < 224; i++) {
string[6][count] = i;
count++;
}
string[6][count] = '\0';

count = 0;
for (i=224; i < 256; i++) {
string[7][count] = i;
count++;
}
string[7][count] = '\0';


glColor3f(1.0, 1.0, 1.0);

for (int font = 0; font < 5; font++) {
GLfloat x = -250.0;
GLfloat y;
GLfloat yild = 20.0;
for (int j=0; j<4; j++) {
y = 275.0-font*120.0-j*yild;
if (font >= 3) {
glRasterPos2f(x, y);
fonts[font]->Render(string[j]);
}
else {
if (font == 2) {
glEnable(GL_TEXTURE_2D);
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
}
glPushMatrix(); {
glTranslatef(x, y, 0.0);
fonts[font]->Render(string[j]);
} glPopMatrix();
if (font == 2) {
glDisable(GL_TEXTURE_2D);
glDisable(GL_BLEND);
}
}
}
}
}


void
my_display(void)
{
glClear(GL_COLOR_BUFFER_BIT);

do_ortho();

draw_scene();

glutSwapBuffers();
}

int
main(int argc, char **argv)
{

glutInitWindowSize(600, 600);
glutInit(&argc, argv);
glutInitDisplayMode(GLUT_RGB|GLUT_DOUBLE);

glutCreateWindow("FTGL demo");

int result = 1;
if ( argc != 2 ) {
cerr << "usage: " << argv[0] << " font_filename.ttf\n";
result = 0;
}
else {
my_init(argv[1]);

glutDisplayFunc(my_display);
glutReshapeFunc(my_reshape);
glutKeyboardFunc(my_handle_key);

glutMainLoop();
}
exit(result);
}

Tuesday 5 January 2010

Intel chip naming

http://hardware.slashdot.org/comments.pl?sid=1496592&cid=30639436


Intel also has three lines that more or less directly correspond to AMDs: Core/Phenom (good), Pentium/Athlon (ok) and Celeron/Sempron (cheap), plus the server Xeon/Opteron. The real pain is the amount of different model numbers and numbering schemes. The secret decoder ring for Intel models is:

A) old three number codes
E.g. Pentium 965, Celeron 450, ...
First digit is the model, second digit corresponds to the speed
These are usually old crap and should be avoided. Celeron 743 and Celeron 900 fairly recent low-end chips that you can still buy.

B) Letter plus four numbers codes, e.g. SU7300:
* S = small form factor
* U = ultra-low voltage (5-10W), L = low-voltage (17W), P = medium voltage (25W), T = desktop replacement (35W), E = Desktop (65W), Q = quad-core (65-130W), X = extreme edition
* 7 = model line, tells you about amount of cache, VT capability etc. Scale goes from 1 (crap) to 9 (can't afford).
* 3 = clock frequency, relative performance within the line. Scale from 0 to 9.
* 00 = random features disabled or enabled, have to look up for specific details.

C) New Core i3-XYZa
Similar to scheme B, with the added dash and more confusing
* i3 = Line within Core brand, can be i3 (cheap, but better than Celeron or Pentium), i5 (decent) or i7 (high-end)
* X = the actual model, tells you the amount of cache and number of cores, but only together with the processor line (i3-5xx is very different from i5-5xx)
* Y = corresponds to clock speed, higher is better
* Z = modifier, currently 0, 1 or 5 for specific features
* a = type of processor: X = extreme, M = mobile, QM = quad-core mobile, LM = low-voltage mobile, UM = ultra-low-voltage mobile

tim's shared items

Blog Archive

Add to Google Reader or Homepage