2013-02-22

Bitflips

Jenkins autobuild of a Qt program failed with a strange error:
/opt/qt/4.8/include/QtGui/qpalette.h:138:18: error: unknown type name 'QBzush'; did you mean 'QBrush'?
    inline const QBzush &buttonText() const { return brush(ButtonText); }
                 ^~~~~~
                 QBrush
/opt/qt/4.8/include/QtGui/qbrush.h:76:20: note: 'QBrush' declared here
class Q_GUI_EXPORT QBrush
                   ^
The error says that the type QBzush was not found, but the name resembles QBrush. The "typo" seemed to be in Qt's header file qpalette.h, but the file was fine and the next build of the exactly same source succeeded.
There is only 1 bit difference between 'z' and 'r'. We must've experienced a bitflip. According to wikipedia these are surprisingly common on commodity hardware.
r = 114 = 1110010
z = 122 = 1111010
ECC memory is a friend. Have to figure out how to get Mac builds on a computer with ECC RAM.

2012-12-02

Adding disk to Proxmox KVM virtual machine without rebooting


We use Proxmox to run KVM virtual machines. Sometimes VMs end up requiring more disk space than originally planned. You can add a new disk from Proxmox but VM has to be rebooted. Proxmox doesn't do disk hotplug but we can plug the disk in from the qm monitor.

I have a KVM virtual machine with VM ID 178. It is Ubuntu Server installed on LVM to allow extending partitions live. For device hotplugging to work, the VM must have acpiphp module installed. You can modprobe it in, and put it in /etc/modules.

Add the disk from Proxmox web interface and find out where the disk image file or underlying LVM volume is. In this case Proxmox created vm-178-disk-3 on local-lvm storage.

Log on to Proxmox platform with SSH and open qm monitor. Command info block shows the current disks.


kvm7:~# qm monitor 178
qm> info block
ide0-cd0: removable=1 locked=0 [not inserted]
virtio1: removable=0 file=/dev/pve/vm-178-disk-2 ro=0 drv=raw encrypted=0
virtio2: removable=0 file=/dev/pve/vm-178-disk-1 ro=0 drv=raw encrypted=0
virtio0: removable=0 file=/dev/pve/vm-178-disk-3 ro=0 drv=raw encrypted=0
qm>

Hotplug the new disk:

qm> pci_add auto storage file=/dev/pve/vm-178-disk-3,if=virtio

If the disk image is a file:


qm> pci_add auto storage file=/var/vz/images/178/vm-178-disk-3.raw,if=virtio

The disk should show up in VM's dmesg. In this case /dev/vdc appeared. Add the disk to LVM and extend the filesystem.

# pvcreate /dev/vdc
# vgextend vg0 /dev/vdc
# lvextend -L +16G /dev/vg0/root
# resize2fs /dev/vg0/root

2009-08-21

Removing data from rrdtool files

We benchmarked our new SQL-server and caused a nasty peak in Munin graphs. The peaks really made the graps quite useless as the new information was flattened to the very bottom of the graph.

Nulling days of data was quite easy as rrdtool has dump and restore commands.

root@foo:/var/lib/munin/example.tld# for i in foo.example.tld-* ; do
rrdtool dump $i > /tmp/foo.xml &&\
awk -f /tmp/fix.awk /tmp/foo.xml > /tmp/bar.xml &&\
rrdtool restore -f /tmp/bar.xml $i
done

The /tmp/fix.awk awk script changes sample values to NaN which rrdtool regards as empty samples.

$2=="2009-08-20" || $2=="2009-08-21" {
$9="NaN";
}

{
print $0;
}

2009-05-20

PostgreSQL transaction stats to MRTG

The following perl fragment does "SELECT sum(xact_commit) FROM pg_stat_database;" and returns four lines suitable for use with mrtg (data1, data2, uptime, hostname):


0
12765

db1.example.tld


We don't have access to uptime, so we ignore it. The transaction count is in the second data field for purely aesthetic reasons.

Try that the script works before configuring mrtg:


#!/usr/bin/perl -w
use DBI;
use strict;

my $dbhost='db1.example.tld';
my $dbname='template1';
my $dbuser='monitor';
my $dbpass='secretword';

my $connstring = "DBI:Pg:dbname=$dbname;host=$dbhost";
my $connection = DBI->connect($connstring, $dbuser, $dbpass, {RaiseError => 0, PrintError => 0}) || die "Unable to access database: ". $DBI::errstr ."";

my $query = "select sum(xact_commit) + sum(xact_rollback) from pg_stat_database;";
my $stmt = $connection->prepare($query);
$stmt->execute() || die "Running query failed: ". $DBI::errstr ."";
my $row = $stmt->fetchrow();

print "0\n";
print $row . "\n";
print "\n";
print $dbhost . "\n";


I use the following MRTG fragment:

Target[sql.xacts]: `/etc/mrtg-sql-tps.pl`
MaxBytes[sql.xacts]: 1250000
Title[sql.xacts]: PostgreSQL transactions per second
PageTop[sql.xacts]: PostgreSQL transactions per second
YLegend[sql.xacts]: Xacts / second
ShortLegend[sql.xacts]: xact/s
LegendO[sql.xacts]: Transactions per second
Options[sql.xacts]: integer,nopercent,noinfo,nobanner,noi


noi option causes mrtg to ignore the first data field (Input) and LegendO sets description for the second data field we use (Output for MRTG).

Remeber to make a new index page with indexmaker.

2009-03-10

Silicon Labs CP2101 based USB to UART adapter on Mac OS X

I bought some embedded controllers and a USB to TTL-level UART serial converter from Omnima web store. After some googling with the the USB vendor and product id I figured out that the converter was based on Silicon Labs CP2101 chip. I installed Mac OS X drivers from https://www.silabs.com/products/interface/usbtouart/Pages/default.aspx but no luck. The device wouldn't be recognised.

System profiler tells me the product id is 0x10c5 and vendor id 0x10ab (product 4293, vendor 4267 in decimal) but there was no entry for that combination in driver's plist - /System/Library/Extensions/SiLabsUSBDriver.kext/Contents/Info.plist.

The trick was to copypaste and modify one of the sections used to register the driver for product+vendor combinations. Just use the decimal form of product and vendor codes. Rebuild of driver cache had to be triggered by touching /System/Library/Extensions.

http://ciz.fi/~jkj/silab-mac-usb-driver.patch

After patching info.plist and rebuilding driver cache, plugging the device generated a device node in /dev/cu.SLAB_USBtoUART

2007-11-27

Using pkgsrc in Mac OS X

Mac OS X has the default of case insensitive HFS+ file system and it seems to me that although case sensitive HFSX+ is supported, the OS is not heavily tested on it. E.g. FileVault won't work with it at all. I opted to case sensitive disk images.

Create and populate /usr/pkgsrc

# mkdir /Library/Images
# hdiutil create -volname pkgsrc -size 10g -type SPARSEBUNDLE -fs HFSX /Library/Images/pkgsrc
# mkdir /usr/pkgsrc
# hdiutil attach /Library/Images/pkgsrc.sparsebundle -nobrowse -mountpoint /usr/pkgsrc -owners on
# cd /usr
# cvs -z3 -d anoncvs@anoncvs.NetBSD.org:/cvsroot co -P pkgsrc
# cd /usr/pkgsrc
# cvs -d:pserver:anonymous@pkgsrc-wip.cvs.sourceforge.net:/cvsroot/pkgsrc-wip login
CVS password:
# cvs -z3 -d:pserver:anonymous@pkgsrc-wip.cvs.sourceforge.net:/cvsroot/pkgsrc-wip co -P wip


Create /usr/pkg

# hdiutil create -volname pkg -size 10g -type SPARSEBUNDLE -fs HFSX /Library/Images/pkg
# mkdir /usr/pkg
# hdiutil attach /Library/Images/pkg.sparsebundle -nobrowse -mountpoint /usr/pkg -owners on


Jvvm figured out how to mount the images at boot time. http://julipedia.blogspot.com/2007/04/mounting-volumes-on-mac-os-xs-startup.html. I made a few changes. There is a configuration file for mounting images:

/etc/hditab:
# HD Image Tab configuration file
# Imagefile options (for hdiutil attach)
/Library/Images/pkgsrc.sparsebundle -nobrowse -mountpoint /usr/pkgsrc -owners on
/Library/Images/pkg.sparsebundle -nobrowse -mountpoint /usr/pkg -owners on

Using jvvm's example I made a startup item directory /Library/StartupItems/DiskImages/ with the following files in it. The latter needs to be executable.

/Library/StartupItems/DiskImages/StartupParameters.plist:
{
Description = "Automatic attachment of disk images";
OrderPreference = "First";
Uses = ("Disks");
}


/Library/StartupItems/DiskImages/DiskImages:
#! /bin/sh
. /etc/rc.common

basePath="/Library/StartupItems/DiskImages"
tabFile="/etc/hditab"

doCompact() {
cat $tabFile | while read image rest ; do
image=${image%%#*}
[ -z "$image" ] && continue
hdiutil compact $image
done
}

StartService() {
doCompact
cat $tabFile | while read line ; do
line=${line%%#*}
[ -z "$line" ] && continue
hdiutil attach $line
done
}


StopService() {
true
}

RestartService() {
true
}

RunService "$1"

The images should be automatically mounted at boot. Test the script by umounting /usr/pkgsrc and /usr/pkg manually and running:
# /Library/StartupItems/DiskImages/DiskImages start

Next pkgsrc needs to be bootstrapped

# cd /usr/pkgsrc/bootstrap
# ./bootstrap --prefix /usr/pkg --pkgdbdir /usr/pkg/pkgdb --sysconfdir /usr/pkg/etc --varbase /usr/pkg/var

To add pkgsrc in your path create file /etc/path.d/pkgsrc with lines /usr/pkg/bin and /usr/pkg/sbin. Manual page search path is set in file /etc/manpath.d/pkgsrc with line /usr/pkg/man. There is a default mk.conf in /usr/pkg/etc/mk.conf. I made the following changes to mine.

/usr/pkg/etc/mk.conf:
COPTS?= -pipe ${DEFCOPTS}
CFLAGS+= -pipe

.ifdef BSD_PKG_MK # begin pkgsrc settings

PKG_DBDIR= /usr/pkg/pkgdb
LOCALBASE= /usr/pkg
VARBASE= /usr/pkg/var
PKG_TOOLS_BIN= /usr/pkg/sbin
PKGMANDIR= man

TOOLS_PLATFORM.pax?= /usr/pkg/bin/pax
TOOLS_PLATFORM.tar?= /usr/pkg/bin/tar

#MAKE_JOBS ?= 2
WRKOBJDIR ?= /usr/pkg/var/obj
MASTER_SORT ?= .fi .se .no .ee

PKG_DEFAULT_OPTIONS += inet6

_ACCEPTABLES=yes

.endif # end pkgsrc settings


I used sparse images here, because I'm short on disk space but there are cons; Sparse images cause disk fragmentation and removing files from the image doesn't free the space until the image is compacted. If you really want to tune it up make ~1GB DMG for /usr/pkgsrc, big sparce image for /usr/pkg and a ~2GB /usr/pkg/var DMG for changing data. The set WRKOBJDIR?=/usr/pkg/var/obj and DISTDIR?=/usr/pkg/var/distfiles in mk.conf.

Comments and hints are very welcome :)