Flush DNS cache on OS X 10.7 and 10.8

As a reminder to myself, here’s how to flush DNS cache on OS X. I sometimes need to do this, ie. when I make changes to /etc/hosts.

# sudo killall -HUP mDNSResponder
Posted in Mac | Tagged , , , | Leave a comment

OS X Lion driver for Keyspan serial adapters

For those of you who have one of Keyspan’s awesome serial port adapters, and who use it on Mac, there’s now a beta version of their driver for Mac OS X Lion (10.7)

Go grab it -> here. For other drivers, or to come back and check for a newer version, go here

Posted in Mac | Tagged , , | Leave a comment

pv – monitor the progress of data through a pipe

As the title says, this tool will let you monitor pipe throughput, something which I find very valuable at times, ie. how fast was my tar or cpio to another host or filesystem, or less useful like how fast does “yes | pv >/dev/null” swallow my data.

pv – Pipe Viewer – is a terminal-based tool for monitoring the progress of data through a pipeline. It can be inserted into any normal pipeline between two processes to give a visual indication of how quickly data is passing through, how long it has taken, how near to completion it is, and an estimate of how long it will be until completion.

Either get it from the official site and compile it yourself, or from your OS vendors package repository. Debian/Ubuntu users can do “apt-get install pv”.

Posted in UNIX | Tagged , , | Leave a comment

Separating a running job from the shell

There’s a little-known, but oh so useful, bash builtin called disown – We’ve all started a job that ends up taking way longer than anticipated, now you really wish you had backgrounded it to begin with, or had used Screen.

From the manpage:

disown [-ar] [-h] [jobspec ...]

Without options, each jobspec is removed from the table of active jobs. If the -h option is given, each jobspec is not removed from the table, but is marked so that SIGHUP is not sent to the job if the shell receives a SIGHUP. If no jobspec is present, and neither the -a nor the -r option is supplied, the current job is used. If no jobspec is supplied, the -a option means to remove or mark all jobs; the -r option without a jobspec argument restricts operation to running jobs. The return value is 0 unless a jobspec does not specify a valid job.

It’s important to understand that disown does’t work on pids or processes, but jobs.
Here’s an example

$ rm -rf millionsoffiles
CTRL-Z
$ bg
[1]+ rm -rf millionsoffiles &
$ jobs
[1]+  Running                 rm -rf millionsoffiles &
$ disown %1

Keep in mind, you need to be using the Bash shell for this to work.

Posted in UNIX | Tagged , , , | Leave a comment

Ubuntu 10.04 PV DomU on XenServer 5.6

Admittedly, I was a bit disappointed to see XenServer 5.6 without support for Ubuntu, Debian 64-bit and Solaris, I wish they’d at least provide working templates as an unsupported download or something, but they don’t. So, if you want to virtualize your Ubuntu installations you have to get your fingers dirty, fortunately it’s very easy with XenServer 5.6 (in 5.5 it’s a bit more work).

As of Ubuntu 8.10 all the Xen stuff has been integrated in the default server kernel, no need to fiddle with different kernels anymore.

  • Create a VM in XenServer, use Other Install media template, customize your installation to your liking but don’t fiddle with the /boot filesystem, pygrub needs this to be ext2 or ext3.
  • Once your new Ubuntu VM is up and running you might as well apply the latest updates, if you do this later and there’s a new kernel AND you’ve specified any PV-bootloader-args you’ll have to go back into XenServer and fix this. At this point I also recommend installing an ssh server, it makes life a lot easier should your new Xen-based console not work.,
    sudo apt-get install ssh
  • Configure virtual console, replace tty1 with hvc0
    cp /etc/init/tty1.conf /etc/init/hvc0.conf
    sed -i 's/tty1/hvc0/g' /etc/init/hvc0.conf
  • Shutdown virtual machine
    shutdown -hP now
  • Find your virtual machine and it’s UUID
    xe vm-list name-label=<your VM name> params=uuid
  • Switch from HVM to PV
    xe vm-param-set uuid=<UUID> HVM-boot-policy=
    xe vm-param-set uuid=<UUID> PV-bootloader=pygrub
  • Find the VBD disk UUID
    xe vm-disk-list uuid=<UUID>
  • Make VBD disk bootable
    xe vbd-param-set uuid=<VBD UUID> bootable=true

Now boot up the VM, if the console appears to hang and you’re sure you’ve setup hvc0 correct, try restarting XenCenter, for some reason it doesn’t always like the change at first.

Installing XenServer Tools is dead easy. Since my XenServers are on AMD, I’ll use the amd64 package.

mount /dev/xvdd /cdrom
dpkg -i /cdrom/Linux/xe-guest-utilities_5.6.0-578_amd64.deb
umount /cdrom

On one occasion I forgot to ‘eject’ the cdrom, the VM then failed to boot saying too many bootable devices, puzzling at first but it was nothing more than ejecting the cdrom and booting again.

I’ve run Ubuntu 10.04 PV in production for a few months now and have had no issues whatsoever.

Posted in Linux, Uncategorized, Virtualization, XenServer | Tagged , , , , | 5 Comments

Wiping Transcend V15 USB stick

Having purchased a Transcend V15 16GB USB stick for a router-on-a-stick project, I was horrified to find out that it came preinstalled with some proprietary Antivirus software which only works under Windows, to make matters worse, the USB stick reports itself as read-only when plugged in under OS X or Linux, you can’t wipe it or even use it.

The proper unofficial and unsupported method (you’ll need Windows) is to download Transcend Online Recovery Tool, disable UAC by going to the User Control Panel (Start Menu -> Control Panel -> User Accounts and Family Safety -> User Account. Set the controller to “Never notify” and reboot the machine. (you can and should change it back later)

Now insert USB stick, start tool and follow the instructions, but once it prompts you for the size of your USB stick you quit the tool. The USB has now been wiped and it’s no longer read-only.

The stick still contains two partitions, but that’s easily fixed under OS X or Linux.

I’d love to hear from people who have successfully wiped the entire USB key without using Windows, and I don’t mean stomping on it ;)

Posted in Hardware | Tagged , , , | 1 Comment

Vyatta 6.1 now available

The good people at Vyatta have announced the immediate availability of Vyatta version 6.1 (VC6.1)

Vyatta Version 6.1 Highlights include:

IPv6 Ready
Vyatta Version 6.1 adds IPv6 BGP, IPv6 DCHP and IPv6 SNMP and this release completed IPv6 Forum IPv6 Ready Phase 2 certification

Cloud Bridging
For cloud providers and enterprises moving applications or servers to the cloud, Layer 2 cloud bridging allows physically separate networks to securely communicate with each other over the internet as if they were on a single Ethernet network.

Enhanced Security – Stateful Firewall Failover / Sourcefire VRT IPS Rules
Vyatta also continued its focused delivery on enterprise security advancements adding stateful firewall failover and enhanced intrusion prevention services through a partnership with Sourcefire.

Vyatta Version 6.1 enhancements include:

Vyatta Core 6.1

  • Layer 2 Bridging / Cloud Bridging
  • Stateful Firewall/NAT Failover
  • LLDP – Link Layer Discovery Protocol
  • QoS Input Interfaces
  • Port Mirroring and Redirection
  • BGP Hop Count Security
  • DHCPv6
  • IPv6 BGP
  • IPv6 SNMP
  • GTSM BGP
  • More than 120 Bug Fixes

Vyatta Subscription Edition

  • IPv6 Forum – IPv6 Ready Phase 2 Certification
  • Configuration Replication
  • Remote Access API Enhancements
  • TACACS+ Enhancements

Vyatta Plus

  • Vyatta Sourcefire VRT IPS Rules: Vyatta now offers the official Sourcefire/Snort VRT rule-base as a subscription service. This new add-on service allows customers to proactively protect systems by accessing updates to the Sourcefire VRT rule-base directly from Vyatta as new vulnerabilities are found.

See the full Release Notes and Reference Guides on the documentation page for more detail on each feature.

Download Vyatta Core Version 6.1 Software: http://www.vyatta.org/downloads

Posted in Networking, Vyatta | Tagged | Leave a comment

Installing XenServer Linux Guest Support

It’s happend a number of times that I’ve had to install Citrix XenServer and have not had the Linux Guest Support CD with me, so I’ve later had to install it remotely, which is actually pretty easy.

  • Transfer ISO file to XenServer using scp/winscp or similar tool
  • Mount ISO
  • Run installer

That’s it.

[root@bxs-phys01 ~]# mkdir /mnt/iso
[root@bxs-phys01 ~]# mount -o loop /root/XenServer-5.6.0-linux-cd.iso /mnt/iso
[root@bxs-phys01 ~]# /mnt/iso/install.sh
Warning: If you are in a pool, you must install this pack on all servers.
Starting installation of Linux Pack...

Verifying templates-debian...
Installing templates-debian...

Restarting XAPI...
Stopping xapi: ..                                          [  OK  ]
Starting xapi: .....start-of-day complete.                 [  OK  ]
Linux pack installation complete.
[root@bxs-phys01 ~]# umount /mnt/iso
[root@bxs-phys01 ~]# rm /root/XenServer-5.6.0-linux-cd.iso

Virtualize and be merry.

Posted in XenServer | Tagged , , | 3 Comments

Parsing Ahsay OBS logfile timestamps

All Ahsay OBS generated logfile timestamps use milliseconds since Unix epoch on January 1, 1970, 00:00:00 GMT, it’s great for scripting and programming as it’s easy to do date math, but if you want to output a humanly readable date you need to convert it.

The below perl script will work for all Ahsay OBS logfiles, pretty much.

#!/usr/bin/perl
#
# Usage:
#  cat /usr/local/obs/system/SystemLog/2009-10-19.log | perl obslog.pl
#
# Converts Java timestamp to humanly readable
#

use Time::localtime;

@wday=('Sun','Mon','Tue','Wed','Thu','Fri','Sat');
@month=('Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec');

sub epochToDate {
   my ($epoch) = @_;

   $tm = localtime($epoch/1000);
   return(sprintf("[%02d/%s/%4d %02d:%02d:%02d]", $tm->mday, $month[$tm->mon], $tm->year+1900,
                                                  $tm->hour, $tm->min, $tm->sec));
}

while(<STDIN>)
{
    ($date, $rest) = /^(\d+)\,(.*)/;
    printf("%s%s\n", epochToDate($date), $rest);
}

Not exactly rocket science but hopefully someone will find it useful and it should work on Windows, Linux, Solaris and pretty much any system Ahsay OBS will run on.

Posted in Ahsay | Tagged , , , , | Leave a comment

Building a 52TB SAS-2 ZFS array

Storage can be very expensive, especially if you want a lot of it, want flexibility and expandability, but it doesn’t have to be!

I’m currently in the process of putting together a new storage system, but because we’re strapped for cash I’m limited to opensource and nearly off the shelf products.
The operating system will be OpenSolaris, the exact kernel is undecided as of yet, hopefully Oracle will come out with an update real soon. Filesystem will be ZFS, although it has issues with tens of millions of files it still offers me flexibility and manageability features which are very useful.

The parts are

Storage head

  • Dell R610 w. 2 x E5620, 24GB memory and internal PERC H200 contr.
  • LSI-9200-8e PCIe
  • 1 x OCZ Vertex 2 Pro 100GB (L2ARC)
  • *Perhaps 10GbE PCIe card


SAS2 JBOD

  • SuperMicro SC847E26-RJBOD1
  • 26 x 2TB SAS2 Seagate Constellation ES 3.5″ 7200rpm
  • 2 x Intel X25-E 32GB (mirrored SLOG)
  • SAS 2-port internal cascading cable
  • 2 x LSISS9252 SAS-to-SATA interposer

Total capacity is currently 52TB if I was to stripe it all, but that wouldn’t give me any fault tolerance, so the 26 disks will be divided into 3 RAIDZ2 vdevs each consisting of 8 disks, the two leftover disks will become hotspares. This gives me roughly 36TB of useable space with room to grow (17 free out of 45 total). While it’s no S7310 system it’s not that far off either, the L2ARC SSD will sit on the storage head as a pass-through disk on the internal controller, and the SLOG SSD’s will sit in the SC JBOD array using LSISS9252 interposer cards.

I’ll post an update on the build process, including photos, once the hardware arrives.

Other posts with similar builds,
http://www.natecarlson.com/2010/05/07/review-supermicros-sc847a-4u-chassis-with-36-drive-bays/
http://hardforum.com/showthread.php?t=1508468

Posted in OpenSolaris, Storage | Tagged , , , , , , , , , , , , | 6 Comments