Archive for the ‘Uncategorized’ Category

Articles

The Myth of Redundancy

In Uncategorized on 15/12/2015 by weirdfellow

I have just read an interesting article: http://www.smbitjournal.com/2012/05/when-no-redundancy-is-more-reliable/ and I am nothing but shocked and astonished…

Astonished how the maths has became a weird, mysterious artefact of the past and logical thinking is now passé. C’mon SMB IT guys – I expect more from writers of such long essays.

Long story short – author claims that today, with massive discs of terabytes, RAID5 did loose its appeal and does not provide sustainable data protection.

I don’t argue with that. Even without huge experience with RAID5 I can imagine that it might be very hard to successfully resilver such array when one drive dies:

  • not only resilvering slows the array down, but it also stresses it much more. That combined with other factors, such as age of the drives means that the failure of second drive during resilvering is much more probable.
  • there is also the case of URE (Unrecoverable Read Errors) which severity I don’t really understand.
    I accept the fact that current implementations will fail the array when during resilvering when an URE happens, but this is just a limitation of the software. URE during resilvering does not destroy the data – it is still there, in exactly same state as it was before an URE. Why the array does not just drop the resilvering and put the array in “unrecoverable/online” state? Data would still be accessible!

But that’s not what did strike me… The author went further and suggest that RAID5 is so bad, that even RAID0 provides better protection. What a blunt piece of nonsense…

To the calculators!

Let’s do some calculations (Bernoulli trial to be precise)… We’ll be using the 3% annual fail rate of a harddrive in RAID0, as stated in the article. I’ll bump the failure rate to 4% for RAID5 – as the initial build puts some stress…

RAID0

Probability of a 8 disk RAID0 array surviving a year (no hard drive failure) equals 78%. Thus probability of failing within a year is 22%.

RAID5

Probability of a 9 disk RAID5 (to provide as much capacity as 8 disk RAID0) array surviving a year (0 or 1 harddrive failures) can be calculated by calculating the simpler, non overlapping cases and adding the probability:

  • zero HDD failures: 69%
  • one HDD failure: 26%
  • zero or one hdd failure: 69%+26% = 95%
  • probability of RAID5 failure within a year  – 5%

You can verify above figures by using piece of paper (the formulas are in the previously linked Wikipedia page) or an online calculator.

Conclusion

RAID0 is more than 4 times more prone to failure than RAID5 (and that goes up to 7 times should I use the 3% failure rate).

Please, don’t believe in some mumbo-jumbo… Believe in numbers… As flawed RAID5 is today, it still much more reliable than RAID0.

 

Articles

DHCPv6 and RA with dnsmasq

In Uncategorized on 05/09/2014 by weirdfellow

Dnsmasq has been a good companion of small network administrators for years. It proved to be a swiss knife for the IPv4 networks, providing DHCP and DNS services…

The IPv6 is coming, whether you like it or not… And dnsmasq can help here too, by providing both DHCPv6 and RA services.

However, there is a bit of confusion what the options flags in dhcp-range setting mean. Man pages help just a tiny bit…

I’ve recently spend some time spoofing the packages and came up with the following results. The table below shows how the M (Managed Address Configuration Flag), O (Other Configuration Flag) and A (Address Configuration Flag) bits. I’ve skipped the O flag when M is set – in such scenario it can be ignored because it does not carry any meaningful information.

 

Options___________________ M O A______ Comment
no options

+ SLAAC disabled, stateful DHCPv6. This is identical to DHCPv4 service
slaac

+ + SLAAC enabled, stateful DHCPv6. Hosts will have at least two addresses – from DHCPv6 and SLAAC
ra-names
ra-names,slaac

+ + SLAAC enabled, stateful DHCPv6. Based on the information received from host while DHCPv4 request, DNS tries to guess the auto-configured IPv6.
ra-stateless + + SLAAC enabled, stateless DHCPv6. Hosts will get only auto-configured address and get additional configuration from DHCPv6
ra-stateless,ra-names + + SLAAC enabled, stateless DHCPv6. Hosts will get only auto-configured address and get additional configuration from DHCPv6. DNS will try to guess the auto-configured addresses.
ra-only + SLAAC enabled, DHCPv6 disabled. Hosts will get only auto-configured address. DNS might get configured by ND RDNSS.

 

Have also in mind that Windows hosts (by default), in order to protect privacy, do two additional things:

  • don’t follow EUI-64 and use different way of generating addresses when doing autonomous configuration. This means that setting the ra-names option will have no effect – DNS will not guess correctly the IPv6 address of those machines, so no entry in DNS will be populated. This behaviour can be changed and revert to EUI-64 by executing netsh interface ipv6 set privacy state=disabled command.
  • configure additonal IPv6 address – so called temporary address – and prefere it when initializing outgoing connections. Again, this can be switched off by executing netsh interface ipv6 set global randomizeidentifiers=disabled

Some android hosts (exp. my Nexus 7 running 4.4.4) still cannot be configured by stateful DHCPv6. The only supported way is SLAAC. My ZTE phone, running 4.2.1, successfully receives the unicast address when SLAAC is not allowed on the link…

Articles

Linux font substitutions

In Uncategorized on 20/07/2010 by weirdfellow Tagged: , , , , ,

Most of the sites out there expect you to have Windows fonts. The layout is usually designed with those fonts and would break if you don’t have them.

Usually the designer is nice enough to include the sans-serif font at the end of the font definition in CSS, but two sans-serif fonts can be pretty different – Tahoma and Arial Black for instance.

It is also possible to install the msttcorefonts package and use the .fonts.conf from Ubuntu wiki, but I don’t really like this approach – Linux comes with a handful of nice fonts, why not to use them?

The most obvious choice for substitutions is the Liberation family – Liberation Serif for Times New Roman, Liberation Sans for Arial and Liberation Sans Mono for Courier New. But what to do with other fonts? Courier does look different too…

I have been playing with the script below (save it as .fonts.conf in /home/username directory) for a while now. Here is the result:

<?xml version='1.0'?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig>
<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Verdana</string></test>
  <edit binding="same" mode="prepend" name="family"><string>DejaVu Sans</string></edit>
</match> 

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Arial Black</string></test>
  <edit binding="same" mode="prepend" name="family"><string>DejaVu Sans</string></edit>
  <edit mode="assign" name="weight"><double>200</double></edit>
</match>

<match target="font"> <!-- Switch off hinging for DejaVu Sans -->
  <test compare="eq" name="family" qual="any"><string>DejaVu Sans</string></test>
  <edit name="hinting" mode="assign"><bool>false</bool></edit>
</match>

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Tahoma</string></test>
  <edit binding="same" mode="prepend" name="family"><string>Droid Sans</string></edit>
</match> 

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Arial</string></test>
  <edit binding="same" mode="prepend" name="family"><string>Liberation Sans</string></edit>
</match>

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Times New Roman</string></test>
  <edit binding="same" mode="prepend" name="family"><string>Liberation Serif</string></edit>
</match>

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Georgia</string></test>
  <edit binding="same" mode="prepend" name="family"><string>Droid Serif</string></edit>
</match>

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Impact</string></test>
  <edit binding="same" mode="prepend" name="family"><string>Droid Serif</string></edit>
  <edit mode="assign" name="weight"><double>200</double></edit>
</match>

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Courier New</string></test>
  <edit binding="same" mode="prepend" name="family"><string>Nimbus Mono L</string></edit>
</match> 

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Trebuchet MS</string></test>
  <edit binding="same" mode="prepend" name="family"><string>Linux Biolinum O</string></edit>
</match> 

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Lucida Console</string></test>
  <edit binding="same" mode="prepend" name="family"><string>DejaVu Sans Mono</string></edit>
</match> 

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Lucida Sans Unicode</string></test>
  <edit binding="same" mode="prepend" name="family"><string>DejaVu Sans</string></edit>
</match> 

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Palatinio Linotype</string></test>
  <edit binding="same" mode="prepend" name="family"><string>URW Palladio L</string></edit>
</match> 

<match target="pattern">
  <test compare="eq" name="family" qual="any"><string>Consolas</string></test>
  <edit binding="same" mode="prepend" name="family"><string>Inconsolata</string></edit>
</match>
</fontconfig>

Liberation fonts are usually installed, but don’t forget to install Droid font family, Inconsolata and Libertine and Biolinum families.

I have also switched off the hinting for DejaVu Sans. It does not get correctly hinted with Polish letters – I guess not everyone will have this problem – the lines 15-18 could be removed in such case.

In the result I get something like that:

I am really happy with the result. How about you?

Articles

2 matchers expected, 1 recorded.

In Uncategorized on 15/07/2010 by weirdfellow Tagged: , , , ,

EasyMock is really a nice framework for mocking interfaces, so you can reliably test your classes.

As always – everything is fine, unless you need to do something more complex. As long as you know all parameters passed to the mocks, there are no problems. But sometimes it is not possible to predict all arguments the mock will receive. A bit of research and here it is:

  MyData myData = EasyMock.createNiceMock( MyData.class );
  MyService serviceMock = EasyMock.createMock( MyService.class );

  expect( serviceMock.doSth( EasyMock.<File>anyObject(), myData ).andReturn( true );

Seems OK – doesn’t it? The code tells serviceMock to expects a call to doSth method. The second argument should be myData object, while we don’t care about the first one.

Unfortunately, you will get something like that:

java.lang.IllegalStateException: 2 matchers expected, 1 recorded.
	at org.easymock.internal.ExpectedInvocation.createMissingMatchers(ExpectedInvocation.java:56)
	at org.easymock.internal.ExpectedInvocation.(ExpectedInvocation.java:48)
	at org.easymock.internal.ExpectedInvocation.(ExpectedInvocation.java:40)
	at org.easymock.internal.RecordState.invoke(RecordState.java:76)
	at org.easymock.internal.MockInvocationHandler.invoke(MockInvocationHandler.java:38)
	at org.easymock.internal.ObjectMethodsFilter.invoke(ObjectMethodsFilter.java:72)
	at org.easymock.classextension.internal.ClassProxyFactory$MockMethodInterceptor.intercept(ClassProxyFactory.java:93)

Unfortunately, it is not possible to mix values and matchers – a anyObject method is a matcher. To make the test execute correctly, all parameters should use matchers instead of just values. Solution is pretty straight forward – use eq matcher:

  expect( serviceMock.doSth( EasyMock.<File>anyObject(), EasyMock.eq( myData ) ).andReturn( true );

In addition to eq matcher, EasyMock does publish a handful of other matchers. Worth noting are aryEq for comparing the array contents and isA for checking the class of given object. The EasyMock site gives a full list of those.

Articles

Ubuntu Lucid and Compcache

In Uncategorized on 15/07/2010 by weirdfellow Tagged: , , ,

I have been very excited about memory compression for years. The performance boost achieved with Quarterdeck MagnaRAM on Windows 98 still brings smile on my face. Without memory compression my 64MB system sounded like it was chopping parsley – definitely there was some swapping. Switching MagnaRAM on kept it quiet (to be precise – it was loading the next level of Colin McRae Rally 2). Not to mention the difference in required time. Unfortunately, I couldn’t find similar tool for any Windows NT system.

Recently I started to build a development machine on Ubuntu Lucid Lynx and decided to give the Compcache a chance. Compilation caused me absolutely no problems – just download the package, unpack it and execute make. Although there are some patches for the kernel, so it can use the memory even more efficient, they are not required and Compcache will work with stock kernel. Compilation generates two things: a kernel module ramzswap.ko and a user-space utility rzscontrol.

It is pretty simple to initialize compressed swap disk by hand, but I wanted a slick and elegant integration with the services management subsystem. A couple of experiments and voila: my Upstart script for Compcache looks like that:

description "Initializes Compcache"

start on runlevel [2345]
stop on runlevel [!2345]

pre-start script
DIR=/usr/local/src/compcache-0.6.2
CONTROL=$DIR/sub-projects/rzscontrol/rzscontrol

# load dependency modules
modprobe lzo_compress
modprobe lzo_decompress

# load ramzswap module
insmod $DIR/ramzswap.ko num_devices=2
sleep 1s

# Initialize devices with 512MB upper limit.
# These devices does not have a backing swap.
$CONTROL /dev/ramzswap0 --disksize_kb=524288 --init
$CONTROL /dev/ramzswap1 --disksize_kb=524288 --init

# Switch the swaps on
swapon -p 5 /dev/ramzswap0
swapon -p 5 /dev/ramzswap1
end script

post-stop script
DIR=/usr/local/src/compcache-0.6.2
CONTROL=$DIR/sub-projects/rzscontrol/rzscontrol

swapoff /dev/ramzswap0
swapoff /dev/ramzswap1

$CONTROL /dev/ramzswap0 --reset
$CONTROL /dev/ramzswap1 --reset

rmmod ramzswap
end script

Update the paths, save the file as /etc/init/compcache.conf and restart your system. Your system should have two additional swap drives – 512MB each – already configured and switched on. Such configuration also enables switching Compcache off by using Upstart commands too.

In my environment Compcache achieved ~65% compression – 1GB of swapped pages would take approximately 350MB of memory. In result system can allocate 1.7GB and use only 1GB of physical memory without any sign of HDD swapping!

My setup uses two separate swapping devices in order to keep both of the CPU cores busy. If running on Quad Core CPU, it is beneficial to create four devices. Kernel utilizes them equally.

Some useful commands to track your swapping:

sudo swapon -s #displays active swap devices and their usage
sudo stop compcache #switches Compcache off.
sudo start compcache #switches Compcache on
sudo rzscontrol /dev/ramzswap0 --stats #displays the stats of the first swapping device.

I am running this environment for couple of days and I see only advantages. Give it a try!

Articles

Waiting for sound system to respond

In Uncategorized on 30/06/2010 by weirdfellow Tagged: , , ,

This error message is around with many Ubuntu enthusiasts since 9.10. User cannot change the volume level, CPU usage gets ridiculous and the fan sounds like a AH-64 Apache…

The solution is actually very simple

  1. Remove directory .pulse from your home folder
  2. Perform the logout – login cycle

Voila…

Articles

GDIpp

In Uncategorized on 29/06/2010 by weirdfellow Tagged: , , , ,

Last time I showed that there isn’t a thing not worth complaining. Fonts and their imperfection in Windows definitely are.

Apparently, I am not the only one… This issue must be especially annoying for people from Far East. Japanese or Chinese alphabets are very complex and the correct shape of glyphs is important. And some guys only bitched about ClearType, but actually decided to do something about it.

That’s how GDI++ started – a first attempt to replace the windows font rendering engine. The idea is simple – replace font related functions from windows gdi32 library with custom ones. Library injection, hooks and other weird things are involved, but in the end it works.

Unfortunately, the project isn’t developed anymore. But not only it showed that the problem might be addressed, but also how to do it. GDIpp started with the ideas derived from its predecessor and although still in version zero, is impressive.

Try it out – there are versions for x64 and x86 versions of Windows. Although minor problems with spacing between characters can be observed from time to time (the font metrics API is not replaced yet), but the overall impression is impressive! Finally nice glyphs in Windows!