The Intersect

I have always liked bots, I have written a few MediaWiki bots in other languages and the first non-trivial Perl 6 I wrote was my own Perl 6 MediaWiki bot framework (still not ready for any live-wiki editing yet). There just seems to be something about writing a program that imitates human behaviour (such as editing a wiki, or chatting on IRC) that interests me. So, given that it had been suggested a few times that #perl6 should have a bot written in Perl 6, about a month ago I looked into writing one and soon gave up. The problem was Rakudo’s IO::Socket.rev() method which would only ever return once the connection closed. This worked fine for a MediaWiki bot operating over HTTP (which can connect, disconnect, receive with no problems) but was no good for an IRC bot (which needs a constant connection). It was, however, possible to have ‘blind’ “bot” that could send messages to the channel, but as fun as that was, the bot wouldn’t be able to see PINGs from the server.

Being so close to having a Perl 6 IRC bot, I eventually come up with this hack and a non-visually challenged quux was born. That hack evolved into this patch and IRC bots were suddenly possible on Rakudo.

Just over one week later and #perl6 now has three IRC bots written in Perl 6:

Zaslon: Polls RSS feeds and announces in the channel whenever a blog it follows is updated. Zaslon also takes commands to add new blogs to follow, remove blogs or post a link to a blog. If anyone is wondering, Zaslon is named after the Zaslon radar.

pointme: Pointme is a very simple bot that posts links to projects tracked by proto. If anything it is an example of how simple an IRC bot can be. Named after hugme, the idea being that hugme embraces people (adds them to the project) while pointme points them in the right direction.

Mubot: Mubot is a karma-tracking bot, written because everyone is sick of lambdabot responding to any message that begins with an “@” or “>” (and there might be a few other trigger characters). Mubot properly supports (Multiple word)++ identifiers (lambdabot would give “word)” karma) and aliases. It is named ‘Mu’ since Mu is the letter after ‘lambda’ in the Greek alphabet.

Of course, there are some issues with Perl 6 IRC bots at the moment.

The first issue is the lack of non-blocking IO, Zaslon is the bot that suffers mainly from this problem; it can’t poll the RSS feeds while it’s receiving messages from the IRC server. This isn’t so much of a problem for bots like Mubot that only need to quickly analyse each message as it comes in, but Zaslon takes a significant amount of time to poll the feeds which means if it was checking the RSS feeds when a command was given it could take ~1 minute for it to respond. This would also be a problem if it missed a PING from the server. So the work around was to split up the IRC and RSS parts into two processes. This and the memory problem I explain below forced me into using what was probably a better solution anyway; to have the RSS part run on a crontab. rakudobug++

The second issue is memory. This was really an accidental discovery on my part. Zaslon’s proof of concept was quuxx (look at the code at your own risk…), I had it running the Rakudo revision in which my recv patch was applied. When I wrote pointme I updated the rakudo, and set both quuxx and pointme going. After a while Parrot ran out of memory on my VPS, figuring it was because I had both running on a VPS where, between Apache, MySQL et al., memory is tight, I started quuxx again and left pointme turned off. After a bit Parrot ran out of memory again. After some experimenting I filed RT #70183. As a workaround I rolled my Rakudo back to 657d55cce1f1ded33fd1f731344bd31b33099cb8 and it has been working fine.

The bots themselves also have (coding) problems, Zaslon starts spouting nonsense if the RSS feed is unavailable when it tries to access it and things break if you try to add an RSS feed that Zaslon’s very limited RSS “parser” can’t grok (or a URL that isn’t an RSS feed).

I haven’t, however, had any of the segfaults I seen with quuxx but perhaps I haven’t had any of the bots running long enough to allow any to happen.

My todo list at the moment looks something like this:

* Make HTTP::Client handle errors better
* Make Zaslon handle RSS feeds it doesn’t understand better
* Make IRC::Client bots able to read/respond to private messages
* Make IRC::Client’s highlight method able to return the nick of the person who triggered it

So that’s that. I think I might start work on an IRC::Server now (although that is blocked by RT #70045)

A while back I bought quite an old box (a Compaq Evo) to use as a test system. This weekend I took a shot at getting OpenSolaris running on it, as the first step in my plan to move to OSOL from Ubuntu. Everything worked fine except networking. Fortunately, OSOL has one of the best pieces of software ever written installed on it by default; Sun’s Device Driver Utility.

It can be opened via the command ddu or, for the terminally-challenged, Applications -> System Tools -> Device Driver Utility.

This clever program informed me that this box used an “Intel 82801 DB PRO/100 VM” and that a third-party driver was required. Right-clicking on the Network card and selecting “Show details” brings up a dialogue which includes a URL pointing to the driver you need. In this case it can be found here. You want the latest “ife driver for intel”, it lists 8255x cards but rest assured that it works with the 82801.

Luckily OSOL detected my USB drive by default, as needing network connectivity to get network connectivity is one of the more frustrating computer paradoxes. It would be nice if it shipped with the driver needed but I assume there are copyright restrictions on it.

Installing it is easy; extract the archive, open a terminal in the directory and run pfexec make install. Restart and voila, network access. Now if I could just get domain names to resolve …

Update: To get domain names resolving I added OpenDNS’ address(es) to my /etc/resolv.conf. The default was what every other machine on my network uses and they work fine, so I’m not sure if it’s an OSOL problem, a hardware problem or a PEBCAR problem (Problem exists between computer and router).

Six months after the drama there is still no viable alternative to NoScript.

Back in May, there was commotion after the Firefox Addon NoScript changed AdBlock Plus to display ads on the NoScript website. The code which accomplished this was obfuscated and the user was not warned that this would happen. At the same time people began questioning whether NoScript’s constant updates (sometimes more than once a day!) where necessary or if it was just because the author wanted a reason to display his ad-laden website (the website is displayed by default after every update). After this I uninstalled NoScript and have been browsing the web NoScript free ever since. However, after some nasty browser hangs because of JavaScript heavy websites (looking at you, Slashdot) I decided to get back in the JavaScript-blocking game only to discover that NoScript is still the only viable choice.

Part of the fun of trying new platforms to test specific software is getting past the initial hurdles. On OpenSolaris it was figuring out how to install Git, Subversion, GNU Make and GCC (which I easily found out how to do). On Haiku it was figuring out why the compile was failing and how I could fix it. On OpenSolaris the second time round it was figuring out why the compile was failing when it didn’t the first time. I have tried a few other platforms and never got anywhere with them due to problems that I could not figure out how to get past; on Minix I had trouble with the compiler failing and the configure script not working, on Hurd I had trouble with … well … getting it to boot. So wondering what challenges I would face I tried OpenBSD.

OpenBSD installed nicely, configured the network correctly (having internet connectivity is a big bonus) and already had GCC. However, getting Rakudo and Parrot turned out to be difficult. Git and Subversion are not installed by default and after wrestling with the package manager and trying to install the dependencies I gave up, tar’d up a fresh Rakudo/Parrot on my main machine and SFTP’d it across. Full of apprehension as to what would go wrong, I ran Parrot’s Configure.pl script. It worked and spat out a Makefile. So I ran make install-dev. It ran for a few seconds and then failed with:

Byte order is not compatible

It was Google to the rescue turning up this PerlMonks post. The problem was I had cloned, checkedout and tar’d the files on a 64-bit machine, so I repeated the process on my laptop, SFTP’d the new tar and Parrot compiled quite nicely.

So I cd’d back to Rakudo and ran perl Configure.pl since I had used the –prefix flag on Parrot to install it to parrot_install/ in the Rakudo directory it should have detected the parrot_config automatically but I got an error:

===SORRY!===
Parrot revision r41448 required (currently r0)

To automatically checkout (svn) and build a copy of parrot r41448,
try re-running Configure.pl with the '--gen-parrot' option.
Or, use the '--parrot-config' option to explicitly specify
the location of parrot_config to be used to build Rakudo Perl.

I wasn’t paying attention and thought it hadn’t found the parrot_config binary, so I tried again with perl Configure.pl --parrot-config=parrot_install/bin/parrot_config and it did the same thing. So after I actually read the error message I figured it out; r0?

Parrot compiles its SVN revision number into the parrot_config binary, but since SVN wasn’t installed on the box it defaulted it to r0. My simple hack was to do:

$ echo 0 > build/PARROT_REVISION

From Rakudo’s directory. This makes the configure script expect a Parrot with revision 0, which it thought I had. After that it created a makefile and Rakudo compiled as expected.

I got the spectests by tar’ing up the directory on my laptop and SFTP’ing it across.

4 hours later I got the results:

S02-magicals/config failed because OpenBSD isn’t a known OS. Now fixed.

S03-operators/arith #163 and S32-num/power #11 fail because, like OpenSolaris, OpenBSD thinks 1**Inf = NaN.

S29-context/sleep #2 and 4 also fail because OpenBSD sleeps for too long if ‘use Test’ is included. I think this is the result of system slowness.

> say sleep 3;
3.25350594520569
> use Test; say sleep 3;
21.6377711296082

What to try next? NetBSD?

I must have commited hubris the last time I tried Rakudo on Solaris and had no trouble.

I booted up a fresh OpenSolaris partition today, the first time in about two weeks, installed all the tools and downloaded rakudo, however Parrot’s Configure.pl script gave out errors and it died during the build process as a result. Utterly confused (as it was working without any hassle two weeks ago) I decided to try Sun Studio’s C Compiler, and it built fine. So I ran the spectest and had a few failures that I do not remember getting last time.

The first failure was in S03-operators/arith.t #163 fails. This test checks that 1**Inf returns 1, however on OpenSolaris:

> say 1**Inf
NaN

On Ubuntu:

> say 1**Inf
1

Utterly perplexed I asked on IRC but received no answer as to why this was happening, so I made a note to report it later.

I moved on then to sedate my curiosity into $*OS. I thought its value depended on the result of uname -s but OpenSolaris reports that as ‘SunOS’ while its $*OS variable is ’solaris’, so while digging into the code (I never did figure out how it works) I found that $*OSVER is defined in Rakudo. On my Ubuntu box it returns:
#49-Ubuntu SMP Tue Aug 18 19:25:34 UTC 2009
and on Solaris it returns nada. It’s defined, but an empty string. Something to do with how Parrot works it out, I tried to dig into the source but didn’t understand it.

So having found a few issues I decided to look into my initial problems; GCC not working. I tried to compile it a few more times, each time it failing until the answer just came to me; Rakudo’s configure script was recently changed to pass –optimize to Parrot by default. It made sense, it Just Worked(tm) two weeks ago, in that time the –optimize flag was made a default and now it doesn’t work, so I tried building Parrot with GCC without the flag and it worked! Here the Parrot bug that was filed.

My original intent was to see what spectests faster, Rakudo with GCC-Parrot or Rakudo with SSCC-Parrot. So I started spectesting the GCC-compiled-Parrot Rakudo and some strange failures came up that didn’t fail on the first run; t/spec/S32-num/log # 12 and 16. (Just an aside, S32-num/power #11 also fails but that’s because of 1**Inf). Confused I looked at the tests but couldn’t work it out, I thought I might have missed those failures in my first run so I rebuilt both Rakudo’s, one with GCC and one with SunStudio CC and set the spectests going. While they were spectesting I had a thought and decided to test the test:

On Ubuntu:
> say log(-Inf)
NaN

On OpenSolaris:
> say log(-Inf)
-Inf

Then I had a weird thought … on the Rakudo using a Parrot compiled with SunStudio CC:
> say log(-Inf)
NaN

Oops?

I have no idea what is going on here, but I do know that Parrot and Rakudo need more people using other platforms

If I was using OpenSolaris as my $ONE_TRUE_OS I would have noticed this two weeks ago … I would love to be doing that but I life has been so easy with Ubuntu I don’t think I could handle the switch.

Just a postscript, when the –optimize change was made I booted my Haiku box, cloned a fresh Rakudo and rebuilt from scratch to test if it affected anything … curses!

Update 27/09/09: As of Parrot revision 41493 none of this is necessary, Parrot’s generic platform code checks if the time functions are available and replaces them if need-be (which is all moving the file from darwin’s code to generic was doing anyway).

The Haiku OS project released their Alpha 1 a few days ago and since then I have been trying it on and off. I like what I see and by their official release I expect it will be a very nice operating system.

As I did with OpenSolaris I wanted to put my time using the system to good use and test Rakudo on it. I happily discovered that GCC, Make, Subversion, Git and Perl are already installed by default so it was just a matter of getting Rakudo and building it.

As I unhappily discovered there are some issues which require a bit of hacking to get it to work. My methods are probably not ideal but they work

The first step was opening a terminal window; right-click, select Add-Ons and select OpenTerminal (or press OPT-ALT-T, opt is the ‘Super key’ or ‘Windows key’).

I checked out Rakudo with the familar:
~> git clone git://github.com/rakudo/rakudo.git
cd rakudo

Generated Parrot:
~> perl Configure.pl --gen-parrot

And it is this point that things go wrong. The Parrot build fails when building the time function, my quick and easy work around was to copy the hires_timer.c file from the darwin-specific code to the generic-code.

~/rakudo/parrot/config/gen/platform> mv generic/hires_timer.c generic/hires_timer.c.bak
~/rakudo/parrot/config/gen/platform> cp darwin/hires_timer.c generic/hires_timer.c

Attempting to rebuild Parrot got me past the initial hurdle but it still fails when dealing with the glut libraries. Luckily running

~/rakudo/parrot> perl Configure.pl --help

Yielded the –without-opengl flag for Parrot. Compiling Parrot without OpenGL (which doesn’t seem to be necessary anyway) makes it skip anything glut-related.

So all I needed to do was pass that command to Parrot, and repeating the help command in Rakudo’s directory informed me that I could do that the easy way with:

~/rakudo> perl Configure.pl --gen-parrot --gen-parrot-option='--without-opengl'

After this, Parrot compiled quite nicely and all was left was:

~/rakudo> make install

And there it is, a working perl6 binary on Haiku:

The spectests completed with only one failure which, as with Solaris, was easily fixed.

So now that you have successfully compiled Parrot and Rakudo on Haiku, run:
~> gcc --version

As you can see, by default, the Haiku project uses gcc 2.5 (which is 8 years old). They do this to retain compatibility with old BeOS programs. There is a GCC4 available, but apparently Parrot doesn’t need it.

However, assuming you don’t want a Parrot from the dark-ages, you can compile it with GCC4 by using:

~/rakudo> perl Configure.pl --gen-parrot --gen-parrot-option='--without-opengl' --gen-parrot-option='--cc=/boot/develop/abi/x86/gcc4/tools/gcc-4.3.3-haiku-090629/bin/gcc'

Make sure you wipe out your parrot_install first with ~/rakudo> rm -r parrot_install

Enjoy

Lately I have been thinking more and more that Ubuntu is not the operating system for me. Don’t get me wrong, it is fantastic and I would easily recommend it to anyone wanting to try Linux but in the interest of expanding my horizons I decided to try something different. My first thought was another Linux distribution, Gentoo being one I have meant to try for a long time now but instead I took the plunge into something completely different and installed OpenSolaris, an open source Unix-like OS based on Sun’s Solaris.

My first impression was how nice the UI looked (funny how we tend to notice the unimportant things first ) , my second impression was that it seemed rather familiar coming from Ubuntu.

To put my adventuring to good use I wanted to test Rakudo on it.

To install Rakudo in OSOL you need GCC, GNU Make, Subversion and Git. Perl is already installed.

To install these I did:

carlin@aethon:~$ pkg install SUNWgcc
carlin@aethon:~$ pkg install SUNWgmake
carlin@aethon:~$ pkg install SUNWgit
carlin@aethon:~$ pkg install SUNWsubversion

Then checked out Rakudo:

carlin@aethon:~$ git clone git://github.com/rakudo/rakudo.git
carlin@aethon:~$ cd rakudo

Configured Rakudo and generated Parrot:

carlin@aethon:~/rakudo$ perl Configure.pl --gen-parrot

And then ran make:

carlin@aethon:~/rakudo$ gmake install

You have to type gmake to use GNU Make. The default make command runs Sun’s make utility and I wasn’t able to get it to work.

Everything seemed to go as expected and at the end I had a working Rakudo:

carlin@aethon:~/rakudo$ ./perl6
> say 'OH HAI';
OH HAI

The spectest didn’t turn up platform-specific usualness, except that Solaris wasn’t a known OS to rakudo; which was easily fixed.

Starting a new blog.

In the past I have been terrible at keeping blogs updated and generally end up deleting them. I will try not to do that this time