Planet Fellowship

Tuesday, 09 February 2010

FOSDEM visitor seems to like my baby

Isabel Drost's blog | 08:19, Tuesday, 09 February 2010

Another picture that was taken before the first session early in the morning:

KDE Software Compilation 4.4.0 and OpenSolaris

Adriaan de Groot (Bobulate) | 08:09, Tuesday, 09 February 2010

Although KDE SC 4.4.0 isn’t officially released yet (or there’s no announcement on the dot yet), I’ve bumped the specfiles in the KDE4-OpenSolaris repository to 4.4.0 and kicked off a rebuild. That means that we should have new packages at about the same time that the major GNU/Linux distributions have them. The release candidate was pretty stable on OpenSolaris as long as you didn’t (re)build any parts of it while logged in, but we’ll probably be hearing enough bug reports in the next few months for OSOL. You can report problems to KDE’s bugzilla with the OS set to Solaris, or use the OpenSolaris bug tracker and choose one of the KDE components there.

Of course, the whole future of OSOL is a little .. fuzzy. The Oracle / Sun takeover has caused a fair amount of shake-up in many of Sun’s projects, but OSOL has not, as far as I know, been either committed-to or deprecated. We’ll see.

In the wake (er, .. the bow shock wave, since I’m getting ahead of myself) of the release, Jos asks for something offensive and the KDE website has been re-vamped. Especially the latter is impressive, for breathing new life into the somewhat moribund site. I believe the call for help still applies as the rest of the content is polished and brought up-to-date.

fl0rian

Florian Boor (Handhelds, Linux and Heros) | 00:34, Tuesday, 09 February 2010

I hope a few people wondered why my blog looked a little bit neglected in the past few months. Well finally I can say that I have been busy with several larger projects I was not supposed to talk about. For two projects I am involved in there are related press releases from our customers and business partners.

One project is the Linux port to the Höft & Wessel skeye.pos mobile – I really like the press release because it mentions the fact the supplied devices are running Linux and what the devices are used for. The filesystem on these devices is built with OpenEmbedded and is based on an older Angström release.

The other big project is closely related to both my job for kernel concepts and OpenEmbedded which is one of my favourite open source projects. The µCross distribution will support chip- and device vendors who are going to ship Linux-based solutions. The main idea is to combine the power OpenEmbedded and its large community with a good portion simplicity and a few additions. I do not want to mention too many boring details here so I will just introduce the basic concept: The idea is to offer customers binary packages matching their target architecture, matching toolchains and tools for assembling and configuring filesystem images for their devices.

There is not really an offical announcement yet but one of our business partners just announced a nice SBC module which will come with a µCross-based SDK. The TK71 is a QSeven format module powered by a Marvell 88F6281 SoC (Sheeva core based).

A third project that gained some love is the updated Linux port to the Toshiba Topas910 and TopasA900 boards. I am trying to maintain an upstream compatible and up to date Linux port to these devices here – for the people who do not want to use several year old kernels or this strange Aura stuff. The latest achievement is that I got some patches to make NAND flash work which is vital for the TopasA900 because its small NOR flash can’t keep a decent filesystem image with GUI.

Ok now I’m done with showing off and I should return to do something useful… such as writing a short report about FOSDEM!

FOSDEM!

Julia E Klein's blog | 00:07, Tuesday, 09 February 2010

Last weekend I was on FOSDEM for the first time. Although I didn’t see a single talk it was very interesting and a lot of fun, because it felt like the ‘cool’ version of CeBIT.

As far as I can say this year’s FOSDEM booth was a big success. We sold lots of T-Shirts with the new designs and colours and rumour has it that there will even be new womens’ T-Shirts soon (I will fight for the longsleeves ;-)). We also introduced our cute and funny Valentine’s Day Chocolate Hearts saying “I love Free Software, but I love you more.” that can be ordered now. I had a look at the drafts of our Valentine’s Day logo and it was very pink.
Additionally, we have won several new Fellows - even one from Greece!

On Monday before FOSDEM I talked to José Marchesi of GNUpdf who told me Richard M. Stallman would come to Frankfurt/Germany in the beginning of March. So we started thinking about a speech by RMS. On FOSDEM-Sunday we even dared to make first plans for possible locations for such a speech. In case we can talk RMS into giving a speech we will have a Fellowship Event there, too. But don’t worry, we will announce it before.
I would like to have some critical discussion e.g. about what happens to the work for Free Software when he passes on the torch and if in his opinion there is already someone who can carry it.
I really hope it works!

Monday, 08 February 2010

FOSDEM 2010 - part 1

Isabel Drost's blog | 21:00, Monday, 08 February 2010

Four years ago I was working in Saarbrücken. From there it is a very short ride over to FOSDEM (little more than 300km). So I decided - hey, why not stay there for a weekend. I found a very nice Brussels bed and breakfast hotel called Rovignon - featuring not only comfortable rooms at reasonable prizes but also cats in the house.

Back then, I barely knew anyone at the conference. However the lineup of speakers including St Peter from XMPP and Georg Greve from FSFE was impressive.

As a result it became a loved tradition of Thilo and myself to drive over to Brussels, attend FOSDEM and watch great talks. Over time there were more and more familiar faces, e.g. at the FSFE booth, among the Debian people…

Last weekend I had an awesome time in Brussels at Fosdem for the fourth time in a row. I am honoured to have been invited by the Fosdem organisers for a main track talk on Hadoop in the scalability slot.

I arrived on Friday afternoon, however being awefully tired I unfortunately could not join the Friday evening beer event (though, as I am not drinking beer, I would probably have missed quite a bit of the fun.

Bits of Privacy

Adriaan de Groot (Bobulate) | 15:20, Monday, 08 February 2010

The organization “Bits of Freedom is a Dutch NGO Privacy watchdog. It has been re-vivified in the past years. It hands out semi-regular Big Brother Awards (say, isn’t the estate of George Orwell going to wake up at some point?) for the worst offenses against privacy in the Netherlands. Worth reading if your Dutch comprehension is ok. BoF has garnered some mainstream media attention as well, which offers hope that privacy isn’t completely a lost cause in this country. [Current schemes of nationwide chipcard travel and road-pricing deny that hope] There’s also a “Winston Prize” for someone fighting for privacy, which went to the Euro-MP Sophie in’t Veld. It took me a minute to figure out the “Winston” name, but since I’d read 1984 (it says so right there in my book list) the light dawned eventually.

Anyway, cheers to BoF, and keep watching over your own privacy.

O noes, book list meme

Adriaan de Groot (Bobulate) | 13:11, Monday, 08 February 2010

I was reading some hip and literary Swedish blogs (no, actually I found this stuff on some tech blog, but swimming down the attribution links brought me these far more acceptable references) which talk about a list of books one might be expected to have read. The list has apparently been circulating for some time, witness the lonely librarian, and continues to be passed around, for instance on the symbian diaries (I can’t find any Symbian content there, though). It might be that the list stems from this story (from 2007?), but I’ve seen the Guardian reported as the original source as well.

The list itself (in the form in which it’s being passed around right now) is a little odd, since there’s some strange entries: the Bible (which version? does it matter? with or without the apocryphal books?), the complete works of Shakespeare (zounds, that’s a lot, but it also duplicates Hamlet much later in the list). The list mixes classics with much more recent work and includes some real cruft (IMO) as well. Anyway, I thought that publishing the list would be far too space-consuming and not interesting on a technical blog, so I wrote a python program to produce HTML and plain text renderings of the list, as well as a compact text representation using the initials of authors and titles and a base-3 representation.

You can run the program with something like python booklist.py text to get plain text output; alternate modes are html, code which outputs a kind of geek code, and short which produces a decimal integer. The program comes with an unread database, so it will print an all-unread text representation, or the integer 0. The integer corresponding to my own reading of the list is 472899411102988434671899921134218056756239761136, and you can add that as a second parameter to get the output showing what I’ve read; in code form that’s:

*PaP-JA *TLotR-JT *HPs-JR *TB-: *WH-EB *NEF-GO *GE-CD *C22-JH *CWoS-S *TH-JT *CitR-JS *M-GE *GWTW-MM *TGG-FSF -WaP-LT *THHGttG-DA -CaP-FD *GoW-JS *AiW-LC *TWitW-KG *CoN-CL *E-JA *P-JA *TL,TWaTW-CL *WtP-AM *AF-GO -OHYoS-GGM *AoGG-LM *THT-MA *LotF-WG -LoP-YM *D-FH *SaS-JA -ASB-VS. *ATOTC-CD *BNW-AH *OMaM-JS *TSH-DT *MD-HM *OT-CD *D-BS *TSG-FHB -U-JJ *ACC-CD -MB-GF *CW-EW *AoSH-SACD *TLP-ADS-E -TWF-IB *WD-RA *TTM-AD *H-WS *CatCF-RD

To generate your own book statuses, you’re going to have to edit the program and insert a status (READ or WANT) into some of the tuples that form the book database. Then run the program with only one argument for the desired output format. Once you have the code or short output, you can post it and keep the booklist meme alive (albeit in a more nerdy form).

The program itself can be downloaded as source from here, and is under the GPLv2 or later (bear that in mind if you send patches for an interactive of Qt version, and please don’t berate my python style).

There’s clearly features missing from the program: there is no Qt interface (with PyQt bindings) and no interactive mode, which would make it much easier to generate your own booklist output. Abstracting the purpose of the list would also be nice, and I can see a Meme-Plasmoid (a memeoid?) somewhere in the future where you get the checklist of the day and can enter your results for that checklist, to share across the social desktop. O noes!

Saturday, 06 February 2010

Medor v0.1 : How to protect your session using RFID tags

Alexandre DE DOMMELIN ( /(BB|[^B]{2})/ ) | 09:55, Saturday, 06 February 2010

As my RFID reader works now, i've written "Medor" a small solution to protect my desktop session.

The idea is quite simple, at work I often leave my computer for meetings ... and don't want to have to xlock / unlock my session.
On login, I start ssh-agent with all my different keys (Work & Personnal), so leaving a terminal open with them loaded looks dangerous to me. Medor checks every 5 seconds if my keys with my RFID tag are present, if they aren't the screen is locked to prevent somebody to use my session. When my keys are back, the session is unlocked.
There's another protection, if Medor detects that the RFID reader is no longer available (somebody tried to unplug it), the screen is locked, all keys in my ssh-agent are removed and a XMPP message is sent to my cell phone.

Medor use the RFIDIOT library to access the RFID reader, have a look at my previous article to know how to use it on Debian GNU/Linux.

Here are the 2 main scripts (up-to-date release will be available on my github under dotfiles/openbox/bin/Medor)

Medor.sh

#!/bin/bash
#
# Medor v0.1
# Alex "laotseu" DE DOMMELIN - http://blog.tuxz.net
#
# This program is free software. It comes without any warranty, to
# the extent permitted by applicable law. You can redistribute it
# and/or modify it under the terms of the Do What The Fuck You Want
# To Public License, Version 2, as published by Sam Hocevar. See
# http://sam.zoy.org/wtfpl/COPYING for more details.
#

RFID_ID="MYTAGID"
CHECK_TAG_SCRIPT="/home/laotseu/.config/openbox/bin/Medor/python-rfid/checkTag.py"
SCREENLOCK="xlock"
SCREENLOCK_OPTS="-mode blank"
XMPP_ALERT="/home/laotseu/.config/openbox/bin/Medor/xmpp_alert.py"



function protect() {
  (ssh-agent -k > /dev/null 2>&1)
  ($XMPP_ALERT "$(date) Security Alert : RFID reader unplugged" > /dev/null 2>&1)
  lock;
}

function lock() {
  ($SCREENLOCK $SCREENLOCK_OPTS &)
}

function unlock() {
  (/usr/bin/killall -9 $SCREENLOCK)
}



## Main Loop ##

ALERT_SENT=0
while [ 42 ];
do
  TAG=`$CHECK_TAG_SCRIPT 2>/dev/null`

  case $? in
    ############################
    ## Reader not present :-( ##
    ############################
    1 )
      if [ $ALERT_SENT -eq 0 ];
      then
        protect;
        ALERT_SENT=1
      fi;
    ;;


    #################################
    ## No tag present, lock screen ##
    #################################
    255 )
      (/bin/pidof $SCREENLOCK > /dev/null 2>&1)
      if [ $? -eq 1 ];
      then
        lock;
      fi;
    ;;


    #############################################
    ## Tag present, check if allowed to unlock ##
    #############################################
    0 )
      if [ "$TAG" == "$RFID_ID" ];
      then
        (/bin/pidof $SCREENLOCK > /dev/null 2>&1)
        if [ $? -eq 0 ];
        then
          unlock;
          ALERT_SENT=0
        fi;
      else
        (/bin/pidof $SCREENLOCK > /dev/null 2>&1)
        if [ $? -eq 1 ];
        then
          lock;
        fi;
      fi;
      ;;
  esac

  sleep 3
done;

checkTag.py

#!/usr/bin/python

import RFIDIOtconfig
import os

try:
  card = RFIDIOtconfig.card
except:
  os._exit(1)

if card.select():
  print "%s" % card.uid
else:
  os._exit(-1)

Friday, 05 February 2010

How to use Touchatag RFID USB reader on Debian GNU/Linux

Alexandre DE DOMMELIN ( /(BB|[^B]{2})/ ) | 23:29, Friday, 05 February 2010

I've bougth a few month ago a Touchatag RFID reader, a cheap USB device. It's reported to work on Windows / MacOS but not GNU/Linux. Here's a solution to use it on Debian.

You need to install some packages :
apt-get install python-pyscard pcscd pcsc-tools python-pycryptopp python-serial python-crypto

Then download the latest release of RFIDIOT (RFID IO Tools) here and simply extract it.

Plug-in the reader and start pcsc_scan, you should see something like that :

found one
Scanning present readers
0: ACS ACR 38U-CCID 00 00

The "0:" is the reader id, so ctrl+c out of pcsc_scan, edit RFIDIOtconfig.py, jump down to the readernum= directive, and change that to the correct id given by pcsc_scan.

You should now be able to test your reader using one of the contributed script such as multiselect.py.

Fully Free GNU/Linux Presentation Next Week

Ali Gündüz (GLOG Free People, Free Software) | 06:49, Friday, 05 February 2010

I will be giving a talk about the fully free GNU/Linux distribution movement in the Free Software Foundation Europe’s Berlin meeting on Thursday, February 11th, next week. The talk will start at 19:30 in the Newthinking Store, Tucholskystraße 48. Hopefully, we will have a chance to continue discussing the topic over a few beers right after the meeting, too.

Here’s the abstract of the presentation:

For the past few years, a growing number of GNU/Linux distributions have been started with the specific goal of ensuring users’ software freedoms before everything else. In this presentation, I will try to draw an overview of the fully free GNU/Linux distribution movement and provoke participants to think about software distributors’ ethical responsibilities. What aspects of mainstream GNU/Linux distributions does this movement not find sufficient? Which distributions align themselves with the movement? What is their rationale for limiting functionality of their software offerings in the name of upholding user freedoms? What does a binary blob mean and what does the Linux-libre Project claim to accomplish that the vanilla kernel Linux doesn’t? How does the fully free GNU/Linux distribution movement affect the rest of the free software ecosystem? I am planning to keep the overall tone of the discussion newcomer-friendly while also providing some food for thought for the technically inclined.

The wording has been deemed “clunky” by Karl Goetz, the lead developer of gNewSense, but it’s the best I managed to do in a busy week. If you are in Berlin next week and interested in discussing this new movement inside the free software world, you’re welcome to join us.

Thursday, 04 February 2010

FOSDEM – here we go

Henrik Sandklef (Free Software) | 23:09, Thursday, 04 February 2010

… having finished a hectic period I must say that leaving for FOSDEM tomorrow morning feels great. I will team up with Daniel Stenberg, Magnus Hagander and of course some GNUs, I know that some of the people at last year’s GNU Hackers Meeting (held the days before FSCONS) will come over to FOSDEM as well . Going to be great meeting Brian Gough and Jose E. Marchesi while I am not stressed of work overload, GHM planning and FSCONS planning. Brian and Jose were organising the GNU Hackers Meeting with me. … and as always I’ll team up with the FSFE folks.

As for the presentations I will surely go and see:

Sylvain Beucler talk about GNU Savannah: 100% free software mass-hosting

a topic I find interesting for many reasons, more on that later

Daniel Stenberg talk about Rockbox: open source firmware replacement for music players

since I missed Daniel talk about this at FSCONS. As a FSCONS organiser I can’t just sit

Mirko Lindner, Jon Phillips talk about Copyleft Hardware and the Ben Nanonote

I missed Mirko’s presentation at FSCONS as well …. always on the move at FSCONS

And, of course I hope I will be able to do some business (Sandklef GNU Labs) down there. During some breaks I will prepare some presentations at “my” University next week and also prepare a Free Software presentation for a company later on in February (more on that later).

This year I’ll be traveling with Andreas Nilsson and Jeremiah Foster. Same procedure as last year…

see you there

… did you see how I managed to mention FSCONS 6 times (including that very last)

Wednesday, 03 February 2010

Scratch that

Adriaan de Groot (Bobulate) | 22:07, Wednesday, 03 February 2010

I’m still sick. I still can’t hear properly, nor breathe, so as FOSDEM draws nearer (where I would man the FSFE booth, attend a few talks, and promulgate a business-friendly selection of Free Software licensing topics) I’m forced to re-evaluate whether I can go there at all: no, I am not going to FOSDEM. I’ll miss Paul’s bits on PIMping the KDE desktop, which is a particular shame. I’ll miss Michael Meeks, with whom I was looking forward to chatting about copyright assignment. I’ll miss Vincent Untz and the Dutch GUADEC team. I’ll miss Claudia and the KDE gang. I’ll miss the OSOL guys. I’ll miss the cute lovable blue Postgres elephant. And the weird devices at the OpenBSD stand. I could go on.

Instead, I’m going to stay home, eat lots of oranges, sleep a lot and hope for a better future. One where I can actually hold a conversation without phlegming anyone to death.

On a totally unrelated note, I’m reading “de telduivel”, a book on mathematics for kids by H.M. Enzensberger (English title “the number devil”, but the book is originally in German). It’s a fun book, and this time I’m actually reading it to the kids. Their minds do wander when we get to topics like fib(n)^2+fib(n+1)^2=fib(2n+1), but it’s inspiring for me as well. It’s been a long time since I sat down to try to do a little proof on paper, and since I don’t have to talk or breathe very hard to do math, it’s fun!

FOSDEM 2010 - 10 years FOSDEM

Isabel Drost's blog | 19:33, Wednesday, 03 February 2010

The final schedule of FOSDEM 2010 is up: Looks like bad news - 306 interesting talks within just one weekend. Lots of interesting talks in the main track including Greg Kroah-Hartman on “Write and Submit your first Linux kernel Patch”, David Recordon from Facebook on “Scaling Facebook with OpenSource tools”, Bernard Li on “Ganglia: 10 years of monitoring clusters and grids”, Andrew Tanenbaum with his “MINIX 3: a Modular, Self-Healing POSIX-compatible Operating System” talk, Benoît Chesneau on “CouchDB! REST and Database!” and many, many more.

In addition there will be many interesting DevRooms, including one on NoSQL, one on Free Java, the Mono DevRoom featuring a talk by Miguel de Icaza…

Looks like a weekend packed with interesting talks and discussions. If you are going there and are interested in an ad-hoc Hadoop-Beer-drinking meetup, make sure to contact me before the event.

YAIGTFP: PIMp My Desktop

Paul J. Adams (Green Eggs and Ham) | 17:56, Wednesday, 03 February 2010

So I am also going to FOSDEM:

I look forward to hugs and beer over the weekend. However, it’ll be an unusual experience for me this year… This will be the first time that I am at FOSDEM “on business”. Much of my time during the event is already preallocated to various meetings. Sadly this leaves very little time for attending talks…

Except my own! Come one, come all to the KDE devroom at 13:45 on Saturday: PIMp My Desktop

In this talk I will be giving an overview of the current state of KDE PIM technologies and the community behind them (yes, green blobs!)… plus a sneaky peek at some awesome things to come in 2010.

There are some truly sexy goings-on in PIMland, so this talk is definitely for you if you care about having control over your personal information.

Fellowship Jabber meeting “Meet the candidates”, Wednesday 17 February 19:00 CET

Fellowship News | 09:49, Wednesday, 03 February 2010

Hello Fellows,

all of you should now have received the voting e-mails for the second Fellowship seat in FSFE’s General Assembly (GA). You will have
time until the end of February to vote.

To give you the possibility to ask Julia and Björn questions which are not answered on the election pages about their candidature we will held a jabber meeting on:

Wednesday 17 February at 19:00 - 20:30 CET

How to participate in the Jabber meeting:

login with your jabber account (see Jabber Howto)
Join the multi user chat (MUC) room “fellowship” at the server conference.jabber.fsfe.org
Ask your questions

We encourage you to test connecting to the MUC before the actual meeting so you can contact us in case you have any difficulties.

Looking forward to see you at the meeting,
Matthias

Tuesday, 02 February 2010

Hadoop trainings in Europe

Isabel Drost's blog | 19:23, Tuesday, 02 February 2010

Recently I received this mail from Christophe Bisciglia on Cloudera Hadoop trainings. Thought it might be interesting to the Hadoop Berlin community:

Hadoop Fans,

Over the next year, you’ll see new options for Hadoop training and
certification from Cloudera. One of the first things you’ll see will
be live sessions outside the US, tentatively planned for the April /
May time frame.

We’ve seen strong interest in Hadoop on all of our international
trips, so we’d like to ask for community input as we decide exactly
which cities to visit next. For cities we come to, we’ll offer our 3
day developer training + certification, and with sufficient interest,
we may also include a 1 day training + certification program for
system administrators.

If you are interested in attending one or both of these sessions,
please fill out a brief survey (link below). If you’re using Hadoop at
work, and it’s time to train more of your team, you can let us know
how large of a group you have. Survey responses aren’t a commitment to
attend, but we may reach out to respondents before we schedule a
session to get a better understanding of actual attendance.

You can fill out survey here: http://www.surveymonkey.com/s/MKGZHG9

If you have any trouble with the survey, or are interested in a
private training session, please don’t hesitate to reach out directly.

Cheers,
Christophe

Monday, 01 February 2010

Shopping at Ikea

Isabel Drost's blog | 19:17, Monday, 01 February 2010

Some weeks ago, Thilo had a tiny little gadget not to be missed in an average geek’s appartment: A server - admittedly a little old and a bit slow, but still usable for playing around. He installed Ubuntu server on it. At the evening we got it configured to run Hadoop. Little later we found out that some friends of us probably, maybe have some usable hardware left as well - we’ll see on Monday.

However having a server on your dinner table is not really practical: There’s always some danger of spilling tea over it… However last week, one of my colleagues posted a link to the Lack Rack wiki page in the eth-0 Wiki on one of our mailing lists.

So yesterday was one of the (very rare) days, when I got Thilo to join me on a trip to Ikea. The result can be seen in the images above. Looks like elephants invaded our living room

KDE SC 4.4 beta 2 on OpenSolaris

Adriaan de Groot (Bobulate) | 16:12, Monday, 01 February 2010

The second beta release of the main collection of software produced by the KDE community, aka KDE SC 4.4 beta 2, was released last week. There are OpenSolaris packages available thanks to the compilation efforts of the Pavels; since we’re still not very good at publishing the packages in a standard repository, nor in managing smooth upgrades, you can find the packages here.

This set of packages includes Qt 4.6.1, which is a marked step up in stability; also Virtuoso support has been added if you build from source, but not in the packages. We’re still wrestling a bit with the C++ stack, as some things still use the older Cstd STL and “our” stack uses the newer Apache stdcxx4. In OpenSolaris, things are supposed to move to the Apache stack, but that takes some time and needs careful coordination, for instance because enchant plugins need to be moved over, but that implies that all of GNOME (the parts of it that use a C++ API anyway) also needs to be recompiled.

The first time I started up 4.4 beta 2 was a real eye-opener, from the new artwork to various bits of polish applied to the desktop (well, on OpenSolaris that includes things like “you can now start an application from the K-Menu again”, which was broken in all the recent versions). Since then I’ve started jotting down comments about aspects of the default installation that bother me — mostly little things, and it’s another indication that the packaging is coming together pretty well on OSOL that little bits that lack in polish are able to attract attention again.

Once I’ve got a more substantial list I suppose I’ll blog about them. That’s a little tricky to do, since many of them might be a “designed that way, but incompatible with [ade]” thing that I just don’t know about and I don’t want to come across as whiny or demanding (I’ll save that for licensing issues). And I don’t think that things like “Akonadi startup screen gets in the way” are worth filing a bug over, not until there’s some consensus that it is a bug. See also the “logout sound” thread on KDE core-devel right now, that feels similar.

Anyway, it’s good to see that the next release will be pretty solid, also on OSOL; if you want to take a peek, bother me at FOSDEM this weekend (just before the 4.4.0 release!).

FOSDEM schedule (my own)

Adriaan de Groot (Bobulate) | 12:40, Monday, 01 February 2010

Like many folks, I’ll be going to FOSDEM this weekend, in Brussels. Thomas Koch posts his whole weekend schedule — gosh, that’s ambitious. I know I never get around to attending one tenth of the talks I might intend to see, simply because there’s too many people to talk to, things to demonstrate, chats to have and random other interesting things to do. Given the state of my health, I think I’m going to remove “excessive amounts of beer” from my list-of-interesting-things, but I’m still looking forward to Brussels Cheese during the social hours. Isabel Drost is giving a talk on Hadoop — that’s one I might actually make a point of attending, because otherwise I’m never going to understand any of her blog posts. I guess like I’m tedious about licensing, she’s tedious about map-reduce. We’ll see.

For most of the weekend you will find me at the FSFE stand near the front entrance. I hope it’s less freezing cold than down the other end of the hall where the KDE stand is, traditionally. I’ll be available for legal and licensing talk, mostly. Governance? Patents? All ears. For technical stuff I will need to escape from the watchful eye of my handler, Hugo Roy, and covertly discuss bits and bytes.

Sunday, 31 January 2010

Hadoop at Heise c’t

Isabel Drost's blog | 13:37, Sunday, 31 January 2010

<surreptitious_advertising>
Interesting for those readers speaking German: Heise published an introductory article on Hadoop in its latest issue. Have fun reading.
<surreptitious_advertising/>

Thanks to Simon for proof-reading and providing valuable input. Thanks to Thilo Fromm for the hadoop graphics (unfortunately none of them got published in its original form), the catchy title, proof-reading the text over and over again and for keeping me sane during several past and coming months.

If you want to know more on Apache Hadoop, come watch my FOSDEM Hadoop talk next weekend. If you want to join discussions on Apache Hadoop and Lucene, stay tuned for a conference in Berlin on these topics.

Saturday, 30 January 2010

Going to FOSDEM 2010

Thomas Koch (Free Software) | 19:52, Saturday, 30 January 2010

I'm going to fosdem 2010.

These are things I'd like to attend:

time	room	topic	speaker
10:45-11:30	Janson	Promoting Open Source Methods at a Large Company	Brooks Davis
13:15-13:45	AY	Packaging Java Software for Debian	Thomas Koch
13:40-13:55	Ferrer	CiviCRM: Common goals of FOSS and Not For Profit Organisations	Xavier DUTOIT
14:00-14:45	H.1308	The Maemo Community Council: a case-study in governance	Dave Neary
14:00-15:30	Guillissen	LPI exam session 1	Klaus Behrla
14:00-14:45	Janson	What is my system doing - Full System Observability with SystemTap	Mark Wielaard
14:30-15:00	H.2213	Building Federated Social Networks on XMPP	Tuomas Koski, Simon Tennant
15:00-15:45	Janson	Ganglia: 10 years of monitoring clusters and grids	Bernard Li
15:00-15:30	H.2213	XMPP and the Social Web	Alard Weisscher, Laurent Eschenauer
15:30-16:00	H.2213	PubSub Gone Wild: Info Sharing at Mediamatic	Ralph Meijer
16:00-17:30	Guillissen	LPI exam session 2	Klaus Behrla
16:00-16:15	Ferrer	SIP Communicator: Skype-like conf calls with SIP Communicator	Emil Ivov
16:20-16:35	Ferrer	Kamailio (OpenSER) 3.0.0: redefinition of SIP server	Daniel-Constantin Mierla
16:40-16:55	Ferrer	asterisk: An introduction to Asterisk Development	Mark Michelson
17:00-17:15	Ferrer	csync: Roaming Home Directories	Andreas Schneider
17:30-18:00	H.2213	Jingle Nodes: An Open Alternative to Skype	Tiago Camargo
18:00-18:30	H.2213	Multi-User Jingle: Voice and Video Conferencing with XMPP	Dafydd Harries, Sjoerd Simons
18:15-19:00	H.1302	Spacewalk: Linux Systems Lifecycle Management	Marcus Moeller, Sandro Mathys


09:15-10:00	H.1302	Linux distribution for the cloud	Peter Eisentraut
10:00-10:45	Janson	RepRap - Manufacturing for the Masses	Adrian Bowyer
10:30-12:00	Guillissen	LPI exam session 3	Klaus Behrla
11:00-11:45	AW1.120	My life with HBase	Lars George
11:00-11:45	Janson	Tor: Building, Growing, and Extending Online Anonymity	Andrew Lewman
12:15-13:00	H.1302	Cross distro packaging with (top)git	Thomas Koch
12:15-13:00	H.1308	Shared libraries in Debian	Sune Vuorela
13:00-14:30	Guillissen	LPI exam session 4	Klaus Behrla
13:15-14:00	AW1.120	CouchDB, a database designed for the web and more	Benoit Chesneau
14:00-14:45	Janson	Large scale data analysis made easy - Apache Hadoop	Isabel Drost
15:00-16:30	Guillissen	LPI exam session 5	Klaus Behrla
15:30-16:15	H.1308	Continuous Packaging with Project-Builder.org	Bruno Cornec
15:30-16:15	AW1.120	Comparing the MapReduce way in CouchDB with the SQL way in a RDBMS	Stéphane Combaudon
16:15-16:45	AW1.120	Designing a scalable content management system on NoSQL technologies	Evert Arckens
16:15-17:00	H.1308	Debian Secrets: power tools for power users	Wouter Verhelst

Joined identi.ca

Matija "hook" Šuklje (blog) | 17:28, Saturday, 30 January 2010

It has finally happened — I succumbed to the microblogging craze!

Until now I never felt any urge to use a microblogging platform, because I think that if something is worth telling, you should tell it right and that SMS'ing the world in general is rather stupid. The only real use I can think of would be to join a group you belong to communicate there when writing a blog post or e-mail would be too much.

So, now that I plan to actively participate in FSFE (and hopefully FSFE Slovenia soon) and therefore found a use for it, I created an identi.ca account.

This being said, I still think people are (ab)using this medium way too much for just outright garbage...

hook out >> trying to figure out how to get Gnash to work in Konqueror

Friday, 29 January 2010

Server problems today

Fellowship News | 16:49, Friday, 29 January 2010

Dear Fellows, this morning the hosting centre where our Fellowship servers are located suffered a partial power outage.
Due to these circumstances beyond our control, the servers were offline until 14:30 today.

Unfortunately, this affected all Fellowship services, including the @fsfe.org mail forwarding. Delivery was delayed, no mail was lost. We apologise for any inconvenience.

Faunus is alive!

Matija "hook" Šuklje (blog) | 12:43, Friday, 29 January 2010

Finally, after an epic battle with disk encryption, the kernel and initramfs — my brand spanking new ~~shiney~~ matt ThinkPad is up and running!

World, be prepared for Faunus — the horned god of the forest!!!

Compiling even KDE on this puppy was a dream. I find it surreal how quiet and cool it is even when put under loads as high as 5 and 8 :3

hook out >> gotta run to the faculty for a meeting

March 2010 Apache Hadoop Get Together Berlin

Isabel Drost's blog | 08:40, Friday, 29 January 2010

This is to announce the next Apache Hadoop Get Together that will take place in newthinking store in Berlin.

When: March 10th, 5p.m.
Where: Newthinking store Berlin

As always there will be slots of 20min each for talks on your Hadoop topic. After each talk there will be a lot time to discuss. You can order drinks directly at the bar in the newthinking store. If you like, you can order pizza. We will go to Cafe Aufsturz after the event for some beer and something to eat.

View Larger Map

Talks scheduled so far:

Bram Smeets (JTeam/ Amsterdam): Spatial Search Abstract

Abstract: The rise in popularity of Google Maps and mobile devices with GPS have resulted in a trend in the search field. People are no longer content with finding results that match a text query, they also want to find results which are near a location. So called spatial search differs considerably from traditional free text search in that it cannot be achieved through common search techniques such as inverted indexes. Instead, new algorithms and data structures had to be developed that achieve efficient and accurate spatial search, that also allow spatial search to have a role in the determination of a result’s relevance. This technology has primarily been found in proprietary closed source search applications, however in the last 12-18 months, considerable effort has been invested into bringing open source spatial search support to Apache Solr and Lucene. While much is still left to be done, this talk will introduce how spatial search is currently supported in Solr, what work is happening currently, and a roadmap for future developments.

Dragan Milosevic (zanox/ Berlin: Product Search and Reporting powered by Hadoop

Abstract:

To efficiently process and index 80 million products, as well as store and analyse 30 billion clicks and 500 billion views daily, Zanox AG is using Hadoop HDFS and Map?Reduce technologies. This talk will present product-processing and reporting frameworks running on 17 node Hadoop cluster, being able to (1) robustly store products and tracking data in distributed manner, (2) rapidly consolidate, normalise and categorise products, (3) merge and aggregate tracking data and (4) efficiently builds indexes for supporting distributed search and reporting, running in several search clusters.

Bob Schulze (eCircle/ Munich): Database and Table Design Tips with HBase

Abstract: Recurring design patterns for the BigTable/HBase storage model.

A big Thanks goes to the newthinking store for providing a room in the center of Berlin for us. Another big thanks goes to Nokia Gate 5 for sponsoring videos of the talks. Links to the videos will be posted here.

Please do indicate on the following Upcoming event if you are planning to attend to make planning (and booking tables at Aufsturz) easier. Registration through Xing is possible as well.

Looking forward to seeing you in Berlin,
Isabel

Thursday, 28 January 2010

libarduino – hacked it a bit

Henrik Sandklef (Free Software) | 22:40, Thursday, 28 January 2010

Found a lib the other day that can be used instead of the Arduino IDE. The library is called libarduino. Using the lib means I can switch entirely to using C when teaching embedded at Gothenburg University/IT University. Only real problem was that the library really wasn’t prepared for producing a stand alone library or for being installed and will therefor not be that easy to use for the students. So I started hacking the makefiles to solve the problems. And now I have a libarduino that creates a static archive (library) and can be installed for later use by other programs (linking the libarduino library (libarduino.a)).

Cool thing is that the makefile for the new program becomes really small. For a new program, do a Makefile looking something like this:

SRC=test1.c

PROG=test1

include <your path>/libarduino/arduino.mk

… and you can now do (among other things)

make

make install

the last target install the application to your arduino.

sniffle (2)

Adriaan de Groot (Bobulate) | 14:18, Thursday, 28 January 2010

Another week gone by an no end to my cold in sight. This is getting to be really annoying, as it both fragments my work day and makes me unreliable in responding to just about anything — so various jobs (both real and hobby) have been falling behind.

The hobby bits are easier to write about, so here goes: I wanted to produce a screenie of Krita on OpenSolaris, but it turns out that it hand not been built in my first packaging (neither had KSpread, for that matter). Turns out they require Eigen. Eigen is a C++ template-based linear algebra library. So I installed the Eigen package (also produced by the KDE4-OpenSolaris project: make no mistake, this is a small group with a big software stack to take care of). That’s where the fun starts — I don’t think anything else makes serious use of Eigen. Maybe some stuff in KDEgames or KDeedu, but obviously because I hadn’t had Eigen installed previously, they (if they are there) have been silently ignored.

So, Eigen. Since the KDE4-OpenSolaris project aims to use the Sun Studio compiler (so that the resulting KDE4 could be included in the base system and so that i t can re-use any other C++ infrastructure in the system), we need to make sure that Eigen works with the compiler. In the case of a giant and complicated template library, that can be a big effort. And I’m immediately stymied by the Matrix class, where the compiler complains about multiple definitions of every type in the Eigen generic interface (Base, Scalar, and another half-dozen). I’ve not even managed to strip the examples down to an example that demonstrates the problem, which makes debugging — or asking the compiler gurus for help — difficult.

Suffice to say that Krita screenshots are not to be expected anytime soon.

On the upside, the release of KDE SC 4.4 RC2 means that the other bits are nearing stability, which means we can re-focus on stability and performance on OpenSolaris for a bit. Since Pavel has set up various continuous builds on our crunchy build box (courtesy of Sun Netherlands) so tracking in future should work a little better — I hope we have fewer separate patches to maintain in future as more stuff gets upstreamed.

iPad and information appliances, a free software angle

Henri Bergius (Motorcycle Adventures and Free Software) | 09:51, Thursday, 28 January 2010

Apple iPad is certainly interesting. It seeks to challenge the concept of PCs by providing something that is at the same time more personal, and a lot easier to use. The personal computer of the future.

Gone is difficult file organization - instead, applications use their own purpose-build content repositories. Instead of seeking software from many places, all of it is easily available in an App Store, all quality-controlled by Apple. And same thing with content - forget about bookshelves and stacks of CDs, instead simply dowloading all you need from iTunes.

This sort of user experience obviously comes with a cost. Important computing concepts like multitasking are not supported. The iTunes/App Store experience means that Apple is in the position to ensure no software or content competing with its or its business partners' business model gets on the device. And most of the content you buy for the device is DRM'd, meaning that you're only renting it for the time allowed by content owners, never buying.

Even with the limitations concerned I can see myself buying an iPad. It would serve as a very nice device for web surfing from the couch and as an e-reader on business trips. I can also see myself running demos and presentations from it instead of a laptop.

Even with the limitations concerned, it is likely that the iPad will happen, and will blaze the trail towards a new way of personal computing. Stephen Fry says it well:

Like the first iPhone, iPad 1.0 is a John the Baptist preparing the way of what is to come, but also like iPhone 1.0 (and Jokanaan himself too come to that) iPad 1.0 is still fantastic enough in its own right to be classed as a stunningly exciting object, one that you will want NOW and one that will not be matched this year by any company. In the future, when it has two cameras for fully featured video conferencing, GPS and who knows what else built in (1080 HD TV reception and recording and nano projection, for example) and when the iBook store has recorded its 100 millionth download and the thousands of accessories and peripherals that have invented uses for iPad that we simply can’t now imagine – when that has happened it will all have seemed so natural and inevitable that today’s nay-sayers and sceptics will have forgotten that they ever doubted its potential.

The success of iPad will mean more than just a completely new level of App Store economy. Other companies will certainly seek to emulate the model, coming up with their own post-WIMP devices and their own content and software ecosystems. This all will be a challenge for the free software movement.

The world of free software is still very much stuck in what computing was in the 90s. We think of desktop computers, we do not integrate with the web. And we do not get the transformation that is happening with personal computers. Taught by smartphones and cloud applications, users are moving from desktops through simple netbooks towards information appliances.

With information appliances you need a seamless user interface. You need an ecosystem where content comes alongside the software to utilize it. You need to move past the old WIMP metaphors and the idea of separation between data stored in a a file system and the software manipulating it.

So far the first convincing attempt towards this direction I've seen in the free software world is KDE's Social Desktop initiative. It allows users to connect with each other straight through the desktop, and it allows discovery of new applications and content to download and use straight in the applications. We also use it with Maemo's new App Downloader.

Threatened by the cloud from one end, and closed-ecosystem appliances from the other, it will be interesting to see how we react. Will we rise to the challenge and start providing new user experiences? Will we build a free cloud? Will we integrate with initiatives like Project Gutenberg and Creative Commons to provide the content integration? Will the open web be our safe haven?

Definitely interesting times to be a software developer.

Wednesday, 27 January 2010

Who To (svn) Blame In Times Of Trouble

Paul J. Adams (Green Eggs and Ham) | 11:20, Wednesday, 27 January 2010

To date, all of my work has looked at SVN artifacts and the contributors who interact with them: add, delete, modify, move etc. On a couple of occasions I have been asked why I do not look into the artifacts. I never really had a good answer and so now I have got around to it.

The question, I guess, is why would anyone want to look into the artifacts when looking into the health of a community? My previous work has shown how certain contributors can be shown to be responsible for lots of artifacts and lots of commits. But what are they doing to those artifacts with those commits? For all we know (based on the work so far) they are making trivial changes to the same line of code over and over.

As a start I have made use of svn blame to gather data one who “owns” each line of code in KMail. The script I used to achieve this looks like this:

#!/bin/bash

svnroot=$1

echo “<?xml version=\”1.0\”?>”
echo “<blame>”

for file in `svn ls -R $svnroot`
do
result=`svn blame –incremental –xml $svnroot$file`
echo $result
done
echo “</blame>”

So I give you fair warning, this script has the potential to:

take an age to complete;
really seriously annoy the svn sysadmins (i.e. do not use this against a project’s main development SVN server).

Having run this against KMail in trunk, I was able to find that the contributor responsible for the most commits and most artifacts was not the person with the highest blame count. This is, of course, a perfectly understandable scenario. It does mean, however, we need to remember an important characteristic of communities when considering if they are in trouble or not…

Embedded knowledge. If the person responsible for the most commits/artifacts was to get run over by a bus and the person with the highest blame was in that bus as it subsequently went over a cliff… Then KMail might be in trouble.

I have made a first (and admittedly very basic) attempt to visualize blame. I have simply visualized each KMail contributor as a blob and the blob is larger the greater the percentage of blame that contributor is responsible for. Here it is:

Now I haven’t checked, but I am pretty sure that there are some contributors that are too small to been seen on this visualization. Make sure you download the image to see it is in its full glory.

Any suggestions for how I might improve this are most welcome.

Speaking about Free Software at Chalmers

Henrik Sandklef (Free Software) | 10:34, Wednesday, 27 January 2010

Today, I have been invited to talk about Free Software at Chalmers. I’ll start off at 13.15…. I assume it is in one of the rooms in “Hörsalsvägen”

Anyhow, the presentation is open I think, so if you care to come over, just do so

Tuesday, 26 January 2010

Google and Open Video Standards

Torsten Grote (FSFE blog) | 14:54, Tuesday, 26 January 2010

As some of you might have heard, Google released an experimental player for Youtube that works without Adobe Flash, last Wednesday. While they are “very excited about HTML5 as an open standard” (from the announcement), they surprisingly chose not to offer the content as such and instead serve videos in a patent-encumbered format. As Google has not yet stated whether it will add videos in an unencumbered format, this clearly strengthens the stand of proprietary solutions in the ongoing debate about the future of video in HTML and disappoints those of us who expected Google to live up to their policies, published just last month, where they say:

whenever possible, use existing open standards. If you are venturing into an area where open standards don’t exist, create them. If existing standards aren’t as good as they should be, work to improve them

So, while some people ponder if Google has not really understood what Open Standards are (I would recommend FSFE’s definition to remedy this), others suspect a strategic interest: promoting their new web-browser Google Chrome which can make use of this new feature in contrast to now-competitor Firefox. Mozilla Firefox is the most-used browser that implements HTML5. Its authors have repeatedly made clear that they will not support proprietary solutions and explained their reasons for that quite well.

I can only appeal to Google, to live up to their self-set standards and be the change for the “open internet” they are always talking about. Don’t be evil.

FSFE awarded medal for good governance

Adriaan de Groot (Bobulate) | 14:15, Tuesday, 26 January 2010

The Theodor Heuss Foundation (German site, but the Wikipedia article is probably more informative), named after Germany’s first post-war president, is a foundation established to remember the political achievements of Theodor Heuss as an example for social commitment, moral courage and the dedication to fostering democracy. The foundation seeks “to bring attention to something, which has to be done and shaped in our democracy, without being finished” (Carl Friedrich v. Weizsäcker, 1965).

The foundation awards a yearly prize and medal to persons of high standing and organisations, which are groundbreaking in these areas of social commitment and the fostering of democracy.

This year the prize has been awarded to Oxfam and to FSFE. The FSFE press release is available (also in German, French, Italian, Dutch and Greek, thanks to our wonderful and dedicated translation teams). In these complicated days, it feels — to me, at least — a little weird to be side-by-side with an organization like Oxfam. They are helping rebuild Haiti. Providing direct support, like water and shelter; “helpin’ people with skin“, as Granny Weatherwax might say. The FSFE helps with skin, too, (for instance with licensing advice, workshops, and promotion booths at events), but for far more abstract goals. Which isn’t to say that Free Software does not help: some Free Software is really useful in a disaster.

It might feel a little weird in the current context, but it is an honor and a privilege.

Monday, 25 January 2010

FSFE’s opening statement at WIPO SCP/14

Karsten Gerloff's blog (Free Software, policy, open standards and all the rest) | 22:01, Monday, 25 January 2010

This week, the WIPO Standing Committee on the Law of Patents is meeting in Geneva. From FSFE’s perspective, the two most important points on the agenda are the relation between standards and patents, and limitations to patentability.

We’ll go into details in the coming days. On patents and standards, one obvious point is that Free Software can’t make use of standards that are patent encumbered - just think of MP3.

The discussion about limitations to what can be patented is clearly very important for Free Software. Here, the delegates at WIPO will discuss, among other things, whether there should be international rules regarding patents on software.

As I said above, we’ll be having those discussions in the coming days. For now, please click through to FSFE’s opening statement, delivered today:

Statement by the Free Software Foundation Europe
submitted at WIPO SCP/14 on January 25, 2010.

The Free Software Foundation Europe would like to congratulate you on your
election to the Chair of this very important committee, as well the
two Vice-Chairs. We are confident that under your able guidance,
discussions will be productive, inclusive and balanced. We would also
like to congratulate Mr Pooley on his new office as Deputy Director
General of WIPO, and wish him success in his work.

We thank the Secretariat once more for the preparation of the Report
on the Internation Patent System (SCP/12/3 Rev 2), which is an
excellent and comprehensive document. We welcome the evidence-based
approach which is permeates large parts of this important document. We
also commend the Secretariat for its hard work in preparing the
preliminary studies now before this committee.

We note in particular that the report on the international patent
system (SCP/12/3 Rev 2) states a clear economic rationale for the patent
system, describing it as one among several tools to promote innovation
and development. The report invites us to consider where this
particular tool can be productively applied; but also to think about
where other regulations — or indeed the absence of regulation –
would do more to promote innovation and development.

In the light of these considerations, we strongly support the proposal
made by Brazil to create a working program in this committee for the
discussion of limitations and exceptions, and their effectiveness in
addressing development concerns.

Getting the relationship between patents and standards right will be
key to safeguarding and promoting innovation in the technology
sector.

In order to make use of the broad selection of tools we have available
to promote innovation, we believe that this committee’s work programme
should include a discussion of open and collaborative approaches to
innovation. We note the support which groups such as the International
Chamber of Commerce have just now expressed for these approaches. The
committee would also be enriched by a discussion of Open Standards as
an approach to enabling innovation and lowering the bar to market
entry.

With regard to the so-called Anti-Counterfeiting Trade Agreement
(ACTA) that is being negotiated in Mexiko this week, we urge the SCP
to call on the negotiating countries to disclose the drafts for this
agreement, so that its consequences for the patent system can be
discussed, and we do not duplicate efforts. FSFE strongly objects to
the intransparent manner and secrecy in which the negotiations for
ACTA are being conducted. Such intransparency and secrecy are not
conducive to building confidence into the outcome of this negotiation
process.

With these preliminary remarks, we would like to conclude our
statement. We are looking forward to making more specific comments at
the time the preliminary studies are discussed.

Karsten Gerloff
President, Free Software Foundation Europe
Geneva, January 25 2010

Sunday, 24 January 2010

KOffice on OpenSolaris

Adriaan de Groot (Bobulate) | 20:44, Sunday, 24 January 2010

KWord on OpenSolaris screenshot Inge Wallin recently blogged about the portability of KOffice — spurred on, no doubt, by the success of the port to the Nokia n900 and to Haiku. So he listed GNU/Linux, Mac OSX, Windows, FreeBSD (thanks, Inge, for checking), Haiku. That list is missing (Open)Solaris though, which as a UNIX flavor. ought to be a pretty simple target.

Of course, Solaris has been a primary target for OpenOffice for ages, so KOffice is a little late to the game on this particular platform. But I guess that’s my fault, since I’m one of the packagers for KDE4 on Solaris, and I hadn’t gotten around to it yet. So this weekend I spent a little under two hours hammering together a specfile (RPM-style) for koffice and getting the whole darn thing to build. Screenshot of KWord in action as proof. I tried KPresenter as well, but that crashed on changing the list style, so I didn’t think that was a good demonstration.

Of course, no port is without its patches, so here they are:

constness — this patch matches the constness of parameters in definitions with those in declarations, so that int foo(const int) is defined with int foo(const int i) and not as int foo(int i). Now, since the last time constness-matching blew up, I’ve learned that this is really a bug in Sun’s compiler, because constness is not supposed to be mangled into the name. However, I’ll claim that code neatness demands that they match, anyway.
double — some math functions like sqrt() can take both float or double, and Sun’s compiler doesn’t just pick one when you pass it integer parameters, so we need some explicit disambiguation. This is common all across the KDE codebase.
NaN — the KOffice code uses val != NAN which uses the gcc-specific NAN #define; probably !isnan() is better.
math — the header file math.h in Solaris uses the identifier “exception”, which becomes ambiguous in the context of the STL, so it needs to be included earlier, rather than later. This patch bungs in math.h as the first include in a number of C++ files — not necessarily something to merge upstream, because it’s mostly working around a bug elsewhere.
stupid — yes, this is a stupid patch. I can’t convince Sun Studio that 8.5 * 1440 is an secretly an integer constant and that it shouldn’t complain about a bunch of initializers in the MS Word import filters. I’m not really sure what’s going on here, it’s something special about static const initializers of class variables, since doing the same in a non-class context works just fine.

So there you have it. Five tiny patches for a codebase of a little over 600000 lines of code. Nice. Tip of the hat (I have lots of hats, but all of them are baseball caps and I don’t have a decent Stetson) to Inge and the KOffice folks for a nice portable office suite. You may be troubled, but your code is good.

Increased Activity In KOffice

Paul J. Adams (Green Eggs and Ham) | 13:32, Sunday, 24 January 2010

My interested has just been piqued by the following Tweet:

Commit rate in !koffice is insane; since 1-1 we already have 954 commits, compare that to the average of 480/month last year

So what did January 2009 look like compared to 2010? The following plots show commits and committers per day in 2009:

… and here for 2010:

We can see that the number of committers has increased and so this has clearly had a knock-on effect to the commits. I have not run these numbers through R yet, but it looks like the increase in activity is proportionally greater than the increase in contributors. So, either the existing contributors are working faster, or are making smaller patches. Or both.

Any other thoughts?

Saturday, 23 January 2010

On freeing a project

Adriaan de Groot (Bobulate) | 18:35, Saturday, 23 January 2010

I saw over on LWN.net (a very valuable resource for technology articles, well worth your $5), that a RAW picture editor called RawTherapee had been re-licensed from Freeware (gratis, but proprietary) to Free Software, under the GPLv3. The application is something I won’t use myself, and I can’t even manage the comparison to DigiKam (which I don’t use either). Technologically it looks quite cool, from the cmake-based build (yes, having a build system that supports cross-platform development effectively can be a wonderful thing) to the (comprehensive and multi-lingual) ODT manuals. I miss the nice KDE reporting of missing dependencies summarized in a list at the end, though. Something to contribute, over there.

The author, Gábor Horváth, states some interesting reasons for freeing the whole thing; I personally think the “family” argument is the strongest one, and it’s important to spend time with your (small) kids. Of course, at 10 months you can still code with the baby on your tummy. The other motivations are largely a matter of growing the community of developers and contributors in order to get better coverage of all the aspects of development (GUI, documentation, algorithms, translations) and smoother development by not depending on a single individual. All good reasons, to be sure. Whether these are good motivations to choose GPLv3, I don’t know (over some other copyleft license that encourages community building).

I’m going to admit a very non-charitable reason for looking more closely at RawTherapee: I was intending to bitch and moan about the state of the source code and the licensing and then point out how it should be done, by gorm. Instead, I can point to RawTherapee (as of SVN revision 31) as an example of a Free Software project done (pretty) well:

(good) Clear license choice; although the license is in a file called “copying” instead of the canonical “COPYING”, it’s the complete text of the GPLv3.
(good) Every file has a complete copyright header, including sufficient license text and an identifiable copyright holder. As the project grows, it will be interesting to see if it manages to keep a good grip on its contributors — that is, if everyone who adds a copyrightable contribution to the code actually adds a meaningful copyright holder line to the file headers. This would be, roughly, clause 5a of the GPLv3.
(bad) The distribution from SVN includes a few pre-compiled libraries (jpeg, lcms, png, tiff, z) for windows and accompanying header files. Of that lot, jpeg doesn’t mention its license terms in the file header (but refers to a README that has been lost). I hunted down the relevant source distribution, and the README contains a license that stipulates that README must be included with the distribution if any source is distributed. Right, minor point, but something to clean up in a next release — although I’d like to think that it also illustrates that the license in the jpeglib header should be clearer. lcms is MIT licensed, the png license doesn’t seem to allow binary distribution at all — the permissions it gives are for source. On the other hand, the use of the code as a component in a product is encouraged — that suggests binary use is ok, but it’s not exactly explicit. Tiff is MIT again (when I say “MIT” I mean “Simple permissive 1-clause, possibly with a restriction on endorsement”) and libz is also permissive. So just one minor blot.

Anyway, a tip of the hat to Gábor for doing it right (and doing the right thing) and I wish him lots of success with his project.

Integrating Mailman With Kolab

Paul J. Adams (Green Eggs and Ham) | 11:53, Saturday, 23 January 2010

And now for something completely different…

Kolab is a groupware server cleverly composed as a lightweight integration of existing technologies… Cyrus, Postfix, OpenLDAP, Apache… Whlist, “out of the box”, Kolab is a excellent solution, there is one commonly-sought feature which does not come as part of the standard distribution… mailing list support by Mailman.

Getting Mailman to play fair with the other components of Kolab is easy when you know how. But the current documentation on the Kolab wiki is out of date. So before I update that, I am publishing the details here to provide a wide audience to the material.

Install Kolab

Follow the instructions on the Kolab website for installing Kolab from source

Grab the Mailman sources
- wget http://ftp.gnu.org/gnu/mailman/mailman-2.1.13.tgz
- tar zxf mailman-2.1.13.tgz
- cd mailman-2.1.13
Build and install mailman
- ./configure --with-username=list --with-groupname=list --with-mail-gid=kolab-n --without-permcheck --prefix=/kolab/usr/mailman --with-var-prefix=/kolab/var/mailman --with-cgi-gid=kolab-n
- make && sudo make install
The configure step is particularly important because it is at this stage that we configure mailman to run as a particular user. This user must be compatible with the rest of the Kolab setup.
Configure Mailman
- Create /kolab/etc/kolab/templates/mm_cfg.py.template
- Paste in the following:
  KOLAB_META_START
  TARGET=/kolab/usr/mailman/Mailman/mm_cfg.py
  PERMISSIONS=0640
  OWNERSHIP=root:list
  KOLAB_META_END
  
  from Defaults import *
  MTA = ‘Postfix’
  POSTFIX_ALIAS_CMD = ‘/kolab/sbin/postalias’
  POSTFIX_MAP_CMD = ‘/kolab/sbin/postmap’
  DELIVERY_MODULE = ‘SMTPDirect’
  SMTPHOST = ‘localhost’
  DEFAULT_URL_PATTERN = ‘http://%s/mailman/’
  DEFAULT_URL = ‘http://@@@fqhostname@@@/mailman/’
  DEFAULT_URL_HOST = ‘@@@fqhostname@@@’
  DEFAULT_EMAIL_HOST = ‘@@@postfix-mydomain@@@’
  DEFAULT_NNTP_HOST = ‘@@@fqhostname@@@’
  add_virtualhost(’@@@fqhostname@@@’)
  add_virtualhost(’@@@postfix-mydomain@@@’)
  
  The first 5 lines of this configuration are actually instructions to the Kolab template engine. Anything contained in @@@ is going to be replaced by the template engine with details you provided at the time Kolab was installed.
- Run /kolab/sbin/kolabconf to process the configuration template
Configure Apache
- Add the following to /kolab/etc/kolab/templates/httpd.local.template
Edit /etc/group and add the kolab-n user to the postfix group

This is important! Apache runs as the koab-n user and this step is required so that postalias can be called from the Mailman CGI scripts. If you do not do this you would will get some evil looking permissions error.

Restart Apache
- sudo /kolab/bin/openpkg rc apache restart
Prepare Mailman
- sudo /kolab/usr/mailman/bin/mmsitepass
- sudo /kolab/usr/mailman/bin/newlist mailman

The first of these commands allows you to set the Mailman site administrator’s password which is required for certain tasks, such as creating new lists. The second commands creates the mailman mailing list which is required as part of the Mailman deployment.

Start Mailman
- sudo /kolab/usr/mailman/bin/mailmanctl start

OK, that’s it. You can now access Mailman from the command prompt using the scripts in /kolab/usr/mailman/bin or through the web interface at http://your-server/mailman/. Archives can be found at http://your-server/archives/. New lists will properly work with the Kolab Postfix installation.

This setup is known to work on Ubuntu Karmic using Kolab 2.2.3 and Mailman 2.1.13.

The 7 deadly sins of (Java) software developers

Isabel Drost's blog | 09:09, Saturday, 23 January 2010

On Lucid Imaginations Blog Jay Hill published a great article on The seven deadly sins of solr. Basically it is a collection of his experiences “analyzing and evaluating a great many instances of Solr implementations, running in some of the largest Fortune 500 companies”. It is a collection of common mistakes, mis-configurations and pitfalls in Solr installations in production use.

I loved the article very much. However, many of the symptoms that Jay described in his article do not apply to Solr installations only. In the following I will try to come up with a more general classification of errors that occur when your average Java developer starts using a sufficiently large framework that is supposed to make his work easier. Happy about any input on your favourite production issues.

Remark: What is printed in italic is quoted as is.

Sin number 1: Sloth - I’ll do it later

Let’s define sloth as laziness or indifference. This one bites most of us at some time or another. We just can’t resist the impulse to take a shortcut, or we simply refuse to acknowledge the amount of effort required to do a task properly. Ultimately we wind up paying the price, usually with interest.

There is even a name for it in Scrum: Technical debt. It may be ok to take a shortcut, given this is done based on an informed decision. As with regular debt, you may get a big advantage like launching way earlier than your competitor. However as with real debt, it does come at a prize.

Lack of commitment

Jay describes the problems that are especially frequent when switching search applications: Humans in general do not like giving up their habits. A nice example described in more detail in a recent Zeit article is what happens each year in December/ January when the first snow falls: It is by no means irregular or not to be expected that it starts snowing in December in Germany. However there will be lots of people who are not prepared for that. They refuse to put on winter tiers in late autumn. They use their car instead of public transport despite warnings in public press. The conclusion of the article was simple: People are simply not willing to change habits they got used to. It does take longer and is a bit less flexible to get to work by public transport instead of your own car. It does require adjusting your daily routine, optimising your processes.

Something similar happens to a developer that is “forced” to switch technology, be it the search server, the database, the build system or simply the version control system: The old ways of doing stuff simply may not work as expected. New tools might be called for. New technologies to learn. However in not so seldom cases developers just blame the new tools: “But with the old setup this would always work.”

Developing software - probably more than anything else - means constant development, constant change. Technologies shift as tasks shift, tools are improved as workflows change. Developing software means to constantly watch closely what you are doing, reflecting on what works and what doesn’t and changing things that don’t work. Accepting change, seeing it as a chance rather than an obstacle is critical.

If however change is imposed on developers though good arguments in favour of the old approach exist, it may be worth the effort to at least take the technical view into account to make an informed decision.

Not reviewing, editing, or changing the default configuration files.

I have extended this one a bit: Developers not changing default configuration files are not that uncommon. Be it the default database configuration, default logging configuration for your modules or default configuration of your servlet container. Even if you are using software pre-packed by your distribution, it is still worth the effort to review configuration files for your services and adjust them to your needs. Usually they are to be used as examples that still need tweaking and customization after roll-out.

JVM settings and GC

If you are running Java application there is no way around to adjust GC settings as well as general JVM settings to your particular use case. There are great tutorials at sun.com that explain both the settings themselves as well as several rules-of-thumb of where to start. Still nothing should stop you from measuring your particular application and its specific needs - both, before and after tuning. Along with that goes the obvious recommendation to simply “know-your-tools” - learning load testing tools shortly before launch time is certainly no good choice. Trying to find out more on Java memory analysis late in the development cycle just because you need to find that stupid memory leak like *now* is no good idea neither.

There are several nice talks as well as several tutorials available online on the topic of JVM tuning, debugging memory as well as threading issues, one of them being the talk by Rainer Jung at Frocson 2008.

Sin number 2: Greed

Running a service on insufficient hardware (be it main memory, harddisks, bandwidth, …) is not only an issue with Solr installations. There are many cases where just adding hardware may help in the short run, but is a dead-end in the long run:

Given a highly inefficient implementation, identifying bottlenecks, profiling, benchmarking and optimization go a long way.
Given an inappropriate architecture, redesign, reimplementation and maybe even switching base technologies does help.

However as Jay pointed out, running production servers with less power than your average desktop Mac has does not help neither.

Sin number 3: Pride

Engineers love to code. Sometimes to the point of wanting to create custom work that may have a solution in place already, just because: a) They believe they can do it better. b) They believe they can learn by going through the process. c) It “would be fun”. This is not meant to discourage new work to help out with an open-source project, to contribute bug fixes, or certainly to improve existing functionality. But be careful not to rush off and start coding before you know what options already exist. Measure twice, cut once.

Don’t re-invent the wheel.

As described in Jay’s post, there are developers who seem to be actively searching for reasons to re-invent the wheel. Sure, this is far easier with open source software than with commercial software. Access to code here makes the difference: Understanding, learning from, sharing and improving the software is key to free software.

However there are so many cases where improve does not mean re-implement but submitting patches, fixing bugs, adding new features to the orignal project or just refactoring the original code and ironing out some well known bumbs to make life easier for others.

Every now and then a new query abstraction language for map reduce pops up. Some of those really solve distinct problem settings that cannot (and should not) be solved within one language. Especially if a technology is young, this is pretty usual as people try out different approaches to see what works and what does not work out so well. Good and stable things come from that - in general the fittest approach survives. However, too often I have heard developers excusing their re-invention by “having had too few time to do a throughough evaluation of existing frameworks and libraries”. The irony here really is that usually, coding up your own solution does take time as well. In other cases the excuse was missing support for some of the features needed. How about adding those features, submitting them upstream and benefitting from what is already there and an active community supporting the project, testing it, applying fixes and adding further improvements?

Make use of the mailing lists and the list archives.

Communication is key to success in software development. According to Conway’s law “Organizations
which design systems are constrained to produce systems which are copies of the communication structures of these organizations.” I guess it is pretty obvious that developing software today generally means designing complex systems.

In Open source, mailing lists (and bug trackers, the code itself, release notes etc.) are all ways for communication. (See also Bertrand’s brilliant talk on open source collaboration tools for that). With in-house development there is even added benefit as face-to-face communication or at least teleconferencing is possible.

However software developers in general seem to be reluctant to ask questions, to discuss their design, their implementation and their needs for changes. It just seems simpler to work-around a situation that disturbs you instead of propagating the problem to its source - or just asking for the information you need.

However asking these questions, taking part in these discussions is what makes software better. It is what happens regularly within open source projects in terms of design discussions on mailing lists, discussions on focussed issues in the bug tracker as well as in terms of code review.

There are several best practices that come with Agile Development that help starting discussions on code. Pair programming is one of these. Code reviews are another example. Having more than two eye balls look at a problem usually makes the solution more robust, gives confidence in what was implemented and as a nice side effect spreads knowledge on the code avoiding a single point of failure with just one developer being familiar with a particular piece of code.

Sin number 4: Lust

Must have more!You’ll have to grant me artistic license on this one, or else we won’t be able to keep this blog G-rated. So let’s define lust as “an unnatural craving for something to the point of self-indulgence or lunacy”. OK.

Setting the JVM Heap size too high, not leaving enough RAM for the OS.

Jay describes how setting the JVM RAM allocation too high can lead to Java eating up all memory and leaving nothing for the OS. The observation does not apply to Solr deployments only. Tomcat is just yet another application where this applies as well. Especially with IO-bound applications giving too much memory to the JVM is grave as the OS does not longer have enough space for disk caches.

The general take-away probably should be to measure and tune according to the real observed behaviour of your application. A second take-home message would be to understand your system - not only the Java part of it, but the whole machine from Java, the OS down to the hardware - to tune it effectively. However that should be a well known fact anyway. For Java developers, it sometimes helps to simply talk to your operations guys to get the bigger picture.

Too much attention on the JVM and garbage collection.

There are actually two aspects here: For one, as described by Jay it should not be necessary to try every arcane JVM or GC setting unless you are a JVM expert. More precisely, simply trying various options w/o understanding, what they mean, what side-effects they have and in which situations they help obviously isn’t a very good idea.

The second aspect would be developers starting with JVM optimization only to learn later on that the real problem is within their own application. Tuning JVM parameters really should be one of the last steps in your optimization pipeline. First should be benchmarking and profiling your own code. At the same stage you should review configuration parameters of your application (size of thread pools, connection pools etc.) as well your libraries and frameworks (here come solr’s configuration files, Tomcat’s configuration, RDBMs configuration parameters, cache configurations…). Last but not least should be JVM tuning - starting with adjusting memory to a reasonable amount, setting the GC configuration that makes most sense to your application.

Sin number 5: Envy

Bah!

Wanting features that other sites have, that you really don’t need.

It should be good engineering practice to start with your business needs and distill user stories from that and identify the technology that solves your problem. Don’t go from problem to solution without first having understood your problem. Or even worse: Don’t go from solution (that is from a technology you would love to use) to searching for a problem that this solution might solve: “But there must be a RDBMS somewhere in our architecture, right?”

Wanting to have a bigger index than the other guy.

The antithesis of the “greed” issue of not allocating enough resources. “Shooting for the moon” and trying to allow for possible growth over the next 20 years. Another scenario would be to never fix your system but leave every piece open and configurable, in the end leading to a system that is harder to configure than sendmail is. Yet another scenario would be to plan for billions of users before even launching: That may make sense for a new Google gadget, however for the “new kid on the block”? Probably unlikely, unless you have really good marketing guys. Plan for what is reasonable in your project, observe real traffic and identify real bottlenecks once you see them. Usually estimations of what bottlenecks could be are just plain wrong unless you have lot’s of experience with the type of application you are building. As Jeff Dean pointed out in his WSDM 2009 keynote, the right design for X users may still be right with 10x the amount of users. But do plan a rewrite at about the time you start having 100x and more the amount of users.

Sin number 6: Gluttony

“Staying fit and trim” is usually good practice when designing and running Solr applications. A lot of these issues cross over into the “Sloth” category, and are generally cases where the extra effort to keep your configuration and data efficiently managed is not considered important.

Lack of attention to field configuration in the schema.

Storing fields that will never be retrieved. Indexing fields that will never be searched. Storing term vectors, positions and offsets when they will never be used. Unnecessary bloat. Understand your data and your users and design your schema and fields accordingly.

On a more general scale that might be wrapped into the general advise of keeping only data that is really needed: Rotate logs on a schedule fit to your business, operations needs and based on available machines. Rotate data written into your database backend: It may make sense to keep users that did not interact with your application for 10 years. If you have a large datacenter for storage that may make even more sense. However usually keeping inactive users in your records simply eats up space.

Unexamined queries that are redundant or inefficient.

Queries that catch too much information, are redundant or multiple queries that could be folded into one are not only a problem for Solr users. Anyone using data sources that are expensive to query probably knows how to optimize those queries for reduced cost.

Sin number 7: Wrath

Now! While wrath is usually considered to be synonymous with anger, let’s use an older definition here: “a vehement denial of the truth, both to others and in the form of self-denial, impatience.”

Assuming you will never need to re-index your data.

Hmm - don’t only backup. Include recovery in your plans! Admittedly with search applications, this includes keeping the original documents - it is not unusual to add more fields or to want to parse data differently from the first indexing run. Same applies if you are post-processing data that has been entered by users or spidered from the web for tasks like information extraction, classifier training etc.

Rushing to production.

Of course we all have deadlines, but you only get one chance to make a first impression. Years ago I was part of a project where we released our search application prematurely (ahead of schedule) because the business decided it was better to have something in place rather than not have a search option. We developers felt that, with another four weeks of work we could deliver a fully-ready system that would be an excellent search application. But we rushed to production with some major flaws. Customers of ours were furious when they searched for their products and couldn’t find them. We developed a bad reputation, angered some business partners, and lost money just because it was deemed necessary to have a search application up and running four weeks early.

Leaving that as is - just adding, this does not apply to search applications only

So keep it simple and separate, stay smart, stay up to date, and keep your application on the straight-and-narrow (YAGNI ). Seek (intelligently) and ye shall find.

sniffle

Adriaan de Groot (Bobulate) | 08:19, Saturday, 23 January 2010

Last week I was in Berlin, and besides getting some work done, picking up some espresso beans and meeting old friends, I also received a wonderful German cold bug. From my boss Karsten, presumably. We all know that parents of small children are plague-bearers, but his kids are apparently in a different pathogen group than mine — as a result of which I’ve been pretty much laid-up all week with sniffles, headaches and the like. I can focus for about 2 hours at a time, after which it’s back to bed. I don’t recall colds hanging around this tenaciously or virulently before; the only good thing I can say about it is that 2 hours is enough time to write blog entries on random legal and licensing topics (and also technical KDE things, but that’s coming up).

Friday, 22 January 2010

Overcompensating

Adriaan de Groot (Bobulate) | 18:16, Friday, 22 January 2010

Overcompensating is a webcomic (that one is safe for work) which I’ve been reading for years. Patently-O is a blog about patents which I read occasionally — I must admit that my interest in comics pre-dates my interest in patents. It (Patently-O) has a link up to a guide to patent damages. I’m not going to recommend to people on PlanetKDE to read it in particular (PlanetFSFE, maybe), but a few things struck me as I paged through it: one, that it’s remarkably readable, for a document produced by the legal profession; two, that it’s a practices guide that is quite concrete in what needs to be done and how things are handled; three, that it doesn’t talk about non-commercial infringement at all, as far as I can tell.

Since patents on software implementations of algorithms and the like continue to be granted in the US — witness HP’s patent on choosing a number base for a given processor or Google’s patent on map-reduce (apply a function to items in a list, then aggregate) — understanding the potential end-results (damages when infringing) is a big deal.

New e-mail address

Matija "hook" Šuklje (blog) | 14:17, Friday, 22 January 2010

Since today, my default e-mail address is coherent with my domain.

When e-mailing me, please use the following form:

<name> [at] <surname> [dot] name

hint: my website's URL is

<name>.<surname>.name

hook out >> making some Iranian tea and studying

RSS 2.0 | FOAF | OPML

Planets in: