A Slide Generator for EasyChair

If you chair a program committee with a physical meeting and use EasyChair as a conference management system, then the EasyChair Slide Generator is your friend. This script

• generates slides structuring the discussion at the PC meeting
• generates personalized discussion lists for each PC member
• generates a conflict-free seating order for all PC members.

EasyChair Slide Generator comes as a Python script, tested on Mac and Linux systems. As input, it takes an EasyChair data export; see the included README.txt file for details.  Enjoy!

=>  EasyChair Slide Generator

My top ten presentation issues in other's papers

I am a frequent reviewer of submissions for scientific journals and conferences.  By far most of my reviews are specific to the individual paper, its contributions, and its issues.  However, there is a small number of presentation problems which I have to report again and again.  All these problem are very minor, and none of them will make me reject your paper.  However, fixing them is a matter of minutes, and will make your paper an easier read for both readers and reviewers:

1. Abstracts without results.  Many abstracts end with the words "We conducted a study to show the effectiveness of our approach".  There's three things wrong in here:
1. You conduct a study to evaluate your approach – if you run your study with a predetermined outcome, I will suspect a flaw in your setup.
2. As a reader of your abstract, I am not so much interested in your process, but in your results; if you want cliffhangers, go and write TV shows instead. 
3. You need results to sell your paper in the review process – in particular for outsiders who take only a brief glance at your paper.
So make this "Our study shows a 25% precision increase" or some other strong argument.
2. Multi-letter identifiers in LaTeX math mode.  In LaTeX, $foo$ stands for the product of three variables f, o, and, o; and LaTeX typesets it as such.  If you want one variable, use $\textit{foo}$.  (For a computer scientist, this is the single biggest design failure of TeX and LaTeX.)
3. Confuse hyphens, minus signs, and dashes.  All these typeset into different characters: 
1. Hyphens: state-1 is blue, state-2 is red.
2. Minus signs: $\textit{bar} = \textit{foo} - 1$
3. En-dashes (for ranges): A car has 3--4~wheels
4. Em-Dashes (for sentences): He said ``captain''---I said ``wot''.  (This can also be typeset using en-dashes with spaces, as in He said ``captain''~-- I said ``wot''.)
4. Bad capitalization in BibTeX references, as in "[1] Jai: a javascript api ide" (should be "[1] JAI: A JavaScript API IDE")  I could include this in all my reviews and be right 60% of the time.
5. Omitting reference information.  I see papers omitting page numbers, conference names, editors, or even authors.  I suppose that "[1] Smith et al., ICSE 2004" is something I should Google, and I hope there's just one Smith at ICSE.  (This even occurs in papers whose venues have no page limit for references!)
6. Orphaned numbers, as in "I want 17<newline>foos."  Tie numbers to items using non-breakable space (Word) or using a tilde (LaTeX): I want 17~foos.
7. Orphaned lines and titles.  Section title at end of page 3, actual section on page 4.  Better yet: a single line of text left under a table, such that you easily confuse content and caption.  "LaTeX gets this wrong" is not an excuse; please fix this.
8. Using monospacing with a non-monospaced font (or vice versa).  This seems to be a default setting of the LaTeX "listings" environment – using Times fonts and arranging the letters at equal distance.  Use monospaced fonts with monospacing, and proportional fonts otherwise.
9. Omitting articles, as in "Main feature is lack of articles".  Now that spell checkers weed out the most blatant errors, the more subtle errors remain – and this is my #1 grammar issue.  If your native language has no articles, take double care.
10. Testing the limits.  Page limits are there for a reason – not to save trees, but to save the time of your readers by requiring you to focus on the main points.  Cramming every little detail into the paper will decrease readability and alienate your readers; it may also lead to desk rejection if you alter the format.  Stick to standard style, cut down your paper to one page less than the limit and have your paper proof read by others, then add whatever your proof readers missed.

All these issues can be fixed through simple proofreading, and leave your readers and reviewers more time to enjoy the actual content.  And now that the syntax is done, let's get back to semantics!

(As a reviewer or reader, what are your favorite gripes?  Use the commenting function below.)

We are creating a Start-up in Web testing

Today, my chair has obtained a 500.000 EUR research transfer grant ("EXIST-Forschungstransfer") for funding a start-up in the area of Web application testing.  This project has been running mostly undercover for the past two years, and I'm more than happy to see it kick off now.

The key product is WEBMATE – a service for systematic testing of Web applications.  The scenario is that you go to our website, enter the URL of the site you want to get tested, and WEBMATE will automatically and systematically explore all the functionality on the site.  In its initial inception, it will do so in parallel on multiple browsers and operating systems ("cross-browser testing"), and automatically detect if something works in one browser, but not on the other.  WEBMATE is very good at exploring functionality, be it in Javascript or server-side, and thus easily can find errors.  Plus,  you can have it run as often as you like, for instance as part of a paid subscription.  A service like WEBMATE also forms a great base for regression testing or security testing, and we're thrilled by the many market and research opportunities.

Right now, the EXIST funding secures the WEBMATE team such that they can prepare for building and monetizing the service.  The team consists of three scientists (Martin Burger, Valentin Dallmeier, and Michael Mirold) as well as an experienced IT entrepreneur (Bernd Pohl).  In the past months, these four have spent countless hours interacting with potential customers, business angels, and venture capitalists.  (If you've seen me in a suit in the past months, that typically was due to one of these activities.)  At this point, let's simply say that both the research and financial prospects of automated system test generation are very promising.

From a personal perspective, I will act as mentor and shareholder of our start-up.  At the chair, we will use WEBMATE for our own research purposes, though; and if our new techniques work well, we  now have a process to bring them to the public (and earn money at the same time).  And sure, it feels good to be a professor and an entrepreneur at the same time :-)

To learn more about WEBMATE, see us at CeBIT (Hall 5, D04 for our start up, and Hall 9, F34 for our research), visit our website, or read our papers.  If you're a journalist, check out our press release.

Twelve tips on how to prepare an ERC grant proposal

In 2011, I have been lucky to obtain an ERC Advanced Grant.

The European Research Council (ERC) is a EU institution that promotes high quality research in Europe.  It funds individual investigators in any field of research – and it does so substantially: With up to 3.5 Million Euros, an ERC grant is Europe's highest research funding for individuals – and a very coveted prize: Only about 12% of proposals get funded, so competition is fierce.

Since I got my grant, other applicants have asked me again and again for hints and samples on how to prepare a proposal.  Of course, there is no single recipe for success, but there were a few points which I found useful in preparing my proposal.  While specific for ERC proposals (and from a computer scientist perspective), these tips should generalize for several other high-profile funding programs.

The Process

1. Understand the process.  The ERC publishes a Guide for Applicants as well as a Guide for Reviewers.  Both should be your bible; at all times, ask yourself how your proposal will stand according to the criteria and the process listed.  Find out what your panel is, who the chair will be, and which past members have been on the panel.  Your proposal will need to win all of them.
2. Start many, many months before the deadline.  Unless your story is a winner straight from the inception, you will need lots of time for refining and revising the main idea and the many problems.  In my case, I started writing the proposal 18 months before the deadline; although 6 months would have been okay, too, refining for another 12 months helped the proposal a lot.
3. Reserve several weeks for writing.  You will need lots of time for collecting data, shaping the story, and checking the references.  Consider a 2–3 week retreat for the writing alone, plus appropriate time for polishing.  Let your friends and family know when you'll be back.
4. Get plenty of feedback.  Your proposal will first be reviewed from people in your discipline, but not necessarily from people in your speciality.  It may also be that your proposal will have to stand against proposals from totally different disciplines.  Hence, your story must appeal to readers no matter what discipline and speciality they're from.  Discussing your ideas and your proposal with as many people as possible and as diverse as possible will help.  In my case, I had the proposal reviewed by 12 internal and 12 external people, and used every possible invited talk to present some sketches of the main ideas.  (Such presentations not only help you to make your ideas explicit, but will also lobby for your ideas, and get feedback from the audience.)
5. Rely on local expertise.  ERC projects are huge, and thus involve substantial budget and resource planning.  If your university has support for EU and/or ERC proposals, rely on their expertise.  (If you have a colleague who is already funded by the ERC, check with her or him as well, of course!)

Your Achievements

6. Sell yourself.  Your proposal will be assessed on two criteria.  50% is your project, and it will be up to you to come up with a great idea.  50%, however, is your past achievements, and you will have to work hard on these.  What you need is irrefutable evidence for impact and excellence.  That is, facts on awards, services, papers, talks, students, tools; lasting impact in academia and industry; your quality as networker and advisor; and, last but not least, your ability to shape and create research fields.  Play by numbers: acceptance rates, citations, downloads.  Check the list of past grantees, their numbers and achievements to get an idea of what you're up against.
7. Have unique selling points.  "So, you're Brad Pitt? That don't impress me much."  When you're surrounded by supermen (and you will be), just being another superman is not enough.  So:
• Don't just say: "I am an ACM Fellow".  But say: "I am the first ACM Fellow from Spain", or "I am the youngest European ACM Fellow in concolic testing".  Replace "European", and "concolic testing" by the most general feature you can find; and replace "ACM Fellow" by your most prestigious designation.  (Hint: In my case, 6 out of 7 reviews began with "The applicant is an ACM Fellow", as if this would disperse all doubts on my abilities; so go for such designations as you can.)
• Don't just say: "Best Paper Award".  But say: "First Best Paper Award for a Debugging Paper written on a one-legged stool".  Exercise: generalize as above.
• Don't just say "700 citations".  Also say: "Most cited testing paper since 1999".
• Avoid any claim that cannot be independently verified.
Coming up with such selling points is hard work; bibliographic query tools are your friends.  Again, reserve lots of time for this work.  (I spent two days googling and digging through the CVs of all European ACM Fellows, for instance; and a successful colleague of mine even has managed to get temporarily banned from Google Scholar.)  Selling yourself this way is hard; if you need to take a shower by the end of the day, that's fine.  But remember that every selling point you can come up with this way makes it harder for detractors to dismiss your achievements, and it makes it easier for champions to sell them to others.  In the end, it will have to be clear that you are the only person on earth who can save the world from this terrible, important problem.  

Your Project Plan

8. No risk, no fun.  The ERC funds high-risk, high-gain projects.  This means that there have to be substantial risks of failure (otherwise, others would have done this before).  However, your specific research plans should help to mitigate these risks and thus bring the high gains promised.  Focus on novelty (why is this new?) and potential impact (why is this needed?).  Avoid standard cliches from your discipline ("If only everybody had used this formal method from the start, the Ariane failure could have been prevented..."); come up with fresh, real stories and insights instead.
9. Clear title, clear abstract.  Think of the reviewer as an old, bored, nasty, uninterested, overloaded, overcommitted, latently aggressive, and totally uninterested ignorant doofus who hasn't gotten laid in a long time. (I'm a frequent reviewer, too, so I know what I speak about.) Even so, he or she should get interested in your proposal after a short glimpse of ten seconds.  The message has to be in the title, in the abstract, in the figures, in the diagram, in the examples.  (Yes, please have a diagram that conveys the approach! And please have an example, too!  All these are weapons in the hands of your champions.)  If you fear the message could be too complex, try again.  If you think the message sounds too trivial to you, it could start to be understandable for the rest of us.  (If, after simplification, your approach no longer sounds as cool as before, don't hide this with words, but go back to the drawing board.)
10. Have a clear structure and plan.  You're a seasoned researcher, so you know how to organize things, don't you?  Now all you need to do is to put this in writing: tasks, dependences, milestones, evaluations, and measurable success criteria.  The point of this exercise is not for the ERC to ask you to follow the plan by the letter once the project starts; the point of this exercise is for the reviewers to see that you can organize things.
11. Get to the point.  The length of an ERC proposals is clearly limited, and that's a good thing.  Get to the point quickly.  Use a clear language: No buzzwords, no yada yada, no lingo.  If your project on "Examining the security interoperability of cloud business process models" cannot be motivated in plain English, don't expect the computer science panel chair to pitch it against "Curing cancer once and for all".
12. Polish. Polish. Polish.   And polish again.  With an ERC grant, you're applying for the highest individual funding one can get in Europe.  Do your homework.

None of these tips guarantees success.  What they do, though, is to prevent misunderstandings.  If the reviewer does not get the point about you and your proposal, you will lose despite being great, and that sends you back to the drawing board.  If the reviewers do get the point about your project and your past achievements, though, then it's a fair game: If you are better than the others, you win; and if you are not, you lose.  Even if you're Brad Pitt, it's perfectly okay to lose against George Clooney.  If you win, though... well, that's great and totally worth it, as I can tell from first-hand experience :-)

What makes good research?

When joining my group, one of my PhD students got blamed by one of my colleagues: With her being best in class across all  theory courses, why would she "waste her talent" with a practitioner like me? Ah, the good old debates between theory and practice. As a researcher in software engineering, none of my work will ever be as elegant as what my colleagues in theory do, attacking the most complex problems with beautiful abstraction. But then, I know colleagues in theory who feel inferior to their more applied colleagues, because what these do actually has impact and interest in the real world.  

Obviously, different people have different ideas on what makes good research. But what are these categories?

Six years ago, I had a conversation with the founder of our faculty, Günter Hotz. Our faculty is well-known for its constant excellent hiring, based on ideas Hotz installed 40 years ago. So I asked him: How do you recognize good researchers? Or better: good research? To which he presented which I hereby name the Hotz model of good research:

It is actually fairly simple. There are three criteria, which are orthogonal. (He raised his hand, spreading off thumb, index, and middle finger at right angles to each other.) You ask the following questions:

First, was it hard? When you look at the research, is this something which required years of training and perspiration, or is this something anyone on the street could have come up with?

Second, is it elegant? Is this something which solves a specific problem, or is this something that you can apply over and over again to new problems?

Third, is it useful? Is this of value only to the academic community, or is this something people outside can use to create value or make money?

If you're good on one of the axes, you are doing good research.  If you're good on two axes, we'll hire you.  And if you're good on all three axes – well, I have yet to meet somebody who excelled in all three.

On this, he looked me in the eye and said:

Actually, now that I think of it, the "usefulness" axis is a fairly recent addition to the system.

There's a number of lessons to be learned from this. First, if your work was very hard, is very elegant, or very useful, you have every reason to be proud – and you deserve the respect from your fellow researchers, even if you don't share the same axes. In software engineering, usefulness is first and foremost, but truly hard or elegant work can still leave me highly impressed. If I get an idea of how it may be put to use, I feel struck by shock and awe.

Second, there is little chance you will be able to excel on all three axes. Usefulness calls for applicability in the real world, whose details will quickly spoil the elegance of your approach. Elegance implies simplicity, and simple is the antonym of hard – unless you claim it was real hard to get your approach simple. Making something really useful is hard, but was it a true intellectual challenge, or just a long engineering process? Difficulty, elegance, usefulness – pick any two.

Third, usefulness is still a recent criterion by academic standards, and as a new kid on the block, it may struggle to get accepted by traditional academics. But computer science has long left the academic ivory towers, and is progressing at a furious pace in the real world. As a young researcher, why limit your impact to dusty publications, or wait until the real world finally adopts your approach?

You have every right to choose the axis (or axes!) on which you plan to excel. Different people make different choices, but remember: The axes are orthogonal, and no axis dominates the others. The only important thing is to push the boundaries as far as you can. Face the hardest problems. Find the most elegant abstraction. Make it really useful. Push, push, push, with all love and respect. That's what makes good research.

August (and Christmas!) should be free of conference deadlines

The deadline for technical papers of this year's ICSE, the International Conference of Software Engineering, has been set to August 17 – a whopping 9 months before the conference takes place, and right in the middle of the busiest holiday month.  In most of Europe, schools and kindergartens are closed for holidays during that time, because it is naturally assumed that you'll be on holiday, too.  I can't even begin to speculate how such a deadline disrupts holidays and family time – and eventually harms the quality of submissions.

In the past ten years, the ICSE deadline has never been that early; except for ICSE 2011, it was always set in September.  For me, this means that half of the papers our group intended to submit for ICSE (essentially, all those which aren't complete next week) will now go to ICST instead.  Not only is the ICST deadline one month later and thus much more family-friendly, the conference is also two months before ICSE, implying a much quicker dissemination of results.

I suggest we as a software research community set up an implicit rule: No paper deadline in August.  If an August deadline is unavoidable, it could be used for abstracts (such that PC members can bid on the submissions), followed by a September deadline for the full papers.  If you agree, go and press the appropriate button.

[Update 2012-12-19: ECOOP 2013 ups the ante by extending their deadline to December 23.  Does the conference really expect researchers to drop their holiday preparations and write a paper instead?]

Are classroom lectures doomed?

I just finished recording an online course on software debugging at Udacity.  The format is superior to anything I could ever deliver in a classroom.  Universities should worry.


Udacity, a startup aiming to "democratize university-level education", offers free online courses on various subjects of higher-level education.  At this point, it is mostly computer science: artificial intelligence, programming languages, software testing – and soon: software debugging.  The Udacity format is pretty unique; what you see for the most time is my hand as it is writing, drawing, and doodling on an electronic sketchpad, while I explain and deliver the material.  Every two to three minutes, you get a quiz (which is checked automatically), and you can only proceed if you answered it correctly.

The nice thing about this course format is that it combines the best aspects of the classroom and the textbook experience.  When I was a student, I have sat through many hours of lectures where math professors delivered one proof after the other.  I usually lost track after 10-15 minutes, so what I got from the lecture was a mental note to work through the textbook proof later at my desk.  I always thought that this was just my own experience, but I recently found this was actually shared by many of my current PhD students (who happen to be among the best of their class, including math).  Simply videotaping such lectures and making them available online will not make them more attractive, and I never felt this to be a viable alternative.

With the Udacity format, this is very different: As I am demonstrating a proof, I can come up with a quiz for every step, thus making sure the student really understands what is going on.  The fast students will quickly answer the questions, while the slower students will be able to repeat the unit at will until they are ready.  In the future, one could even imagine additional explanations for students who keep on failing the quiz, or extra challenges for students who come up with quick correct answers.  The course would thus automatically adapt to the weaker as to the stronger students – which is way superior to having a "one-size-fits-noone" classroom lecture.

Yes, preparing an online lecture in this format is quite some work; it is like preparing a textbook and its presentation at the same time. But then, your lecture scales to an unlimited number of students.  And if the students find the online format more attractive than the classroom format, this will have far-reaching consequences for higher-level education.

One could hope that professors will have much more time for face-to-face interaction – in seminars and lab projects, for instance –, and for inspiring and judging the creativity of their students.  What I see, though, is that lectures as we know them may face the same fate as the CD, as the DVD, or as the encyclopedia – namely to be replaced by well-done digital offers.  That alone is not so much of a problem.  But think of the fate of record stores, video rentals, or soon book stores – and you may get an idea of what universities should be fearing.