Since I follow "Getting Things Done", I get more reminders, not less

As a professor, I have many responsibilities and tasks to take care of, and many of these have deadlines.  To keep track of them, I use an automated "To-Do"-Tracker, which allows me to enter new tasks (and ideas!) quickly, to assign categories and priorities to them, and to mark them as completed when I'm done.  Most importantly, I can attach a "due date" to every task, and tell my tracker to automatically have a task pop up a few days in advance – typically the number of days I will need to complete it.  So far, this works pretty well: When someone asks me whether I'd be available for a task, I can get a good assessment of my commitments (implying the answer is typically "no"); but if I commit to a task, it will almost certainly be done by its deadline.

Now, here's the fun thing.  Since I use this approach, I get way more reminders than before.  Journal editors tell me that my review is due in only five days.  PhD students send me worried letters that I haven't sent out the referral letter yet, due on Friday.  Conference chairs send me friendly reminders that I am an author, yet haven't registered for the conference, due in a week.  Thank you for the reminders – actually, I know all that, as my to-do tracker nicely tells me so, and it is all factored in. Today, I need to work on this paper, this book chapter, this slide deck – all stuff which gets better the earlier it can be reviewed; and I know that the task you remind me off can still wait until due day, or the N days it will take me to complete it.  In 2013, I completed 700+ tasks, and there was only one reminder which was effective (because I had mistyped the completion date).

Overall, I believe that this "just in time" production makes me more effective, as it puts things in the right order (and the right priority).  But then, it apparently strains the nerves of everyone else involved.  Should I set up a rule to complete everything a week before it is due?  Or should I set up a disclaimer of sorts?

(And now back to my "To-Do"-tool: Five days left to complete this paper.  Better start today...)

Post-Doc Position with Andreas Zeller at Saarland University

[Update 20-01-2014: The position is no longer available.  Thank you!  -- Andreas Zeller]

At the Software Engineering Chair at Saarland University (Andreas Zeller), we have a post-doc position available for initially two years, starting March 1, 2014 or later.  We are looking for applicants in all areas of Software Engineering, with a special focus on experience in

• Program analysis and testing
• Specification and specification mining
• Empirical software engineering
• Security and privacy

Our group currently consists of one professor, two post-docs, and ten PhD students, all working on how to increase programmer productivity and program quality. We have a record of substantial impact at international venues, are well funded, and are fun to work with. We cooperate with researchers around the world, including companies like Google, Microsoft, or SAP. Our students are constantly looking for research challenges in their BSc and MSc theses.

Saarbrücken is one of Europe's leading places in computer science research. It features Computer Science at Saarland University, two Max-Planck-Institutes for Informatics and Software Systems, the German Center for Artificial Intelligence, the Intel Visual Computing Institute, an Excellence Cluster in Multimodal Computing and Interaction, as well as the Saarbrücken Graduate School for Computer Science totaling more than 400 researchers. There are ample opportunities for local cooperation beyond the chair.

English is the working and teaching language all over computer science. Germans generally have reasonable command of English; and in day-to-day life, you can easily get along with English as your only language. (You may wish to learn the German food names, though, to avoid surprises.)

Mandatory conditions of employment are:

• PhD in Computer Science or Software Engineering
• Fluency in English

Full-time positions are divisible in principle (§ 7 Abs. 1 TzBfG).

Salary will be paid according to German TV-L.

The university aims to increase the number of women in this field. Therefore, women are especially encouraged to apply for this position.

Severely handicapped persons are preferentially employed in case of similar qualification.

The position is available as of March 1, 2014. You should apply before December 22, 2013, although later applications may also be considered.

Please send a detailed CV, including two referees, as a PDF document to:

Prof. Dr. Andreas Zeller
Saarland University – Computer Science
office@st.cs.uni-saarland.de
Tel.: +49 681 302-70970

Applications have to include reference no. W793.

Summarizing your presentation with miniature slides

The slide that drives me nuts

As part of my job, I listen to talks a lot.  There's good talks, and there's bad talks.  But nothing can drive me as crazy as making a slide like this your final slide:

What's wrong with showing "Thank you!" at the end?  Or "Questions?"?  Or both?  The problem is that most talks are followed by a discussion.  With a slide like this showing up at the end, there's three problems:

• Parts of your audience will have lost contact during your talk, and wake up for the final slide only.  With this showing up, they won't get the message.
• The idea of the discussion is to get valuable feedback.  But to ask questions, your audience will have to get the technical terms right.  If they don't remember the exact name of your technique, they won't ask questions for fear of embarrassment.
• If the discussion goes for five minutes, this is the single slide that shows up for the longest time.  This is such a missed opportunity for delivering your message!

The bullet summary style

A much better alternative is to simply say "Thank you" and to use the screen space for summarizing your talk.  In standard bullet style, the slide would look something like this:

Your audience can now use the slide to ask questions ("Your... errr... Bumblebee technique: If it is better for amblefuns, wouldn't it be worse for fumbleduns?"); people who have slept throughout your talk still get the message; and anyone who finds the discussion too special have all the time to enter the URL into the browser of choice to learn more about the project. Much better already.

Summarizing with miniature slides

The "bullet style" summary still has issues, though.  Many talks have diagrams that summarize processes or results.  These won't show up on a bullet style slide, and as these invariably attract the most questions ("Why is it that this column is so high?"), you frequently have to switch back to earlier slides, which (a) takes time and (b) again takes the summary of your talk away from the audience.

Here's my solution for the last slide.  Take the four most important slides of your talk (say: problem, process, results, and discussion), and paste them together on one single slide.  The result looks like this:

All these are slides that were shown earlier during the talk; hence, I can make it easy for my audience to recapitulate what I said.  During discussion, I can simply point to the appropriate detail on the slide, and I never have to switch back to an earlier slide.  Even if the text is hard to read at 50% of the original size, this does not matter much, as the audience will remember most from the talk anyway.  I have been using this style for eight years in a row now, and it has worked ever since.  Enjoy!

Update 1: How do I produce such miniatures?  On a Mac, the process is straightforward.  In Keynote or Powerpoint, select a slide and copy it to the clipboard.  In Preview, use Command-N to open the clipboard, and Command-C, which copies it back again, but now as a bitmap.  Go back to Keynote/Powerpoint, paste it in, resize and position appropriately.  That's all!

Update 2: Actually, it might have been me who invented this style of summary, or at least introduced it to the community.  On December 6, 2005, I gave my first talk on "Model Mining", using miniatures on my final slide, and I don't think having seen this before. Anyone with earlier "miniature" slides? Please step up!

Update 3: Here's my (or "the"?) first slide with miniatures from December 2005:

Faster, better, stronger: The case for one-phase reviews

When running a scientific conference, one has to decide which submitted papers get published (and presented), and which ones are not.  Each paper gets a number of reviews; and based on these, the program committee decides which papers get accepted.

Flooded with submissions, several SE conferences went for a two-phase model to reduce workload: A paper first gets two reviews, and only if at least one of these favors acceptance, the paper gets a third review.  As PC chairs of ASE 2013, we went back to a single-phase model, though.  Several people have asked us why this was so.  The answer is: time.  With a one-phase review, you can cut the time required for reviews in half, thus allowing for more, better, and more recent papers.


The math is as follows. Let's assume your conference gets 314 papers submitted (as we had for ASE):

• In a two-phase model,
•  314 papers would have gotten two reviews, totaling 618 reviews.
• 1/3 of papers are eliminated in the first round, leaving 210 which get a third review.
• Total number of reviews: 618 + 210 = 828.
• In a one-phase model,
•  314 papers get three reviews, totaling 942.

By having the one-phase model, your conference will thus have 14% more reviews – in our case, 22–23 papers to review per PC member instead of 20.  However, going for two phases requires a lot more administrative overhead – for the PC chairs, of course, but also for the reviewers.  In particular, you need to allocate two phases in which to do the reviews – instead of, say, eight weeks, where reviewers would be free to allocate their load, you now have six + four weeks, each of which may be more easily blocked by travel, holidays, etc.  Plus, as a PC chair, you need at least another week in between to reallocate and inform the new reviewers.

All in all, you thus trade 14% more reviews against an extension of the review period by three weeks.  Not necessarily a good thing.  In addition, this assumes that papers are all equal; but they’re not: The 14% extra papers are mostly papers that would be quick rejections anyway.  So, not much damage either.

Assuming that more than a third of papers may be rejected in the first round favors the two-phase model.  But then, there’s the concern that by having only two reviewers, you run a greater risk of one reviewer being incompetent, unwilling, lazy, etc.  (It happens to anyone.)  So you may be have to introduce rebuttals to compensate for potential misjudgments, and here go another two weeks of reviewing period.  And then, someone will have to read these rebuttals, which adds more overhead.

To illustrate how two-phase reviews and rebuttals eat up time in contrast to one-phase reviews, let's take a look at ICSE 2014, whose papers are due September 13, 2013.  Notification is January 17 – that is, more than four months later.   For ASE (one phase, no rebuttals), this was May 17 to July 25 – two months and one week, or roughly half the time ICSE takes.

With its short reviewing period, ASE 2013 had more than three times the number of submissions compared to last year, which gave us a great choice of papers.  ICSE will also have a record number of submissions; but if you miss the deadline, chances are you will find another conference which will publish your paper well before ICSE.

Summary: Get rid of frills, and allow authors six more weeks to prepare their submissions.  You will get more, better, and more recent papers – and more, better, independent reviews at the same time.

A Slide Generator for EasyChair

If you chair a program committee with a physical meeting and use EasyChair as a conference management system, then the EasyChair Slide Generator is your friend. This script

• generates slides structuring the discussion at the PC meeting
• generates personalized discussion lists for each PC member
• generates a conflict-free seating order for all PC members.

EasyChair Slide Generator comes as a Python script, tested on Mac and Linux systems. As input, it takes an EasyChair data export; see the included README.txt file for details.  Enjoy!

=>  EasyChair Slide Generator

My top ten presentation issues in other's papers

I am a frequent reviewer of submissions for scientific journals and conferences.  By far most of my reviews are specific to the individual paper, its contributions, and its issues.  However, there is a small number of presentation problems which I have to report again and again.  All these problem are very minor, and none of them will make me reject your paper.  However, fixing them is a matter of minutes, and will make your paper an easier read for both readers and reviewers:

1. Abstracts without results.  Many abstracts end with the words "We conducted a study to show the effectiveness of our approach".  There's three things wrong in here:
1. You conduct a study to evaluate your approach – if you run your study with a predetermined outcome, I will suspect a flaw in your setup.
2. As a reader of your abstract, I am not so much interested in your process, but in your results; if you want cliffhangers, go and write TV shows instead. 
3. You need results to sell your paper in the review process – in particular for outsiders who take only a brief glance at your paper.
So make this "Our study shows a 25% precision increase" or some other strong argument.
2. Multi-letter identifiers in LaTeX math mode.  In LaTeX, $foo$ stands for the product of three variables f, o, and, o; and LaTeX typesets it as such.  If you want one variable, use $\textit{foo}$.  (For a computer scientist, this is the single biggest design failure of TeX and LaTeX.)
3. Confuse hyphens, minus signs, and dashes.  All these typeset into different characters: 
1. Hyphens: state-1 is blue, state-2 is red.
2. Minus signs: $\textit{bar} = \textit{foo} - 1$
3. En-dashes (for ranges): A car has 3--4~wheels
4. Em-Dashes (for sentences): He said ``captain''---I said ``wot''.  (This can also be typeset using en-dashes with spaces, as in He said ``captain''~-- I said ``wot''.)
4. Bad capitalization in BibTeX references, as in "[1] Jai: a javascript api ide" (should be "[1] JAI: A JavaScript API IDE")  I could include this in all my reviews and be right 60% of the time.
5. Omitting reference information.  I see papers omitting page numbers, conference names, editors, or even authors.  I suppose that "[1] Smith et al., ICSE 2004" is something I should Google, and I hope there's just one Smith at ICSE.  (This even occurs in papers whose venues have no page limit for references!)
6. Orphaned numbers, as in "I want 17<newline>foos."  Tie numbers to items using non-breakable space (Word) or using a tilde (LaTeX): I want 17~foos.
7. Orphaned lines and titles.  Section title at end of page 3, actual section on page 4.  Better yet: a single line of text left under a table, such that you easily confuse content and caption.  "LaTeX gets this wrong" is not an excuse; please fix this.
8. Using monospacing with a non-monospaced font (or vice versa).  This seems to be a default setting of the LaTeX "listings" environment – using Times fonts and arranging the letters at equal distance.  Use monospaced fonts with monospacing, and proportional fonts otherwise.
9. Omitting articles, as in "Main feature is lack of articles".  Now that spell checkers weed out the most blatant errors, the more subtle errors remain – and this is my #1 grammar issue.  If your native language has no articles, take double care.
10. Testing the limits.  Page limits are there for a reason – not to save trees, but to save the time of your readers by requiring you to focus on the main points.  Cramming every little detail into the paper will decrease readability and alienate your readers; it may also lead to desk rejection if you alter the format.  Stick to standard style, cut down your paper to one page less than the limit and have your paper proof read by others, then add whatever your proof readers missed.

All these issues can be fixed through simple proofreading, and leave your readers and reviewers more time to enjoy the actual content.  And now that the syntax is done, let's get back to semantics!

(As a reviewer or reader, what are your favorite gripes?  Use the commenting function below.)

We are creating a Start-up in Web testing

Today, my chair has obtained a 500.000 EUR research transfer grant ("EXIST-Forschungstransfer") for funding a start-up in the area of Web application testing.  This project has been running mostly undercover for the past two years, and I'm more than happy to see it kick off now.

The key product is WEBMATE – a service for systematic testing of Web applications.  The scenario is that you go to our website, enter the URL of the site you want to get tested, and WEBMATE will automatically and systematically explore all the functionality on the site.  In its initial inception, it will do so in parallel on multiple browsers and operating systems ("cross-browser testing"), and automatically detect if something works in one browser, but not on the other.  WEBMATE is very good at exploring functionality, be it in Javascript or server-side, and thus easily can find errors.  Plus,  you can have it run as often as you like, for instance as part of a paid subscription.  A service like WEBMATE also forms a great base for regression testing or security testing, and we're thrilled by the many market and research opportunities.

Right now, the EXIST funding secures the WEBMATE team such that they can prepare for building and monetizing the service.  The team consists of three scientists (Martin Burger, Valentin Dallmeier, and Michael Mirold) as well as an experienced IT entrepreneur (Bernd Pohl).  In the past months, these four have spent countless hours interacting with potential customers, business angels, and venture capitalists.  (If you've seen me in a suit in the past months, that typically was due to one of these activities.)  At this point, let's simply say that both the research and financial prospects of automated system test generation are very promising.

From a personal perspective, I will act as mentor and shareholder of our start-up.  At the chair, we will use WEBMATE for our own research purposes, though; and if our new techniques work well, we  now have a process to bring them to the public (and earn money at the same time).  And sure, it feels good to be a professor and an entrepreneur at the same time :-)

To learn more about WEBMATE, see us at CeBIT (Hall 5, D04 for our start up, and Hall 9, F34 for our research), visit our website, or read our papers.  If you're a journalist, check out our press release.