tag:blogger.com,1999:blog-87470859029025108372024-03-13T18:25:57.587+01:00Andreas Zeller's Old BlogOld blog of Andreas Zeller. All posts are now at
https://andreas-zeller.github.ioAndreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.comBlogger32125tag:blogger.com,1999:blog-8747085902902510837.post-51559162210254319752019-10-17T18:07:00.003+02:002019-10-17T18:25:40.786+02:00When Results Are All That Matters: Consequencesby Andreas Zeller and Sascha Just; with Kai Greshake<br />
<br />
<h2>The Case</h2>
In our previous post "<a href="https://andreas-zeller.blogspot.com/2019/10/when-results-are-all-that-matters-case.html" target="_blank">When Results Are All That Matters: The Case of the Angora</a> <a href="https://andreas-zeller.blogspot.com/2019/10/when-results-are-all-that-matters-case.html" target="_blank">Fuzzer</a>", we reported our findings when investigating the Angora fuzzer [1]. If you have not read that post yet, you should stop here and read our <a href="https://andreas-zeller.blogspot.com/2019/10/when-results-are-all-that-matters-case.html" target="_blank">write-up</a> first. There, we focus on our findings and problems that surprised us when experimenting with Angora.
<br />
In this article, we have collected some suggestions to advance the field of fuzzing and have a long-term impact on the reliability of software.<br />
<br />
<h2>1. Science is about insights, not products.</h2>
To ensure scientific progress, we need to know which technique works, how, and under which circumstances. We write papers to document such insights such that the next generation of researchers, as well as the non-scientific world, can build on them. The value of a paper comes from the impact of its insights.<br />
<br />
<h2>2. Scientists and companies <i>can</i> create tools.</h2>
It is fun to build a tool, and if it works well, the better. Typically, this will involve not one single magical technique, but a multitude of techniques working together. Tools will have to succeed on the market, though, and will be evaluated not based on their insights, but their effectiveness.<br />
<br />
Evaluating tools for their effectiveness can be part of a scientific approach. However, evaluation settings should<br />
<ol>
<li>be <i>fair</i> and thus not be defined by tool authors; and</li>
<li>avoid <i>overspecialization</i> and thus involve tests not previously known to tool authors.</li>
</ol>
In other words, the only way to obtain reliable performance comparisons is by independent assessment. Other communities do this through specific tool contests that operate on secret benchmarks created for this very purpose. And of course, tools need to be available for evaluation in the first place. It is nice to see the security community to adapt such techniques, such as artifact evaluation.<br />
<br />
<h2>3. Combinations of techniques must be assessed individually.</h2>
If results depend on a larger set of novel processing steps, the contribution of each must be – for instance, by replacing each processing step by a naive approach and assessing the impact of the change. All decisions affecting performance must be well motivated and documented.<br />
<br />
Without assessing the impact of each step individually, one can still have a great tool, but the insight on what makes it great will be very limited. As an analogy: We know that Usain Bolt is a record shattering sprinter; the scientific insight is to find out why.<br />
<br />
<h2>4. Document your hypotheses, experiments, and results.</h2>
Good scientific practice mandates that experiments and their results be carefully documented. This helps others (but also yourself!) in assessing and understanding the decisions in the course of your project. If you make some design decision, such as a parameter setting, after examining how your software runs on some example, it is important that the motivation for this design decision can be traced back to the experiment and its result.<br />
<br />
If this sounds like lot of work, that's because it is. We're talking about the scientific method, not some fiddling around with parameters until we reach the desired result on a benchmark. Fortunately, there are great means to help you with these tasks. Jupyter Notebooks [8], for instance, allow you to collect your hypotheses (in natural language), your experiment design, its results (in beautiful and interactive graphs, among others), and your next refinement step – allowing anyone (as well as yourself) to understand how a specific result came to be. Be sure to place your notebooks (and code) under version control from day one, and throw in some tests and assertions for quality assurance. Control your environment carefully to make results reproducible for anyone.
<br />
<br />
<h2>5. Having benchmarks to compare tools and approaches is helpful, but brings risks.</h2>
Benchmarks are helpful means to assess the performance of tools. However, they bring two risks.<br />
<ol>
<li>First, there is the risk of having researchers <i>focus on the benchmark</i> rather than insights. It is nice to have a well-performing tool, but its scientific value comes from the insights that make its performance.</li>
<li>Second, benchmarks bring the risk of researchers knowingly or unknowingly optimizing their tools for this very benchmark. We have seen this with compilers, databases, mobile phones, fault localization, machine learning, and now fuzzing. To mitigate the risk of overspecialization, tool performance should be compared on <i>programs they have not seen before.</i></li>
</ol>
A benchmark like LAVA-M is representative for detecting buffer overflows during input processing but very little else. As the LAVA creators state themselves, "LAVA currently injects only buffer overflows into programs" and "A significant chunk of future work for LAVA involves making the generated corpora look more like the bugs that are found in real programs." [3].<br />
<br />
It has been shown that optimizing against the artificial LAVA bugs, such as 4-byte string triggers, can have very naive approaches yield impressive results [2]. The conceptual match between the features injected by LAVA and those features exploited by fuzzers such as Angora is striking.<br />
<br />
The question of what makes a good benchmark for fuzzers and test generation is still open. One possible alternative to LAVA-M is Google's fuzzing test suite which contains a diverse set of programs with real bugs [5]. Michael Hicks has compiled excellent guidelines on how to evaluate fuzzers [4, 6].
<br />
<br />
<h2>6. Researchers must resist the temptation of optimizing their tools towards a specific benchmark.</h2>
While developing an approach, it is only natural to try it out on some examples to assess its performance, such that results may guide further refinement. The risk of such guidance, however, is that development may result in overspecialization – i.e., an approach that works well on a benchmark, but not on other programs. As a result, one will get a paper without impact and a tool that nobody uses.<br />
<br />
Every choice during implementation has to be questioned "Will this solve a general problem that goes way beyond my example?", and one should take that choice only with a positive, well-motivated answer, possibly involving other experts who would be asked in the abstract. We recommend that during implementation, only a very small set of examples should be used for guidance; the evaluation should later be run on the full benchmark.<br />
<br />
Good scientific practice mandates to create a research and evaluation plan with a clear hypothesis well before the evaluation, and possibly even before the implementation. This helps to avoid being too biased towards one's own approach. Note that the point of the evaluation is not to show that an approach works, but to precisely identify the circumstances under which it works and the circumstances under which it does not work.<br />
<br />
Papers should investigate those situations and clearly report them. Again, papers are about insights, not competition.<br />
<br />
<h2>7. It is nice to have tools discovering vulnerabilities...</h2>
...especially as these vulnerabilities have a value on their own. However, vulnerabilities do not follow statistical distribution rules (hint: otherwise it would be easier to find them). Having a tool find a number of vulnerabilities in a program, therefore, is not necessarily a good predictor to find bugs in another program.<br />
<br />
In any case, the process through which vulnerabilities were found must be carefully documented and made fully reproducible; for random-driven approaches such as fuzzers, one thus needs to log and report random seeds. Obviously, one must be clear not to optimize tools towards given vulnerabilities.<br />
<br />
For fuzzing tools, the technical challenge is to find inputs that cover a wide range of behavior across the program and not only during input processing and error handling. Let us remind you that during testing, executing a location is a necessary condition for finding a bug in that very location. Since we are still far from reaching satisfying results in covering functionality, improvements in code coverage are important achievements regardless of bugs being found.<br />
<br />
<h2>8. What does this mean for reviewers and authors?</h2>
Papers must clearly show how the insights of the paper contribute to the result, both in terms of motivation as well as in evaluation.<br />
<br />
In many cases, it will be hard to describe all the details of all the necessary steps in the paper. Therefore, it will be necessary to supply an artifact that allows for not only reproduction but also applying it on subjects not seen before; again, all design decisions in the code must be motivated and documented. This is tedious; this is rigorous; this is how science works.<br />
<br />
Reviewers should be aware that an approach is not simply "better" because it performs well on a benchmark or because it found new bugs. Approaches have a long-term impact not only through performance, but also through innovation, generality, and simplicity. Researchers are selected as reviewers because the community trusts them to assess such qualities. Tool performance that is achieved through whatever means has little scientific value.<br />
<br />
Having said that, conference organizers should create forums for tool builders and tool users to discuss lessons learned. Such exchanges can be extremely fruitful for scientific progress, even if they may not be subject to rigorous scientific assessment. Tool contests with clear and fair rules would allow assessing the benefits and fallbacks of current approaches, and again to foster and guide discussions on where the field should be going. A contest like Rode0day [7] could serve as a starting point.<br />
<br />
<h2>Conclusion</h2>
Having tools is good, and having tools that solve problems is even better. As scientists, however, we also must understand <i>why</i> <i>what</i> works and what does not. As tools and vulnerabilities come and go, it is these insights that have the longest impact. Our papers, our code and our processes, therefore, must all be shaped to produce, enable, assess, and welcome such insights. This is the long-term path of how we as scientists can help to make software more reliable and more secure.<br />
<br />
<i><b>Acknowledgments.</b> Marcel Böhme, Cas Cremers, Thorsten Holz, Mathias Payer, and Ben Stock provided helpful feedback on earlier revisions of this post. Thanks a lot!</i>
<br />
<br />
<h2>References</h2>
[1] P. Chen and H. Chen, "<a href="https://ieeexplore.ieee.org/document/8418633" target="_blank">Angora: Efficient Fuzzing by Principled Search</a>." 2018 IEEE Symposium on Security and Privacy (IEEE S&P), San Francisco, CA, 2018, pp. 711-725.<br />
[2] <a href="http://moyix.blogspot.com/2018/03/of-bugs-and-baselines.html" target="_blank">Of bugs and baselines</a><br />
[3] B. Dolan-Gavitt et al., "<a href="https://ieeexplore.ieee.org/document/7546498" target="_blank">LAVA: Large-Scale Automated Vulnerability Addition</a>," 2016 IEEE Symposium on Security and Privacy (S&P), San Jose, CA, 2016, pp. 110-121.<br />
[4] George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. 2018. <a href="https://arxiv.org/pdf/1808.09700.pdf" target="_blank">Evaluating Fuzz Testing</a>. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS '18). ACM, New York, NY, USA, 2123-2138.<br />
[5] <a href="https://github.com/google/fuzzer-test-suite" target="_blank">Google's fuzzing test suite</a><br />
[6] Michael Hicks, <a href="http://www.pl-enthusiast.net/2018/08/23/evaluating-empirical-evaluations-for-fuzz-testing/" target="_blank">Evaluating Empirical Evaluations (for Fuzz Testing</a><br />
[7] <a href="https://rode0day.mit.edu/" target="_blank">Rode0day</a><br />
[8] <a href="https://jupyter.org/" target="_blank">Project Jupyter</a><br />
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-6667563117306432392019-10-10T12:17:00.000+02:002019-10-17T18:11:41.291+02:00When Results Are All That Matters: The Case of the Angora Fuzzer<em>by Andreas Zeller and Sascha Just; with Kai Greshake</em>
<br />
<h2>
The Case</h2>
"Fuzzers" are programs that generate random inputs to trigger failures in tested programs. They are easily deployed and have found numerous vulnerabilities in various programs. The last decade has seen a tremendous increase in fuzzing research [4].
<br />
<br />
In 2018, Chen and Chen published the paper "Angora: Efficient Fuzzing by Principled Search" [1] featuring a new gray box, mutation-based fuzzer called Angora. Angora presented extraordinary results in its effectiveness and shines with its stellar performance on the Lava-M benchmark [3], dramatically outperforming all competition.
<br />
<br />
The reason behind this breakthrough and leap in effectiveness towards deeper penetration of target programs was cited as the combination of four techniques: scalable byte-level taint tracking, context-sensitive branch count, input length exploration, and search based on gradient descent. The former three techniques had already been explored in earlier work; but the novel key contribution, as also indicated by the paper title, was the <em>Gradient Descent</em> approach to solve constraints modeling branch conditions as functions and approximating their gradient.
<br />
<br />
One of our students was intrigued by the outstanding performance and wanted to investigate why and how optimization strategies affect the fuzzing performance that much - with the initial goal of actually further improving on Angora. To properly measure the effect of his work, he went and worked directly on the Angora source code. During his study [2], he came across some confusing findings, which we summarize below.
<br />
<br />
<h2>
The Findings</h2>
<h3>
Angora's Gradient Descent performs similar to Random Search on LAVA-M</h3>
Contrary to what is stated in the Angora paper, we could not find real differences between using a random search algorithm and using Gradient descent to solve constraints. After further investigation, it turns out that in almost all optimization cycles, the constraint is solved immediately when the first input is executed. That first input is generated by using magic byte extraction – and only when it is replaced by a random starting point does the performance of Random Search drop off. Since 32-bit magic byte values comprise all of the bugs artificially injected into the LAVA benchmark, it does not surprise that this feature dominates the performance instead of Gradient Descent.<br />
<div>
<br />
Of course, this raises the question of why Angora is so successful. Magic byte extraction is not a new technique, but in the case of this benchmark, in combination with taint tracking, good heuristics, and other improvements it leads to spectacular results on LAVA-M. However, Gradient Descent contributed little to that success.</div>
<div>
<br />
These findings are confirmed by Angora's authors [5]: "If the fuzzer can extract the magic bytes and copy them to the <em>correct</em> location in the input, then obviously the fuzzer can solve the constraint immediately." They point out that Gradient Descent will show and did show advantages outside of solving magic byte constraints, even if this is not their evaluation setting. And indeed, our findings do not imply that Gradient Descent would not work; on the contrary, we still find it a highly promising idea. However, whether and when it can make a difference is an open question.
<br />
<br />
<h3>
The Angora Evaluation Methodology is Inadequate</h3>
In the Angora paper, the authors claimed to have investigated the efficacy of each contribution individually and proved the effectiveness of Gradient Descent. Unfortunately, the comparison they present to support this claim is biased towards Gradient Descent. They tried to compare the constraint-solving capabilities of random search, random search with magic byte extraction and Gradient descent. They collected an input corpus using AFL and then started Angora with each algorithm and the same input corpus. This way, all algorithms start with the same set of constraints to solve and the authors reported the corresponding solve rate for each algorithm. </div>
<div>
<br />
Since AFL uses random mutations to solve constraints, it is not surprising that all constraints that can be easily solved with random mutations have been already solved in the input corpus. This gives random search an immediate disadvantage. </div>
<div>
<br />
While inspecting the code of Angora, we also found another issue. The evaluation ([1, Section 5.4]) states that Gradient Descent is compared to random search with Magic Byte extraction. However, the Angora Gradient Descent implementation itself makes use of Magic Byte Extraction, thus guaranteeing that it always performs <em>at least as</em> well as its closest competitor in the comparison. This methodology is inadequate and the original conclusions about the efficacy of Gradient Descent are not supported by our research. </div>
<div>
<br />
The Angora authors state that their "exploitation routines are a part of the enhancements aforementioned. During gradient descent, Angora first descends by the entire gradient. Only when this fails to solve the constraint does Angora try to descend by each partial derivative as a crude attempt of salvation." [5] According to our findings, however, these heuristics dominate the search of Gradient Descent for constraints with a large number of input dimensions – a fact that should have been discovered in an adequate evaluation setting.
<br />
<br />
<h3>
Odd Heuristics and Parameters</h3>
The Angora code contains a number of optimizations and behaviors that are not documented in the paper. While the volume of information that can be published in a paper is limited, many of these had a significant impact on the potential performance; yet, they are not documented or motivated in the released source code. These are some examples:
<br />
<ul>
<li>
The <a href="https://github.com/AngoraFuzzer/Angora/blob/master/fuzzer/src/search/gd.rs" title="https://github.com/AngoraFuzzer/Angora/blob/master/fuzzer/src/search/gd.rs">Gradient Descent implementation</a> goes beyond what is described in the paper. It contains special exploitation routines that inject fixed values that are known to trigger overflows and numerical bugs. The algorithm also does not only use the entire gradient but instead descends by each partial derivative.
</li>
<li>
Parameters like the optimization budget are set to fixed, arbitrary values without justification or documentation for the <a href="https://github.com/AngoraFuzzer/Angora/blob/master/common/src/config.rs" title="https://github.com/AngoraFuzzer/Angora/blob/master/common/src/config.rs">chosen parameters</a>:
<pre style="background-color: rgba(220, 220, 220, 0.4); border-bottom-left-radius: 3px; border-bottom-right-radius: 3px; border-top-left-radius: 3px; border-top-right-radius: 3px; font-size: 14px; font-variant-ligatures: normal; orphans: 2; overflow: auto; padding: 16px; white-space: pre-wrap; widows: 2;"><code class="code-line language-Rust code-line language-Rust" data-line="89" style="color: var(--vscode-editor-foreground); font-family: Menlo, Monaco, Consolas, "Droid Sans Mono", "Courier New", monospace, "Droid Sans Fallback"; font-size: 1em; line-height: 1.357em; position: relative; tab-size: 4;"><span class="hljs-comment" style="color: green; font-style: italic;">// SEARCH</span>
<span class="hljs-keyword" style="color: blue;">pub</span> <span class="hljs-keyword" style="color: blue;">const</span> ENABLE_DET_MUTATION: <span class="hljs-built_in" style="color: blue;">bool</span> = <span class="hljs-literal" style="color: #a31515;">true</span>;
<span class="hljs-keyword" style="color: blue;">pub</span> <span class="hljs-keyword" style="color: blue;">const</span> MAX_SEARCH_EXEC_NUM: <span class="hljs-built_in" style="color: blue;">usize</span> = <span class="hljs-number" style="color: #b8d7a3;">376</span>;
<span class="hljs-keyword" style="color: blue;">pub</span> <span class="hljs-keyword" style="color: blue;">const</span> MAX_EXPLOIT_EXEC_NUM: <span class="hljs-built_in" style="color: blue;">usize</span> = <span class="hljs-number" style="color: #b8d7a3;">66</span>;
<span class="hljs-keyword" style="color: blue;">pub</span> <span class="hljs-keyword" style="color: blue;">const</span> MAX_NUM_MINIMAL_OPTIMA_ROUND: <span class="hljs-built_in" style="color: blue;">usize</span> = <span class="hljs-number" style="color: #b8d7a3;">8</span>;
<span class="hljs-keyword" style="color: blue;">pub</span> <span class="hljs-keyword" style="color: blue;">const</span> MAX_RANDOM_SAMPLE_NUM: <span class="hljs-built_in" style="color: blue;">usize</span> = <span class="hljs-number" style="color: #b8d7a3;">10</span>;
<span class="hljs-keyword" style="color: blue;">pub</span> <span class="hljs-keyword" style="color: blue;">const</span> GD_MOMENTUM_BETA: <span class="hljs-built_in" style="color: blue;">f64</span> = <span class="hljs-number" style="color: #b8d7a3;">0.0</span>;
<span class="hljs-keyword" style="color: blue;">pub</span> <span class="hljs-keyword" style="color: blue;">const</span> GD_ESCAPE_RATIO: <span class="hljs-built_in" style="color: blue;">f64</span> = <span class="hljs-number" style="color: #b8d7a3;">1.0</span>;
<span class="hljs-keyword" style="color: blue;">pub</span> <span class="hljs-keyword" style="color: blue;">const</span> BONUS_EXEC_NUM: <span class="hljs-built_in" style="color: blue;">usize</span> = <span class="hljs-number" style="color: #b8d7a3;">66</span>;
</code></pre>
<br />
Values like <code style="color: var(--vscode-textPreformat-foreground); font-family: Menlo, Monaco, Consolas, "Droid Sans Mono", "Courier New", monospace, "Droid Sans Fallback"; font-size: 1em; line-height: 1.357em;">MAX_SEARCH_EXEC_NUM</code> (376; the number of iterations to solve a constraint) or <code style="color: var(--vscode-textPreformat-foreground); font-family: Menlo, Monaco, Consolas, "Droid Sans Mono", "Courier New", monospace, "Droid Sans Fallback"; font-size: 1em; line-height: 1.357em;">BONUS_EXEC_NUM</code> (66; the number of additional iterations added under certain conditions) would be odd choices in an experiment design; computer scientists would normally prefer multiples of powers of 10, 2, or 5. These choices are not justified or documented; and in any case, the impact of these choices (compared to others) would have to be determined.
<br />
The value of 376 for <code style="color: var(--vscode-textPreformat-foreground); font-family: Menlo, Monaco, Consolas, "Droid Sans Mono", "Courier New", monospace, "Droid Sans Fallback"; font-size: 1em; line-height: 1.357em;">MAX_SEARCH_EXEC_NUM</code> is a particularly odd choice. Since the few constraints that Gradient Descent operated on were actually either solved after a maximum of 50 iterations or not at all, setting the iteration budget to at most 50 should actually increase the performance on LAVA-M.</li>
</ul>
The Angora authors confirm that the "current values seem to work well on most programs, but we did not rigorously evaluate them or search for their optimal values." [5] This implies that the authors themselves do not know which factors contribute to Angora's evaluation performance, or why their choices would be particularly well suited to settings outside of the evaluation.
<br />
<br />
<h3>
Reproducibility</h3>
All of the above findings refer to the <em>published Angora code</em>, which may differ from the one used in the paper evaluation. We have therefore asked the Angora authors to provide us with the version used for the experiments described in the paper. They said that they "have been steadily maintaining and improving the Angora repository over the past year, with numerous bug fixes and enhancements. Its current performance is similar to, if not better than, the version on which our paper's evaluation was based." Unfortunately, the history of the public repository does not go back to the time the paper was published, and our request for the original version remains unfulfilled. We do not know whether the Angora authors would be able to reproduce their published results.
<br />
<h2>
Summary</h2>
The performance and effectiveness of the Angora tool are uncontested, notably on the LAVA-M benchmark. If you want to fuzz a program that is part of the LAVA-M benchmark or very similar, Angora is likely to give you good results. Finally, the authors are to be commended for making their code available, and for providing detailed and timely answers to our questions.
However,
<br />
<ol>
<li>Angora's novel Gradient Descent approach has little to no impact on its performance on LAVA-M;</li>
<li>The performance of Angora is impacted by several decisions that are not documented, motivated, assessed, or evaluated individually or sufficiently; and</li>
<li>The evaluation methodology is inadequate to support the paper's conclusions on Angora's performance.</li>
</ol>
Guidelines for good scientific practice mandate that all information and decisions that helped to achieve a specific result be well documented. This is not the case here – neither in the paper nor in the code.</div>
<div>
<br />
It would be a mistake, however, to point at the Angora authors only. Chen and Chen <em>did</em> make their code available, allowing for independent assessment. Several recent fuzzing papers use similar techniques and benchmarks as Angora, <em>yet only few make their code available</em>. Can we trust their results?</div>
<div>
<br />
This calls for the community to discuss and question its procedures and criteria on what makes good research. Reviewers and researchers must not only focus on great results, but also assess how these results have been obtained, whether they have been rigorously evaluated, and how they translate into insights that will stand the test of time. How this can be achieved for programs with thousands of lines of undocumented code is a pressing question.</div>
<div>
<br />
For now, the Angora case tells us how much we do <em>not</em> know, and how much further insights into what works and what does not will be needed. The good news is that this opens lots of opportunities for discussion – and future research.<br />
<br />
<br />
<i><b>Acknowledgments.</b> Marcel Böhme, Cas Cremers, Thorsten Holz, Mathias Payer, and Ben Stock provided helpful feedback on earlier revisions of this post. Thanks a lot!</i><br />
<i><br /></i>
<i>If you liked this post, also see our <a href="https://andreas-zeller.blogspot.com/2019/10/when-results-are-all-that-matters.html" target="_blank">follow-up post</a> with consequences.</i><br />
<i><br /></i>
<i>You can contact Kai Greshake as <a href="mailto:development@kai-greshake.de">development@kai-greshake.de</a>.</i><br />
<i><br /></i>
<br />
<h2>
References</h2>
<ul>
<li>[1] P. Chen and H. Chen, "<a href="https://ieeexplore.ieee.org/document/8418633" title="https://ieeexplore.ieee.org/document/8418633">Angora: Efficient Fuzzing by Principled Search</a>." 2018 IEEE Symposium on Security and Privacy (IEEE S&P), San Francisco, CA, 2018, pp. 711-725.</li>
<li>[2] K. Greshake, “<a href="https://www.dropbox.com/s/43ovybqfiugen1w/thesis_greshake.pdf?dl=0" title="https://www.dropbox.com/s/43ovybqfiugen1w/thesis_greshake.pdf?dl=0">Effective Search Algorithms for Grey Box Fuzzing</a>” BSc. Thesis at Saarland University, August 2019.</li>
<li>[3] B. Dolan-Gavitt et al., "<a href="https://ieeexplore.ieee.org/document/7546498" title="https://ieeexplore.ieee.org/document/7546498">LAVA: Large-Scale Automated Vulnerability Addition</a>," 2016 IEEE Symposium on Security and Privacy (IEEE S&P), San Jose, CA, 2016, pp. 110-121.</li>
<li>[4] Valentin J.M. Manes, HyungSeok Han, Choongwoo Han, Sang Kil Cha, Manuel Egele, Edward J. Schwartz, Maverick Woo, "<a href="https://arxiv.org/abs/1812.00140" title="https://arxiv.org/abs/1812.00140">The Art, Science, and Engineering of Fuzzing: A Survey</a>". arXiv:1812.00140</li>
<li>[5] P. Chen, H. Chen, A. Zeller. "<a href="https://www.dropbox.com/s/qku87fw3fmwdw95/angora_exchange.pdf?dl=0" title="https://www.dropbox.com/s/qku87fw3fmwdw95/angora_exchange.pdf?dl=0">Questions on Angora tool vs Angora paper</a>". E-mail exchange, September/October 2019.</li>
</ul>
<br />
<br />
<br />
<br />
<br /></div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com1tag:blogger.com,1999:blog-8747085902902510837.post-61894055405494895892018-02-01T12:47:00.002+01:002018-02-01T12:53:19.724+01:00Where your conference fees go to<div class="separator" style="clear: both; text-align: center;">
</div>
<a href="https://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><a href="https://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a>Fees for scientific conferences can be a cause for concern. Since most researchers are paid by public money, the question is whether this money is put well to use. So where do your conference fees go? Let me illustrate this on an example – the <a href="https://issta2016.cispa.saarland/" target="_blank">ISSTA conference I organized in 2016</a>.<br />
<br />
ISSTA is a <b>nonprofit event</b>, run by dozens of volunteers who run the conference as part of their paid job (or throw in extra hours for fun and honor); none of the researchers presenting, reviewing, or organizing make any profit from this. ISSTA is run by ACM, which means that it covers losses but also gets any surplus; again, ACM is a nonprofit organization.<br />
<br />
ISSTA 2016 attracted 113 visitors to its main conference, and 48 visitors to its workshops. As with most conferences in our domain, students make the majority of visitors to the main conference, followed by ACM members. (As an ACM member, you get a discount that is higher than the annual ACM membership fee, so it pays off to join ACM even if just for one year.) <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxfQo8_gm-3WRZjackY8AriKvK1BFQP7s7UZBPwQjE6GSzkpjYm_ffBpERnoETvNFkNZCmoz_afsNEpL1W4-rRpTSxBtfKsHUyb7RIKfZWkwQiBMUj9UgaoYUA_R6AKIfPn14JBsm3S2BA/s1600/Attendance.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="781" data-original-width="780" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhxfQo8_gm-3WRZjackY8AriKvK1BFQP7s7UZBPwQjE6GSzkpjYm_ffBpERnoETvNFkNZCmoz_afsNEpL1W4-rRpTSxBtfKsHUyb7RIKfZWkwQiBMUj9UgaoYUA_R6AKIfPn14JBsm3S2BA/s400/Attendance.png" width="398" /></a></div>
<br />
<br />
<a href="https://issta2016.cispa.saarland/registration/" target="_blank">Registration fees</a> for the main conference vary between 300€ for students registering early and 750€ for non-members registering late; workshops fees were between 125€ and 300€. On top of these fees comes a VAT of 19%. These attendees make the conference, and their fees make the conference income – in the case of ISSTA 2016, exactly <b>70,460€.</b><br />
<br />
When setting up the budget, you make <b>non-students subsidize students</b>: At ISSTA 2016, an additional student would actually incur a small cost; but this would easily be covered by additional non-students coming in (especially when showing up on-site!). We also got a small amount of donations; other conferences and conference chairs are much better at getting these.<br />
<br />
Of course, you do not know how many people will attend, so you calculate conservatively – I estimated 100+ visitors for the main conference, and planned accordingly. The key point in planning, though, is the <b>expenses</b>. This is what the expenses for ISSTA 2016 eventually broke down to:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWUCSv5_-GUy8mUx_c3PDhOtMmPb1pkLonxqitk90wsDhmDLi5oO-2pHDWHq5Zvt2elaOOTxTHaHajAtC423rVAhIYogf4h-W0Mx3ImekG6pxwWh7CNsugdwv4B4UDY2kiY8sQ6HtiY-sE/s1600/Expenses.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="711" data-original-width="745" height="381" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgWUCSv5_-GUy8mUx_c3PDhOtMmPb1pkLonxqitk90wsDhmDLi5oO-2pHDWHq5Zvt2elaOOTxTHaHajAtC423rVAhIYogf4h-W0Mx3ImekG6pxwWh7CNsugdwv4B4UDY2kiY8sQ6HtiY-sE/s400/Expenses.png" width="400" /></a></div>
<br />
<br />
Starting clockwise at the top, the biggest category is <b>food and drinks</b>, including social events like reception and dinner. We spent about 50€ per person and day, which is pretty reasonable for two full meals, coffee, drinks, snacks, and all. We ran ISSTA at my university, meaning that we could buy drinks and snacks at retail price; if you run a conference at a hotel, you would be charged hotel prices, say $3 or more for each and every can of soft drink, and you don't want to know about beer or wine. <b>Workshops</b> at ISSTA 2016 had relatively high fees, but little cost (essentially coffee, cookies, and a light lunch), and thus brought in revenue. All expenses also incur <b>taxes</b> – in our case, again 19% VAT.<br />
<br />
The next big block is <b>registration,</b> which covers most of actually organizing and running the conference. Most of this is the fee charged by the conference organization agency of my university. They run a few dozen conferences every year and are very well-organized: their fee includes organizing costs (the folks who answer your queries and write visa letters), university overhead, and more. This item also includes the <b>staff</b> on site – in our case, student aides who gave out bags, served drinks, handled your queries, or oversaw the AV – as well as free <b>bus tickets</b> for all attendees (rides from hotel to the conference venue and back). Overall, I estimate that the agency saved me two months full time of planning and organizing, so contracting them was a great return on investment.<br />
<br />
<b>Financial</b><i style="font-weight: bold;"> </i>refers to 2% credit card fees. Depending on your country and your financial provider, these may vary considerably.<br />
<br />
<a href="https://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><a href="https://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a>ISSTA features a <b>physical meeting of the program committee</b> – that is, all reviewers meet for two days to discuss which papers should be presented. This contributes to the quality of the conference, but also is very expensive. The reviewers have to assume their travel, and the conference pays for meeting rooms, day catering, and a dinner. We co-located the PC meeting with the ICST 2016 conference in Chicago. This attracted more attendees to ICST; in return, ICST subsidized the PC meeting from the extra revenue, keeping the cost low. This item also includes <b>invited speakers</b>, for whom ISSTA 2016 covered travel expenses as well as free conference and workshop tickets.<br />
<br />
<b>On-site logistics </b>primarily refers to <b>meeting rooms. </b>These can be expensive, depending on the venue. My university has reasonable prices for scientific meetings, including all technical equipment. Hotels often offer meeting rooms for free if you bring in sufficiently many guests, but then they may charge you for extras such as AV or Wi-Fi – and the rental of a single AV system can easily exceed the cost of an entire meeting room at the university.<br />
<br />
The <b>program</b> needs to be printed; for <b>publicity</b>, we distributed leaflets at conferences and otherwise used free Facebook and Twitter channels. Other conferences would throw in paid advertising and throw in extras such as the proceedings on a USB stick. ISSTA gave you a linen bag and an online access code for the proceedings.<br />
<br />
<b>Conference management</b> refers to services around the conference, notably <b>Proceedings</b>, the cost of editing the accepted papers and making sure they all end up in the digital library, all properly tagged; as well as the <b>Conference Management System</b>, which handles submissions and the review process. We used specialized services for both, again at reasonable cost.<br />
<br />
Low cost and high attendance, especially at workshops, gave ISSTA 2016 a high <b>surplus of 25,000€ </b>– which all went to ACM such that ACM and SIGSOFT could keep on organizing and supporting research in our field. Note, though, that <b>such a surplus is not the rule</b>. You are supposed to plan for some contingency and overhead, but most conferences have a much smaller surplus. Despite best efforts of conference organizers, some conferences come up with a loss that would then be covered by the sponsoring society. With ISSTA 2016, we were lucky – and if you ever go and organize a conference, I wish you the same luck, too!<br />
<br />Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-3056402127364552062018-01-25T09:54:00.000+01:002018-12-21T12:16:42.264+01:00I am leaving Saarland University, and the call for my successor is openThis year, I will leave Saarland University, where I have been professor for software engineering for 16 years. Our <a href="https://cispa.saarland/" target="_blank">CISPA Center for IT Security</a> is about to segregate from Saarland University to become a Helmholtz Center, and I will be moving with it.<br />
<br />
With base funding for 500 researchers, the new Helmholtz Center is <a href="https://www.saarland.de/231505.htm" target="_blank">set to become one of the largest research institutions in IT security</a> – and hopefully one of the most renowned, too. The recent research of my group in program analysis and test generation (watch out for papers this year!) positions us right at the intersection of software engineering and security, and I will be happy to contribute both to the research agenda as to the management of CISPA. I will retain a footprint at Saarland University, though, keeping my professor title and reduced teaching obligations.<br />
<br />
While the label on my office will change, my loyalty to the <a href="https://saarland-informatics-campus.de/en/" target="_blank">Saarland Informatics Campus</a> will not falter the slightest. I am grateful to work in one of the greatest research environments that could possibly exist, surrounded by colleagues who have proven their incredible excellence again and again, and who all work together every day for the excellence of the site as a whole. When I started 16 years ago, we were maybe 15 professors in Computer Science; when I retire, there will be ten times as many.<br />
<br />
This great environment can be your environment, too. If your work is related to IT security, please check out our <a href="https://cispa.saarland/career/positions/" target="_blank">tenure-track and tenured career options at CISPA.</a> If your work is in Software Engineering, though, the call for my successor (<a href="https://www.uni-saarland.de/fileadmin/user_upload/verwaltung/stellen/wissenschaftler/W1314_W3-Professur_f%C3%BCr_Informatik_im_Bereich_Softwaretechnik.pdf" target="_blank">German</a> / <a href="https://www.uni-saarland.de/fileadmin/user_upload/verwaltung/stellen/wissenschaftler/W1314_W3_for_Computer_Science_in_the_area_of_Software_Technology.pdf" target="_blank">English</a>) is open as well. It has been one of the best positions I could ever think of, and I am sure it could be the same for you!Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com1tag:blogger.com,1999:blog-8747085902902510837.post-23789912802300590442017-11-02T17:50:00.000+01:002017-11-02T17:50:31.057+01:00The Rejection Song<i style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px;">To be performed by a <b>scientific program committee (choir) </b>and its <b>chair (solo voice)</b> to the tunes of the song "<a href="https://www.youtube.com/watch?v=Uhn6KduyJHw" target="_blank">Go West</a>" (Village People / Pet Shop Boys). A program committee decides which scientific works get published.</i><br />
<i style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px;"><br /></i>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We‘re the committee</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We‘re the experts, see?</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We will read your work</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We will make us heard</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal; min-height: 14px;">
<br /></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! This is our song</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! cos the data‘s wrong</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! it’s been done before</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! send it out the door</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal; min-height: 14px;">
<br /></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We select the best</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We reject the rest</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) and the best is us</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) so why make a fuzz?</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal; min-height: 14px;">
<br /></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! This is out of scope</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! it’s a slipp‘ry slope</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! It’s irrelevant</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! I don’t understand!</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal; min-height: 14px;">
<br /></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We define the field</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We do form a shield</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) We are the elite</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
(Together) Life can be so sweet</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal; min-height: 14px;">
<br /></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! no related work</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! you are such a dork</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! not my expertise</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! would you fix that, please</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal; min-height: 14px;">
<br /></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! cos it’s really bad</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! cos I’m getting mad</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! there’s a missing bit</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
Re-ject! it‘s a piece of (fade out)</div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
<br /></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
<br /></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
<i>Music: Victor Willis, Henri Belolo and Jacques Morali</i></div>
<div style="color: #454545; font-family: "Helvetica Neue"; font-size: 12px; font-stretch: normal; line-height: normal;">
<i>Above lyrics: Andreas Zeller</i></div>
<div>
<br /></div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-1230574714156107282017-04-26T13:38:00.002+02:002017-04-27T15:49:17.089+02:00Mit dem Fahrrad aus der Saarbrücker Innenstadt zur UniIch bin die meisten Tage des Jahres mit dem Fahrrad aus der Saarbrücker Innenstadt zur Uni und zurück unterwegs. Für alle, die auch gerne mit dem Fahrrad zur Uni wollen, habe ich hier meine drei Lieblingswege auf <a href="https://www.google.com/maps/d/viewer?mid=1CYa2Y8apVZ2wux96pILq7_ze9ZA&ll=49.239832188951986%2C7.0244841534423585&z=14" target="_blank">einer Google-Karte</a> zusammengestellt; Details folgen unten.<br />
<br />
<iframe height="520" src="https://www.google.com/maps/d/u/0/embed?mid=1CYa2Y8apVZ2wux96pILq7_ze9ZA" width="520"></iframe><br />
<br />
Und hier sind die drei Wege, sortiert von Nord nach Süd:
<br />
<br />
<h2>
Der Schnelle: Am Meerwiesertalweg entlang</h2>
<div>
<i>Ein nützlicher Weg, getrennt geführt neben starkem Autoverkehr.</i><br />
<br />
<b>Hier geht es lang: </b>Vom Hauptbahnhof aus über den Bormannpfad (Alternative: durch Parkhaus Hela schieben), die Dudweilerstraße kreuzen und auf den Meerwiesertalweg. Dort führt ein für Radfahrer freigegebener Radweg immer geradeaus mit zunächst moderater Steigung bis hoch zur Uni. Auf dem letzten Stück zwischen Uni-Parkhaus und Pforte wird es steil.</div>
<div>
<br /></div>
<div>
<b>Ein Weg für: </b>Normalfahrer, die schnell von A nach B wollen.<br />
<br />
<b>Aufpassen: </b>Am unteren Ende des Meerwiesertalwegs sind zahlreiche Einmündungen, wo Autofahrer insbesondere von den stadteinwärts auf der linken Seite fahrenden Radfahrern überrascht werden; erst ab der Jugendherberge wird es entspannt. Der Weg am Meerwiesertalweg ist ein für Radfahrer freigegebener Gehweg, also gilt es, besondere Rücksicht auf die (wenigen) Fußgänger zu nehmen. (Radfahrer dürfen zwar auch auf der Fahrbahn des Meerwiesertalwegs fahren, was aber wegen starken Autoverkehrs auf enger Fahrbahn keinen Spaß macht.)</div>
<div>
<br />
<b>Bester Moment:</b> Wenn es auf dem Rückweg am Parkhaus entlang steil herunter geht – das ist der Rücksturz zur Erde.</div>
<div>
<br /></div>
<div>
<b>Alternativen:</b> Innenstadt über Scheidter Straße verlassen (siehe unten), dann über den Ilseplatz und Waldhausweg in den Meerwiesertalweg umschwenken. Wer die Steigung am Parkhaus nicht mag, kann auch das Gelände der Sporthochschule erkunden.<br />
<br /></div>
<h2>
Der Sportliche: Durch den Stadtwald zur Uni</h2>
<div>
<i>Mein morgendlicher Weg zur Uni – steil und schön</i><br />
<br />
<b>Hier geht es lang: </b>Die Innenstadt über Beethovenstraße/Blumenstraße verlassen (eine ruhige Strecke entlang Einbahnstraßen, die laut Verkehrsentwicklungsplan irgendwann zur Fahrradstraße ausgebaut werden soll), auf die Scheidter Straße wechseln und bis zu deren Ende immer höher fahren. Hinter der Wendeschleife geht es in den Wald, wo wir sofort links an der Schranke vorbei auf einen asphaltierten Waldweg wechseln, der uns bis zur Uni führt. Der Weg führt steil durch den lauschigen Stadtwald hinauf. Etwa auf halbem Wege geht es dann bergab, man rollt entspannt zur Uni herab, und lässt den Schweiß im kühlenden Fahrtwind zurück.<br />
<br />
<b>Ein Weg für: </b>Mountainbiker, Elektroräder, Naturliebhaber.<br />
<br /></div>
<b>Aufpassen:</b><b style="font-weight: normal;"> </b>Der Radweg entlang der Scheidter Straße ist durch parkende Autos von der Fahrbahn getrennt; ausfahrende und parkende Autofahrer übersehen hier Radfahrer gerne. Der Waldweg ist unbeleuchtet; Schnee wird spät geräumt. Auf dem Rückweg geht es auf der Scheidter Straße steil bergab, also besser auf der Fahrbahn bleiben.<br />
<br />
<b>Bester Moment:</b> Morgens im Wald den Berg hinauf. Nichts zu hören außer Vogelgezwitscher.
<br />
<br />
<div>
<b>Anschlüsse:</b> Noch höher hinaus? Auf den Schwarzenbergturm für ein kleines Workout und oben die Aussicht genießen.<br />
<br /></div>
<div>
<h2>
Der Flache: Über Schafbrücke und Scheidt</h2>
</div>
<div>
<i>Feierabendstrecke mit nur moderaten Steigungen, die Hälfte auf ruhigen Wegen.</i></div>
<div>
<i><br /></i></div>
<div>
<b>Hier geht es lang: </b>Die Innenstadt an der Saar entlang verlassen; spätestens an der Ostspange nach links abbiegen und auf die B51 nach rechts fahren. Auf dem Radweg geht es an der B51 eher unproblematisch immer geradeaus. Nach den Römerkastell geht es auf einer Fahrradspur leicht bergauf; danach geht es nach links in die Breslauer Straße, und dann gleich rechts auf die andere Seite der Bahnstrecke. Jetzt kommt der schöne Teil: Es geht auf ruhigen Wegen durch Schafbrücke immer an der Bahn entlang, bis wir in Scheidt nach der Überführung links abbiegen und wieder mit leichter Steigung zur Einfahrt Ost der Uni fahren.</div>
<div>
<br /></div>
<div>
<b>Ein Weg für:</b> Entspannte Kilometerfresser. Leute, die zum Ostteil der Uni wollen.</div>
<div>
<br /></div>
<div>
<b>Aufpassen:</b> In der Innenstadt sind entlang der Mainzer Straße wie auch an der Saar viele Fußgänger unterwegs, also Rücksicht nehmen. Der Abschnitt Römerkastell-Breslauer Straße ist verkehrsreich; beim Linksabbiegen in die Breslauer Straße auf eine Rotphase warten, dann den Extra-Wartebereich für Fahrradfahrer nutzen, um sich links einzuordnen. Von Scheidt zur Uni führt der Radweg getrennt auf der linken Seite entlang; Vorsicht bei Einmündungen.<br />
<br />
<div>
<b>Bester Moment:</b> Auf ruhigen Wegen flott durch Schafbrücke.</div>
</div>
<div>
<br /></div>
<div>
<div>
<b>Alternativen: </b>Zwischen Römerkastell und City sind Preußenstraße und/oder Halbergstraße verkehrsarme Fahrrad-Alternativen zur B51. Wer im Saar-Basar einkaufen mag, kann das Gelände im Westen über eine Pforte via Eschberger Weg und Im Heimerswald erreichen. Statt des asphaltierten Radweges von Scheidt zur Uni kann man auch am Rückhaltebecken links abbiegen und einen unbefestigten Waldweg nehmen.<br />
<br />
<b>Anschlüsse:</b> Sehr schöne Radwege gibt es die Saar entlang Richtung Saarlouis oder Sarregemuines. Der Radweg nach Scheidt führt auf ruhigen Wegen weiter an der Bahn entlang Richtung St. Ingbert. Alle Radwege sind ausgeschildert.</div>
</div>
<div>
<br />
<div>
<b>Hinweis:</b> Als Rückweg (also von Uni zur Innenstadt) attraktiver, da (a) leichtes Gefälle (b) am Römerkastell weniger Kreuzungen und (c) der verkehrsreiche Abschnitt zwischen Breslauer Straße und Römerkastell ist schnell überwunden. </div>
<div>
<br /></div>
</div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-25398586213260159542017-01-13T16:05:00.002+01:002017-01-24T09:38:02.196+01:00Twelve LaTeX packages to get your paper accepted<i>(with </i><i>Abhik Roychoydhury and Aditya Kanade)</i><br />
<br />
Why do some people get all their papers accepted, and others do not? You may already know that in many disciplines, using the LaTeX typesetting system correlates with having your paper accepted (in contrast to, say, Word). What you may not know is that there is a number of LaTeX packages whose usage may be crucial for success. Here we go:<br />
<ol>
<li><b>The <i>pagefit</i> package.</b> This immensely useful package makes your paper exactly fit within a given page limit, applying a genetic search algorithm to reduce baseline distances, white space, font sizes, or bibliographic references until it exactly fits. Just write <span style="font-family: "courier new" , "courier" , monospace;">\usepackage[pages=12,includingbibliography]{pagefit}</span> and enjoy. </li>
<li><b>The <i>autocite</i> package. </b>Cites all relevant work that needs to be cited. The "<span style="font-family: "courier new" , "courier" , monospace;">citepc</span>" option additionally cites the entire program committee, whether their work is relevant or not.</li>
<li><b>The <i>translate</i> package. </b>Auto-translates your paper into a given target language (default is English). Just type<a name='more'></a><span style="font-family: "courier new" , "courier" , monospace;"> \begin{translate}Endlich kann ich in meiner Muttersprache schreiben!\end{translate}</span> to obtain "Finally, I can write in my mother language!" (Hint: You can also translate <i>English into English</i> to fix typos and other mistakes.)</li>
<li><b>The <i>significance </i>package. </b>Alters your experiment settings until results become statistically significant, repurposing LaTeX's built-in formatting algorithm for advanced p-hacking. Use as <span style="font-family: "courier new" , "courier" , monospace;">\usepackage[p=0.05]{significance}</span>.</li>
<li><b>The <i>boast</i> package.</b> This extends the <i>nlp</i> package to automatically alter your writing style according to a set of parameters. For instance,
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace;">\set\relevance=\Large </span>% Set relevance (values range from <span style="font-family: "courier new" , "courier" , monospace;">\tiny </span>to <span style="font-family: "courier new" , "courier" , monospace;">\Huge</span>)</li>
<li><span style="font-family: "courier new" , "courier" , monospace;">\set\novelty=0.5 </span>% Sets novelty claimed, from 0.0 to 1.0. </li>
<li><span style="font-family: "courier new" , "courier" , monospace;">\set\formality=0.75</span> % Increase or decrease formal content (formulas, theorems, greek letters, etc.). For the humanities, use lower values.</li>
</ul>
Hint: If you get a LaTeX "overclaim" warning, reduce these values; you can also use the <span style="font-family: Courier New, Courier, monospace;">[maximize]</span> option to have LaTeX find a maximum without overclaim. Also, be sure to reduce <span style="font-family: "courier new" , "courier" , monospace;">\relevance </span>for sections that discuss related work.</li>
<li><b>The <i>accept</i> package.</b> This package does what it says: All our published papers have a line that says <span style="font-family: "courier new" , "courier" , monospace;">\usepackage{accept}</span>. If you do not have the <i>accept</i> package, at least try to comment out the <span style="font-family: "courier new" , "courier" , monospace;">\usepackage{reject}</span> line found in so many journal submission templates.</li>
<li><b>The <i>coauthors</i> package. </b>Automatically searches for co-authors who have done well-respected work related to the paper and includes them as co-authors to boost chances of acceptance. The option <span style="font-family: "courier new" , "courier" , monospace;">[silent=true]</span> (default) does so without their knowledge. Use <span style="font-family: "courier new" , "courier" , monospace;">[related=false] </span>to include any tall figure (say, Paul Erdős) as co-author.</li>
<li><b>The <i>prostrate</i> package.</b> Puts an acknowledgement section at the end profusely thanking the reviewers, even if the reviews were not even close to being relevant, helpful, insightful, or constructive. After all, you should always thank the reviewers for accepting your paper that you yourself would not have accepted! Automatically expands to fit the page limit; see the <i>pagefit </i>package, above.</li>
<li><b>The </b><b style="font-style: italic;">autosubmit package.</b><i> </i>Run your paper through LaTeX and have it automatically submitted to the most suitable venue. Use <span style="font-family: "courier new" , "courier" , monospace;">[field=physics] </span>to narrow down the field or <span style="font-family: "courier new" , "courier" , monospace;">[conference=ICSE] </span>to explicitly specify the venue. Be careful: These options also accept <i>wildcards</i> – <span style="font-family: "courier new" , "courier" , monospace;">[field=humanities,journal=*]</span> will auto-submit your paper to <i>all</i> humanity journals at once. (See also the <i>autoreject</i> package.)</li>
<li><b>The <i>award</i> package.</b> Makes your paper win an award, as in <span style="font-family: "courier new" , "courier" , monospace;">\usepackage[bestpaper]{award}</span>. Options include "<span style="font-family: "courier new" , "courier" , monospace;">impact</span>", "<span style="font-family: "courier new" , "courier" , monospace;">beststudentpaper</span>", and more; be aware that "<span style="font-family: "courier new" , "courier" , monospace;">bestpaper</span>" and "<span style="font-family: "courier new" , "courier" , monospace;">impact</span>" are mutually exclusive. Donald Knuth and Leslie Lamport have extended this package with a "<span style="font-family: "courier new" , "courier" , monospace;">turing</span>" award option, but never publicly released their extension; Philip Roth is said to have asked for a "<span style="font-family: "courier new" , "courier" , monospace;">nobel</span>" option.</li>
<li><b>The <i>trump</i> package.</b> Makes your paper great again, but shortens it to 140 characters, dismissing all scientific evidence. So overrated!</li>
<li><b>The <i>tenure</i> package.</b> Keep on writing to obtain this package. If you can also get a <i>relocation</i> package, a <i>healthcare</i> package, and a <i>retirement</i> package, you'll be all set!</li>
</ol>
<div>
Note that these packages are normally customized towards your institution (using institution-specific relevance and boast settings, for instance). Therefore, you will not find these packages as part of your standard LaTeX distribution; use your institutional download site instead. Keep on LaTeXing!</div>
<div>
<br /></div>
<div>
<i>Coming up next: "More and more schools prohibit the infamous 'autowrite' package", and "How to use the 'politically' package in the correct way."</i><br />
<br /></div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com9tag:blogger.com,1999:blog-8747085902902510837.post-85202582057779967252016-10-14T12:20:00.000+02:002016-10-14T12:20:12.193+02:00PostDoc Position, Software Testing and Analysis / Software EngineeringAt <a href="https://www.st.cs.uni-saarland.de/" target="_blank">my Chair at Saarland University</a>, we currently have a PostDoc position available, to be filled during the Spring of 2017. We are looking for applicants in all areas of Software Engineering, with a special focus on experience in<br />
<div>
<ul>
<li>Software Testing and Analysis</li>
<li>Security and Privacy</li>
<li>Specification and Specification Mining</li>
<li>Data Mining and Machine Learning</li>
</ul>
<div>
Your job is to conduct bold and risky research, possibly interacting with the several great PhD students at the chair. Our most recent projects include</div>
<div>
<ul>
<li><a href="https://www.st.cs.uni-saarland.de/models/autogram/" target="_blank">mining of input grammars to create powerful fuzz testers</a>,</li>
<li><a href="https://www.st.cs.uni-saarland.de/appmining/backstage/" target="_blank">mining thousands of apps to detect UI inconsistencies</a>, and </li>
<li><a href="http://www.boxmate.org/" target="_blank">mining sandboxes as a universal security solution</a>.</li>
</ul>
</div>
<div>
If such work inspires you, and if you see chances to combine your expertise and creativity with ours, we'll be happy to hear from you. Please have a look at the <a href="http://www.uni-saarland.de/fileadmin/user_upload/Campus/Service/Stellenausschreibung/Wissenschaftliches_Personal/W1153_engl.pdf" target="_blank">official announcement</a> and apply before December 1.</div>
</div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com1tag:blogger.com,1999:blog-8747085902902510837.post-26958499865518262382016-09-07T06:43:00.000+02:002016-09-07T06:46:58.332+02:00On Facts and Dreams in Software Research<br />
<div class="p1">
<i><b>Where is the economic argument in program verification research, and where are the salvation messages in software engineering research?</b></i></div>
<div class="p1">
<i><b><br /></b></i></div>
<div class="p1">
I just had the pleasure of attending <a href="http://www.comp.nus.edu.sg/~david/" target="_blank">David Rosenblum’s</a> keynote at <a href="http://www.ase2016.org/" target="_blank">ASE 2016</a>, in which he praised the power of probabilistic thinking (using stochastic reasoning to estimate risks and support decisions) versus an absolutistic view in which there is only 100% truth or 100% failure. His takeaway message was that software engineers should embrace probabilistic methods and permeate them throughout the development process.</div>
<div class="p2">
<br /></div>
<div class="p1">
As a software engineer, probabilistic thinking is a central part of my day-to-day job – in development as in research. Whatever I build and design, I think about how useful it might be, which benefits it may bring, and what the associated risks are. In Software Engineering research, <a href="http://andreas-zeller.blogspot.sg/2012/10/what-makes-good-research.html" target="_blank">the dominating metric is usefulness</a>, which eventually translates into an <i>economic</i> argument: What does it cost? What are the benefits? What are the risks? These are day-to-day questions in software development, and my research is expected to provide facts to help answer them.</div>
<div class="p2">
<br /></div>
<div class="p1">
In other fields of computer science, such as software verification, the economic perspective is much less argued about. Instead, the message touted is an absolutistic message of <i>salvation:</i> If you apply this technique, you will get a 100% guarantee that your software satisfies certain requirements – that the computation will be correct, that it will not crash, or that it will terminate within specific time bounds. For developers and society, this message feels like the second coming of Christ: The day will come and all your problems will be gone. </div>
<div class="p2">
<br /></div>
<div class="p1">
Up to now, there are great examples of how software verification works, and there are impressive examples of it being successfully used in practice; and just to be clear: By no means would I want to reduce these research efforts. But the systems formal verification is applied to are still small and constrained; and getting it to scale and widen will require enormous human effort reshaping existing systems – before salvation comes repent. Do we actually know how costly formally verified software is? Do we know how to teach its techniques to developers who do not hold a PhD? Can we estimate the risks it brings to rely on freshly written specifications, rather than on mature systems? Might "good enough" software be good enough? When talking to researchers in software verification, such economic arguments are frequently missing in the debate. But as a society, we need to understand where money is best spent, and answers to such questions could very well guide the field of software research.</div>
<div class="p2">
<br /></div>
<div class="p1">
</div>
<div class="p1">
Conversely, just as formal verification may benefit from introducing an economic perspective, Software Engineering may profit from adapting a stronger <i>salvation</i> perspective. What is it that Software Engineering research could produce that would be seen as a salvation in software development – a significant reduction of costs or risks? Recent topics that come to my mind are <a href="http://automated-program-repair.org/" target="_blank">automatic software debugging and repair</a>, steering development processes based on <a href="http://www.msrconf.org/" target="_blank">mining software histories</a>, or massive <a href="http://www.evosuite.org/" target="_blank">automated test generation</a>. Can we translate our capabilities into <i>grand challenges</i> that will provide salvation in the future, possibly even including guarantees? Yes, I know there are “<a href="https://en.wikipedia.org/wiki/No_Silver_Bullet" target="_blank">no silver bullets</a>” in software development. But any great research community should pursue great dreams as well as provide facts – and in this, all of us computer science researchers are in the same boat.</div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com2tag:blogger.com,1999:blog-8747085902902510837.post-2995560829606582082016-06-28T10:48:00.002+02:002016-06-30T08:50:58.799+02:00Spiegel Online nutzt unsichere Cäsar-Chiffre: So lassen sich Spiegel Plus-Artikel lesen, ohne zu bezahlenDer Spiegel-Verlag wagt einen neuen Versuch, mit Online-Journalismus Geld zu verdienen: Mit "<a href="http://www.spiegel.de/spiegelplus/" target="_blank">Spiegel Plus</a>" werden einzelne Artikel aus dem Online- und Magazin-Angebot gegen Geld angeboten. Ich finde das aus mehreren Gründen gut:<br />
<ul>
<li>Guter Journalismus will bezahlt werden</li>
<li>Ich zahle lieber für Artikel, als dass ich mit Werbung zugefüllt werde; und </li>
<li>Ich bin sowieso Spiegel-Abonnent und kann somit ohnehin auf die Artikel zugreifen.</li>
</ul>
Die Art und Weise, wie Spiegel Online seine Paywall implementiert hat, ist allerdings noch verbesserungsbedürftig: Jeder Depp mit elementaren Programmier- und Verschlüsselungskenntnissen (= ich) kann die Sperre bequem umgehen, und mit Verfahren, die sich in jedem Kindersachbuch zum Thema Verschlüsselung finden lassen:<br />
<br />
1. Man nehme einen Mac und surfe mit dem Safari-Browser die gewünschte Seite an – etwa <a href="http://www.spiegel.de/politik/ausland/donald-trump-hotels-mitarbeiter-werfen-trump-ausbeutung-vor-a-1099404.html" target="_blank">diese</a>. Es folgt die Zahlungsaufforderung. (Dummerweise gibt es keine Möglichkeit für Abonnenten wie mich, sich einzuloggen – was die nächsten Schritte motivierte.)<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjatjo-vAHciLyURPt9ibeDYy8oAVRrOh0zfethPTmOdrxx4baQa6jOSHIg9rrZDmbQSS6Reskfgw1-E9cV4nZZP8Xqi4aQ6H4sw1ehwwosCGePpamt87w6fEsuoMIbvJjpJlnOQvsDQbXj/s1600/Screen+Shot+2016-06-28+at+10.17.52.png" imageanchor="1"><img border="0" height="409" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjatjo-vAHciLyURPt9ibeDYy8oAVRrOh0zfethPTmOdrxx4baQa6jOSHIg9rrZDmbQSS6Reskfgw1-E9cV4nZZP8Xqi4aQ6H4sw1ehwwosCGePpamt87w6fEsuoMIbvJjpJlnOQvsDQbXj/s640/Screen+Shot+2016-06-28+at+10.17.52.png" width="640" /></a><br />
2. Man schalte den Browser in den Lese-Modus (Shift-Command-R). Jetzt kommt der gesamte Artikel – allerdings ist der zu bezahlende Teil verschlüsselt.<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijxKd7g-Nq5nROQFB2Ibq2yz4KYggImh9bd_hL9X5cVJMlJlRugWyWoCjb5ZhZKg25pTiS08dDlOPfhOuGaYdjf1pzbiB-TF1l6eQSy4PUp301aJLptOe2Zhch-4ZVzoDtNEYS6Z_TKqcL/s1600/Screen+Shot+2016-06-28+at+10.20.06.png" imageanchor="1"><img border="0" height="409" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEijxKd7g-Nq5nROQFB2Ibq2yz4KYggImh9bd_hL9X5cVJMlJlRugWyWoCjb5ZhZKg25pTiS08dDlOPfhOuGaYdjf1pzbiB-TF1l6eQSy4PUp301aJLptOe2Zhch-4ZVzoDtNEYS6Z_TKqcL/s640/Screen+Shot+2016-06-28+at+10.20.06.png" width="640" /></a><br />
3. Offensichtlich werden hier einzelne Zeichen durch andere Zeichen ersetzt – Leerzeichen und Absatzzwischenräume sind nach wie vor erkennbar. Gleich zwei Absätze beginnen mit "Jo" – offensichtlich ein häufiges Wort. Nehme ich jeweils den vorangegangenen Buchstaben, erhalte ich hier "In". Gleichermaßen wird "Tfju" zu "Seit", und "Ebt" zu "Das". Die Entwickler haben den Text mit einer <i><a href="https://de.wikipedia.org/wiki/Caesar-Verschl%C3%BCsselung" target="_blank">Cäsar-Chiffre</a></i> verschlüsselt – jeder Buchstabe wird durch seinen Nachfolger ersetzt. Das ist das <i>einfachste</i> und <i>unsicherste</i> Verfahren der Verschlüsselung – und eine Beleidigung für jeden Sicherheitsexperten.<br />
<br />
4. Wir markieren den verschlüsselten Text und kopieren ihn in die Zwischenablage (Command-C). Dann surfen wir zu einer Seite, die die Cäsar-Verschlüsselung entschlüsselt – etwa <a href="https://gc.de/gc/caesar/" target="_blank">hier</a>. Dort fügen wir den kopierten Text ins untere Feld, geben "1" als Verschiebung ein, und drücken auf "Decode" – und schon erscheint oben der entschlüsselte Artikeltext. Umlaute und "z" sind zwar noch kaputt, aber dies sei dem geneigten Programmierer zur Übung überlassen.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyacanO_mW1TK-rnUDVhkqFS9EqAKMm354bRKz9RotsWVlKa-RkXy2TzMKG_1M2-_SS-FNIDD4Y5K25y4nIfw1WZScoEyJ1hC3NlV129mlI3jL0U_GBEeas1Zwt-n1BH7pbe6NQ15Hp2-_/s1600/Screen+Shot+2016-06-28+at+10.31.07.png" imageanchor="1"><img border="0" height="459" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyacanO_mW1TK-rnUDVhkqFS9EqAKMm354bRKz9RotsWVlKa-RkXy2TzMKG_1M2-_SS-FNIDD4Y5K25y4nIfw1WZScoEyJ1hC3NlV129mlI3jL0U_GBEeas1Zwt-n1BH7pbe6NQ15Hp2-_/s640/Screen+Shot+2016-06-28+at+10.31.07.png" width="640" /></a><br />
Danke, lieber Spiegel, dass Du Dich seit Jahrzehnten an die Speerspitze des deutschen investigativen Journalismus stellst. Jetzt musst Du nur noch das Schlusslicht in Sachen Verschlüsselung abgeben, und Du hast meinen vollen Respekt. Kleiner Tipp: Unser <a href="http://www.cispa.saarland/" target="_blank">CISPA-Institut</a> gibt gerne Hinweise zu sicheren Verschlüsselungs- und Zahlungsverfahren :-)<br />
<br />
<i>Update vom 29.06.2016: Matthias Streitz von der Spiegel-Chefredaktion hat sich bei mir gemeldet – er dankt für den Hinweis und will prüfen, ob ihnen "noch etwas Schlaueres einfällt als die Cäsar-Verschiebung". Ich bin optimistisch, dass ihnen das gelingen wird.</i><br />
<i><br /></i>
<i>Update vom 20.06.2016: Die "niedrige Paywall" ist auch <a href="https://twitter.com/kantorkel/status/747419175321276416" target="_blank">von anderen</a> beobachtet wurden.</i>Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-58717015520164868142016-05-23T18:49:00.002+02:002018-04-02T14:37:45.349+02:00My first talk at a scientific conference: A complete and utter disasterI am just returning from ICSE 2016 in Austin, Texas; and once more, I have been impressed by the great many research talks. For many of the presenters, this might have been there very first talk, and I was happy to see that it went pretty well for everyone. <i>My</i> very first presentation at a scientific conference did not go so well. Actually, it was a complete disaster. But it eventually made me a professor.<br />
<br />
This happened in 1993, at the German Software Engineering "SE" Conference in Dortmund. I was a PhD student in my second year, and my advisor, Gregor Snelting, had asked me to present a summary of our group's research. At this time, we had worked on <i>semantics-based retrieval of software components:</i> You would enter the desired pre- and postconditions of your component in a search field, and the system would automatically retrieve a suitable component. The novelty was that we were using <i>theorem provers</i> to find possibly weaker preconditions or possibly stronger postconditions. At the time, theorem provers were just about to enter the programming language space, so this was all new and unheard of.<br />
<br />
So, here I was, standing on the stage, in front of about 100 German-speaking SE researchers, presenting the NORA system. (NORA stood for No Real Acronym.) And I would be showing how to search for a function given a simple postcondition – something like<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: center;">
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGlUZizoklQQXqpCFYqujQ0fXuIoUt3dUh75iaOiR57TP12SsK0LIPdBcFTbUohIqRJXHUcOmARF7938ML3PGQZvGdg5tAnbeOGvsBebzM4AfrGI5MNZygZlKU9lgUgRlW6xmTSiFI9Mrk/s1600/Untitled.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="31" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGlUZizoklQQXqpCFYqujQ0fXuIoUt3dUh75iaOiR57TP12SsK0LIPdBcFTbUohIqRJXHUcOmARF7938ML3PGQZvGdg5tAnbeOGvsBebzM4AfrGI5MNZygZlKU9lgUgRlW6xmTSiFI9Mrk/s400/Untitled.jpg" width="400" /></a></div>
<br /></div>
This, as the specialist will immediately recognize, is the postcondition for a <i>sorting</i> function: The output a'[] is sorted (1), and also is a permutation of the input a[] (2). I think I was right in the middle of explaining the formula, when I saw that some member of the audience had stood up, looking right at me. Then, with a defiant look on his face. he shouted:<br />
<br />
<div style="text-align: center;">
"When <i>I</i> want to search for a sorting function, I do <i>grep sort!"</i></div>
<div style="text-align: center;">
<i><br /></i></div>
<div style="text-align: left;">
"grep sort" is a UNIX command, and it would mean to simply search through a list of textual descriptions to find a sorting function. The audience was absolutely silent. For a second. Then, the whole room erupted with laughter. I peeked at Gregor, my advisor, who was sitting in the front row. He crossed his arms and then grinned at me: How would I get out of this? I stammered something along the lines: Well, that's of course true, but assume you have no knowledge of sorting, you don't even know that the word "sort" exist, then with our method, you would still be able to find something. The shouter would just chuckle, shake his head, and then sit down. The idea that a programmer could not now what sorting means did not stroke him as realistic.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
I was just about to recover from that blow and about to put up the slide with the theorem prover diagram, when the next guy stood up, a mocking smile on his lips. "Yes?", I said.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: center;">
"You know, to me this looks like a <i>solution</i> looking for a <i>problem</i>.<i>"</i></div>
<div style="text-align: center;">
<i><br /></i></div>
<div style="text-align: left;">
Again, the entire room erupted with laughter. I don't remember how I replied, but it was just as unconvincing as before. I was finished, publicly humiliated and ridiculed. Putting together the rest of my dignity, I quickly showed the remaining slides. I think I got some applause at the end, and even a question. Still, for the rest of the day, I was marked. Every time a participant saw me, a smile would flash over her or his face for a fraction of a second, remembering my ridicule.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
"Who <i>were</i> these folks?" I asked Gregor. "Congrats", he said. "These were Professors Manfred Nagl and Jochen Ludewig, two really big shots in Germany's Software Engineering. You managed to provoke them. That's good." – "<i>Good?"</i> I said. "These folks just ridiculed me in the front of the entire audience. How would that be good?" – "Well, we'll be known!", he said. "Yes, but for what?" I replied. All the way home, I would be furious at these two hecklers who had so rudely ruined my presentation. And I vowed that one day, I would become a real big researcher, and I would have the last laugh over them.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Over the course of the next twenty years, I would be busy fulfilling my vow, and yes, it sort of worked – never again would a member of the audience shout stuff at me. (But then, this may also be due to me never again presenting formal methods at a Software Engineering audience.) </div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The last laugh, though, never came to be. Two years ago, as I last met Professor Nagl and Professor Ludewig, I asked them whether they would still remember how we met the first time, with me on the stage, and them standing up in the middle of the talk; and I wanted to thank them for how their comments had fueled and ignited my ambition for decades. Of course, they would not remember a thing. For them, it was just a minor laugh among many.</div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com1tag:blogger.com,1999:blog-8747085902902510837.post-36434590920406050432016-04-17T14:15:00.001+02:002016-04-17T14:36:12.803+02:00The new ICSE Erdős penalty, or why we should create incentives for frequent reviewersEver heard of <a href="https://en.wikipedia.org/wiki/Paul_Erd%C5%91s" target="_blank">Paul Erdős</a>? The 20th century Hungarian mathematician is not only known for his numerous contributions to Mathematics, but also for his multiple collaborations, engaging more than 500 collaborators. Frequently, he would just show up on their doorstep, work with them for some hours, and then get a joint paper out of that. A low <a href="https://en.wikipedia.org/wiki/Erd%C5%91s_number" target="_blank">Erdős number</a> indicates academic closeness to Erdős, and is something one can brag about at academic venues. Yet, if today, Paul Erdős knocked on your door, and asked whether you would like to work with him, you <i>should avoid any collaboration with him </i>– if you work in Software Engineering, that is. Why is that?<br />
<br />
The <a href="http://www.icse-conferences.org/" target="_blank">International Conference on Software Engineering</a>, or ICSE for short, is the flagship conference of the field of Software Engineering. If you want to publish and present your greatest work, this is where you submit it. An ICSE submission is reviewed by three peer researchers, whose assessment eventually determines whether your work is accepted or not. Even if your work gets rejected, you at least get detailed reviews and high quality feedback.<br />
<br />
Over the years, ICSE has observed that there were authors who apparently were way more interested in the reviews than in getting their papers accepted; authors who would submit up to ten papers, of which none got in; but the authors would at least get thirty reviews, all for free. This motivated ICSE to install a new limitation: <a href="http://icse2017.gatech.edu/?q=technical-research-cfp" target="_blank">Any single author can now appear only on up to three papers</a>. If you have four papers ready for submission, then you are supposed to select the best three.<br />
<br />
The ICSE program chairs <a href="http://www.cs.mcgill.ca/~martin/blog/2016-04-15.html" target="_blank">argue</a> that few authors and even fewer acceptances would be affected by this decision. But the problem with this decision is not the factual impact. It is the <i>potential impact.</i> What if a modern Paul Erdős knocked on your door and offered you to work with him? You'd have to say no, because he would have too many co-authored submissions already. What if you could not submit to the past conference, because you were the one organizing it, and still have work that wants to get published? What if four of your students all have great results at the same time, results that should be shouted out to the world? Well, too bad: you can only submit three of them, causing depression in the fourth student how is left out. None of these is likely to happen, but the fact that it <i>could</i> happen is causing <a href="https://www.cs.cmu.edu/~ckaestne/icse17/" target="_blank">concerns and anxiety</a>, and rightly so. An open <a href="https://docs.google.com/document/d/1Zeepje-CDp5k73JaO5kFcwZ6Pid_mwTHn42jphiHfks/edit" target="_blank">petition</a> asking ICSE to revert its new rules has gained dozens of supporters overnight. (Disclaimer: me too.)<br />
<br />
The Software Engineering community has members who have literally devoted their <i>lives</i> to Software Engineering research. They have no spouses, no kids, they work day and night. The boys would be out for a skiing weekend, and the girls would be out in their summer clothes – these folks are busy on the paper that they hope will make them famous. They serve on program committees, they write reviews, they organize conferences, they help others on their PhD theses. They are amazingly productive both on their own work as well as on the work of others. And these are the men and women whom the new ICSE rules send the message: Thank you, but no, thank you.<br />
<br />
The problem of ICSE – and our community in general – is not so much an abundance of papers. It is the <i>lack of reviews</i>. It is our publications that determine our academic worth; much less so teaching; and even less so service. Great papers get you tenure and a raise, whereas great reviewing might get you a committee dinner. Rationally thinking, why should one spend time on reviews while one might just write papers that would get reviewed by others? Fortunately, the large majority of our community is still driven by the <a href="https://en.wikipedia.org/wiki/Categorical_imperative" target="_blank">Categorical Imperative</a>: We profit from the reviews of others, so we review their papers, too. What we don't like is members who game the system by not only submitting lots of papers, but also <i>not participating in the review process.</i><br />
<i><br /></i>
Therefore, what our community needs to do is twofold. First, we need to think about reviewing processes that scale well and get high-quality reviews. The ICSE program board model is a step in the right direction; a <a href="http://vldb.org/pvldb/papers/p40-jagadish.pdf" target="_blank">VLDB-like journal model</a> might be even better. Second, we should not penalize researchers for their own productivity; but instead create <i>incentives</i> for researchers who spend great effort on reviews and service. Rule by the carrot, not by the stick.<br />
<br />
Such incentives for service should not be monetary (these wouldn't motivate researchers anyway); nor should they result in a different reviewing or acceptance process (this would be perceived as unfair). But how about raising the limit of submissions if you have a co-author who is also a frequent reviewer? Or allowing reviewing volunteers to apply for a one-day extension to the conference deadline? (You'd get plenty of applications on the last day :-) Or provide "fast track" journal reviewing for those authors who sport a status of "distinguished reviewer"? With such incentives, if a prolific reviewer like Paul Erdős knocks on your door, you would not boot him, but <i>embrace</i> him instead.<br />
<br />Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com4tag:blogger.com,1999:blog-8747085902902510837.post-67750970065230464392016-04-09T07:15:00.000+02:002016-04-12T09:19:27.076+02:00Four security flaws illustrated, all on one conference registration siteWhen you're organizing a big scientific conference – conventions where scientists from across the world would convene to exchange their latest and greatest –, you have to think about zillions of different things: rooms, projectors, food, coffee, budget, accommodations, badges, speakers, leaflets, dinners, just to name a few. It is impossible not to make mistakes, but it is generally possible to fix them once you know. The worst mistakes, though, are the ones you never thought they could be made.<br />
<br />
Some time ago, I registered for one of these scientific conferences. The process is simple: You enter your details, select optional packages, finally enter your credit card data, and you're done. This being a computer science conference, you would think your data is all secure in the hand of experts. As I can now tell you from experience, this assumption is wrong. Very wrong. This single registration system contained not just one security flaw, but <i>four</i> – all independent of each other.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaC-pD-pdvdnAn7_ynyo0VzX_e58G-ei4ARaX2kJrR6gQKOO4zipBLlw_nvkoViB6s_-_cqmMQFxvChSAzjfv6VVhHYCqIYSj4e9YLN6wBoIk2uL-7XNiUZ4SXOjgl0vP1hTYP1bmCKOah/s1600/Screen+Shot+2016-04-08+at+21.35.14.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="250" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjaC-pD-pdvdnAn7_ynyo0VzX_e58G-ei4ARaX2kJrR6gQKOO4zipBLlw_nvkoViB6s_-_cqmMQFxvChSAzjfv6VVhHYCqIYSj4e9YLN6wBoIk2uL-7XNiUZ4SXOjgl0vP1hTYP1bmCKOah/s400/Screen+Shot+2016-04-08+at+21.35.14.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">My Registration Screen</td></tr>
</tbody></table>
<br />
<h2>
Security Flaw #1: The identifiable ID, or How I would be able to access the data of every conference participant</h2>
The fun began when I got my confirmation e-mail. Apparently, I was the first person to have registered with the conference – because my participant number was one. ("Hey – look at me; I am participant number one!" I said.) In my e-mail, I also got a link with which I would be able to access my registration. Following it would immediately lead me to the above registration screen.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDW_OM-zTL6YBasavH2TNP8zIlSdkLphwU_NdwZPFrAr1DrCRbg_2pbcak8hoc_UEeTJpjHSCtbafdL0iQNZlTq72ERY12ffPYm4biMIDlZU4U8bLegzbPiGV6DYJEOGbnHZZc0wyieHOe/s1600/Screen+Shot+2016-04-08+at+21.47.02.png" imageanchor="1"><img alt="" border="0" height="140" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDW_OM-zTL6YBasavH2TNP8zIlSdkLphwU_NdwZPFrAr1DrCRbg_2pbcak8hoc_UEeTJpjHSCtbafdL0iQNZlTq72ERY12ffPYm4biMIDlZU4U8bLegzbPiGV6DYJEOGbnHZZc0wyieHOe/s640/Screen+Shot+2016-04-08+at+21.47.02.png" title="" width="640" /></a><br />
<br />
The link was a bit unusual, because it would not contain any other "secret" information or token other than my participant number, though. (You might assume it would encode my name, my ZIP code, or some other information tied to me only.) So I asked myself: What would happen if I change the link from "?parm1=1" to, say, "?parm1=2" – that is, participant number two? I entered the link into my browser, and immediately, I saw the registration screen of Lars Grunske, a colleague of mine in Stuttgart, Germany.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_6N9Vo-NmFrUGfPtaZgLDj1N853XJp7Ed8mcQe78YeFyKmmteU6mv5GXkG8YTkXztxFExVfPNupQWoiyaXcBoc_QNAhcvE1w0rZZkyVMmWNTeI1e2vpNkHZvpMcZF4vvtn5VCI8S1CDDP/s1600/Screen+Shot+2016-04-08+at+22.00.04.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_6N9Vo-NmFrUGfPtaZgLDj1N853XJp7Ed8mcQe78YeFyKmmteU6mv5GXkG8YTkXztxFExVfPNupQWoiyaXcBoc_QNAhcvE1w0rZZkyVMmWNTeI1e2vpNkHZvpMcZF4vvtn5VCI8S1CDDP/s400/Screen+Shot+2016-04-08+at+22.00.04.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Lars Grunske's registration screen</td></tr>
</tbody></table>
<br />
Now having the same privileges as Lars, I would be able to read and change all data at will. The idea arose to have him buy a few extra dinner tickets at his expense, but only briefly so. Trying the same for further participants gave the same (Hi Abhik!, ¡Hola Yasiel!).<br />
<br />
In 2011, a similar mistake was made by UNESCO, who also used consecutive numbers for its internship applicants, and who thus leaked hundreds of thousands of applicant records on the Web. (<a href="http://www.spiegel.de/netzwelt/web/datenleck-bei-uno-organisation-unesco-entbloesst-hunderttausende-bewerber-im-web-a-759538.html" target="_blank">German Article</a> on Spiegel.de) What do you do when you discover such a problem? To protect the integrity of participant data, I dutifully reported the problem to the organizers, who immediately replied the issue would be fixed as soon as possible.<br />
<br />
<b><i><span style="color: red;">Lesson 1: When handling personal data, set it up such that access requires a secret that cannot be easily guessed.</span></i></b><br />
<b><i><span style="color: red;"><br /></span></i></b>
<br />
<h3>
</h3>
<h2>
Security Flaw #2: The Unsanitized input, or How I easily bypassed password checks</h2>
The next day, I got a new mail from the organizers. In addition to lots of high end security stuff (which would not protect from guessing a participant number), they now had introduced a secret word only known to the registrant, commonly known as a password.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYqNfXHa0kWoB-5lio6eH2gqk0pfZNEnZmQTEBnR1f_X7LuarXSkhyphenhyphenGuSqDHU-fkhrODUsw68bZzZ-U0rTMlJqPAg9-kNrL-KKLZsJylZUzNofsLAISL2Wv_sXe9dZDXTsinRLFMj5pDbW/s1600/Screen+Shot+2016-04-08+at+22.08.26.png" imageanchor="1"><img border="0" height="138" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiYqNfXHa0kWoB-5lio6eH2gqk0pfZNEnZmQTEBnR1f_X7LuarXSkhyphenhyphenGuSqDHU-fkhrODUsw68bZzZ-U0rTMlJqPAg9-kNrL-KKLZsJylZUzNofsLAISL2Wv_sXe9dZDXTsinRLFMj5pDbW/s640/Screen+Shot+2016-04-08+at+22.08.26.png" width="640" /></a><br />
<br />
Okay. I went to the site, and it indeed now requested that I enter my ID and password.<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIn4OGdLQYmPR2SPR16LFPRjDw3VUdJLcPP220ihPrEgSFR6N5n1_Vb5JbCdQy74izBI5InyTPxO-dieax90qnvMiUpW-AQQbs3c9YjQf6ZUwFiZxozCpGMdMu7BjSafrX0oQoBOORU56k/s1600/Screen+Shot+2016-04-08+at+22.15.44.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="125" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiIn4OGdLQYmPR2SPR16LFPRjDw3VUdJLcPP220ihPrEgSFR6N5n1_Vb5JbCdQy74izBI5InyTPxO-dieax90qnvMiUpW-AQQbs3c9YjQf6ZUwFiZxozCpGMdMu7BjSafrX0oQoBOORU56k/s400/Screen+Shot+2016-04-08+at+22.15.44.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Revised login interstitial screen</td></tr>
</tbody></table>
Problem solved? Not at all. I sent the above mail to my Post-Docs Marcel and Juan Pablo "JP" Galeotti, whom I had talked about the problem the day before. Minutes later, Marcel Böhme sent me back an intriguing message:<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmnamo_Pd6X-5kGTgM-5hcBWtjGZha3G76toOOXJfh42hb6jXDkag5k_OvXyLTiK7BOVSeJ6FSU2VWV-9ytb5QhfjsjUn3NK8dj44Jv2yHsaXjNjudvZI0t7V_oBwtpqdAK6uSdpnCxhaF/s1600/Screen+Shot+2016-04-08+at+22.18.02.png" imageanchor="1"><img border="0" height="75" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmnamo_Pd6X-5kGTgM-5hcBWtjGZha3G76toOOXJfh42hb6jXDkag5k_OvXyLTiK7BOVSeJ6FSU2VWV-9ytb5QhfjsjUn3NK8dj44Jv2yHsaXjNjudvZI0t7V_oBwtpqdAK6uSdpnCxhaF/s640/Screen+Shot+2016-04-08+at+22.18.02.png" width="640" /></a><br />
<br />
Incredible. Could one really attack the system this way? Ten minutes later, JP chimed in with<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_vna4fBqEdOFZ0C6bT0fIleb3XdpFZ3XC6YOS278z9GA5lrCEzZecM0B0iaHfms4sPkOY7H7W4AxLJ4hJNFLWvx-hbdmd9Ka-y8-qt-ClNoiFZsMMN1hI8BtDK05mtHniycexro0GeGky/s1600/Screen+Shot+2016-04-08+at+22.20.21.png" imageanchor="1"><img border="0" height="102" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_vna4fBqEdOFZ0C6bT0fIleb3XdpFZ3XC6YOS278z9GA5lrCEzZecM0B0iaHfms4sPkOY7H7W4AxLJ4hJNFLWvx-hbdmd9Ka-y8-qt-ClNoiFZsMMN1hI8BtDK05mtHniycexro0GeGky/s640/Screen+Shot+2016-04-08+at+22.20.21.png" width="640" /></a><br />
<br />
Ha! Indeed, this worked like a charm. Eventually, I would simply enter "<span style="font-family: "courier new" , "courier" , monospace;">2' -- </span>" as my ID, and any string as my password – and again, I would be Lars Grunske, and would be able to alter his data at will. Likewise, anyone with the above trick could do the same to my data. <br />
<br />
How does this work? Internally, the conference registration system uses a database that is controlled by so-called SQL commands. When I enter my ID, say, "1", and my password, say, "1234", the system selects my data from the database using a SQL command looking like this:<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">SELECT * FROM REGISTRATIONS WHERE ID = '1' AND PASSWORD = '1234'</span><br />
<br />
Note how the number I entered as ID ("1") becomes part of the command. By entering "<span style="font-family: "courier new" , "courier" , monospace;">2' -- </span>" as ID and "whatever" as password. we get the command<br />
<br />
<span style="font-family: "courier new" , "courier" , monospace;">SELECT * FROM REGISTRATIONS WHERE ID = '2' -- 'AND PASSWORD = 'whatever'</span><br />
<br />
In a SQL command, anything starting with two dashes "--" is treated as a comment and ignored. So the system simply fetches the data from the registrant whose ID is 2, <i>ignoring</i> the password. This is known as a <a href="https://en.wikipedia.org/wiki/SQL_injection" target="_blank">SQL injection</a> attack, and the standard way to avoid these is to filter out all characters that would have a special meaning in SQL commands (like "'" or "--").<br />
<br />
Refining my ID to, say, "<span style="font-family: "courier new" , "courier" , monospace;">2'; DROP TABLE REGISTRATIONS; -- </span>", I might even have been able to <i>delete</i> all registration data. (I hope they do backups!) How could one set up a SQL-based system and never have heard about <a href="https://en.wikipedia.org/wiki/SQL_injection" target="_blank">SQL injection</a>? Now this was beginning to get embarrassing.<br />
<br />
<b><i><span style="color: red;">Lesson 2: When setting up a publicly accessible service, identify common attack vectors and protect against them – for Web sites: buffer overflows, SQL injection, cross-site scripting, etc.</span></i></b><br />
<b><i><span style="color: red;"><br /></span></i></b>
<br />
<div>
<h3>
</h3>
<h2>
Security Flaw #3: Plaintext Passwords, or How I would now also steal personal passwords from all participants</h2>
</div>
<div>
But the embarrassment was not over yet. Remember how the e-mail above asked users to set up their own passwords? It turned out that the passwords were actually <i>stored and displayed in the clear</i>, as seen on Lars' revised registration screen:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTm-3r5OpW9BXcmjK_r74KTtbQidGlmHRvaETfg-_h2vwoXGIkWkq3Pah9Xf_1oL-L1yCOW516hLaqXDJG6-VgjHfgH9hMWB43NyKW_7m04OTz6dzU9aagys6NzoH2jgoP5H3R6Ty1A1I0/s1600/Screen+Shot+2016-04-08+at+22.31.35.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTm-3r5OpW9BXcmjK_r74KTtbQidGlmHRvaETfg-_h2vwoXGIkWkq3Pah9Xf_1oL-L1yCOW516hLaqXDJG6-VgjHfgH9hMWB43NyKW_7m04OTz6dzU9aagys6NzoH2jgoP5H3R6Ty1A1I0/s400/Screen+Shot+2016-04-08+at+22.31.35.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Lars Grunske's registration screen, now with password</td></tr>
</tbody></table>
<br />
The password listed was Lars' password; saving it would allow me to log in with his user ID and password. I could easily have skimmed all passwords of all participants and I could have logged in long after the SQL vulnerability had been fixed.<br />
<br />
But storing passwords in the clear is a bad practice for many more reasons. It gives the administrator access to all passwords, which provides an opportunity for thefts. Plus, and this is probably the worst: Many people tend to use<i> the same passwords for different sites.</i> Had Lars indeed changed his password as requested, and for instance chosen the same password he would use for Amazon or eBay, I would be able to log in at these sites on his behalf, and happily order stuff.<br />
<br />
Had Lars used the same password he also uses for his mail, I could have accessed <i>all</i> of his passwords, everywhere – a simple click on "I have forgotten my password" links would have triggered "password reset" mails to his account, which I could easily have skimmed. I'm a nice guy, so I did none of this. (Plus, I have a cool joint research project with Lars.)<br />
<br />
To cut a long story short: I sent another urgent report to the organizer, and hours later, the SQL vulnerability was closed. The one we had found, that is. No idea whether other vulnerabilities would be hidden somewhere in there, or how the system had been tested for security, if at all.<br />
<br />
<b><i><span style="color: red;">Lesson 3: Passwords should never be stored, displayed, or transmitted in the clear. Store hashes instead; and if a user requests a new password, create a new one instead.</span></i></b><br />
<b><i><span style="color: red;"><br /></span></i></b>
<br />
<div>
<h3>
</h3>
<h2>
Security Flaw #4: Compromised Forever, or How nobody would be able to change their passwords</h2>
</div>
Was it really the case that Lars had ignored the instructions and kept his original password? After all that had happened, I thought that maybe someone had done the same that I had done, and thus now had access to my conference password. So I decided to change it online. However, it turned out that <i>changing the password did not work</i> – you would always retain the old one, which would still be happily displayed for you. The good news was that this way, nobody would have been able to reuse existing passwords – and anyway, had I really wanted my password changed, another mail to the organizers might have done the trick. At this point, I decided not to further stress the relationship between the organizers and their software developers and leave this be.<br />
<br />
<b><i><span style="color: red;">Lesson 4: Always allow your users to reset their access data if they fear it may have been compromised.</span></i></b><br />
<div>
<br /></div>
All was well that ended well: The conference was truly magnificent, and as far as I know, nobody's data got compromised in any way. Of course, anybody could have gone through the steps described above, skimming data without ever reporting. But luckily, the first registrant (me) pointed out the issues before some fraudster could have spotted it, and of course, any of my colleagues would have done just the same. When your customers are nice people, consider yourself lucky.<br />
<br />
<div>
<h2>
Post Scriptum: The Horrible Homebrew, or Why it may be better to build on well-tested platforms</h2>
</div>
<div>
This brings me to the meta lesson to be learned here – and this tells something about process rather than product. If you set up a system from scratch, be it a conference management system, a shop, a student registration system, whatever, be aware of the many risks this entails, and be sure to have independent and thorough security testing. Using an existing, established, well-tested system instead may lower risk and overall cost, even if it may cost more upfront. When the damage is done, you wish you had decided differently – but then, it may be too late.</div>
<br />
<i><b><span style="color: red;">Final Lesson: </span></b></i><i><b><span style="color: red;"> When deciding between building and using a system, consider all risks and associated costs.</span></b></i><i><b><span style="color: red;"> If you build a new system, thoroughly test it for security. If you use an existing system, be sure it is well tested.</span></b></i>Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-35620051750264289112016-03-10T14:30:00.003+01:002016-03-10T20:26:04.700+01:00Mining Sandboxes for SecurityFor the past three years, my students and I have worked on a novel and general approach to address software security. This week at CeBIT, we're happy to spread some good news. <br />
<div>
<br /></div>
<div>
The concept of "Mining Sandboxes" protects against <i>unexpected changes of software behavior</i> and thus drastically reduces the attack surface of software systems. Our "Boxmate" prototype automatically mines program behavior by <i>executing generated tests</i>, systematically exploring the program’s behavior together with the accessed resources. The collected behavior rules form a <i>sandbox</i>, which at production time prohibits <i>behavior not seen during testing</i>. This brings several compelling features:<br />
<div>
<br /></div>
<div>
<div>
<b>No unexpected behavior changes.</b> The mined sandbox prevents <i>behavior changes </i>caused by latent malware, vulnerability exploitations, malware infections, or targeted attacks.</div>
<div>
<br /></div>
<div>
<b>Closing the backdoors. </b>The mined sandbox protects against <i>backdoors</i> that would not be discovered during normal usage.</div>
<div>
<br /></div>
<div>
<b>No malware patterns required.</b> The approach assumes no information about earlier or future attacks; it protects against known and novel attacks alike.</div>
<div>
<br /></div>
<div>
<b>No training in production.</b> In contrast to anomaly detection systems, all “normal” behavior is already explored during testing. The program is protected even before its first deployment.</div>
<div>
<br /></div>
<div>
<b>No code required. </b>We require no knowledge about source or binary code, and thus can handle obfuscated, obscure, or adverse programs.</div>
</div>
<div>
<br />
Want to get a brief overview of how it works? Here's a video, narrated by yours truly:</div>
<div>
<br />
<iframe allowfullscreen="" frameborder="0" height="315" src="https://www.youtube.com/embed/99Mi5i_ZNYI" width="560"></iframe>
</div>
<div>
<br />
Conceptually, the techniques of Mining Sandboxes scale to arbitrary code size and can be applied to mobile apps, embedded systems, and server software alike. Mining sandboxes is fully automatic, such that vendors, developers, and users can mine, inspect, compare, and exchange sandboxes at any time. And for the testing researchers among us: This research leverages the incompleteness of testing to turn it into an advantage; and to actually produce guarantees from testing.
</div>
<div>
<br /></div>
<div>
At this point, all of this is still research, so you cannot yet buy this in a shop near you. And as with any big set of benefits, there's also drawbacks, in particular with legitimate functionality not found during testing – and there's still loads and loads of things to do. But our first results with our prototype on real-world apps are more than promising; we have a nice paper coming up at the ICSE conference in May 2016, Austin, Texas. And I am even more excited about this work than any other pioneering work of ours before, even more than say, Delta Debugging, or Mining Software Repositories – because if it succeeds, it would not only impact the lives of software developers, but actually address many of the software security problems we see in the news every day.</div>
<div>
<br /></div>
<div>
If this has captured your attention, you can read more about the project at its site: <a href="http://www.boxmate.org/">http://www.boxmate.org/</a>. Or you can visit us at the CeBIT computer fair in Hannover between March 14 and March 18 (Hall 6, Stand D 28); I will be on site Monday and Tuesday. I'd love to get engaged in discussions!</div>
</div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-63848735547215927102015-03-15T10:06:00.000+01:002015-03-15T10:23:20.541+01:00If you ran an EasyChair conference in the past, here's how to shut down access to past dataIf you have published or reviewed computer science papers in the past years, you may have ran across <a href="http://www.easychair.org/" target="_blank">EasyChair</a>. EasyChair is a conference management system, storing and managing submitted papers, reviews, and final decisions. It is a great tool for anyone organizing a conference, and it is extremely popular: According to its own data, it has served over <i>a million</i> registered users since 2007.<br />
<br />
The problem with EasyChair, however, is exactly that it <i>is</i> so popular. Again since 2007, I have acted as reviewer in more than 50 conferences and workshops organized through EasyChair. All their data is still there: As a reviewer, I have access to <i>all</i> papers and <i>all</i> reviews written for each of these conferences. This opens up interesting possibilities for misuse. For instance, I could download all reviews written by my colleagues, <a href="https://www.schneier.com/blog/archives/2011/08/identifying_peo_2.html" target="_blank">train a classifier on their writing styles</a>, and identify them for my past and future papers. I could also check for papers of my colleagues submitted and rejected several years ago, and confront them with the sins of their youth.<br />
<br />
<a href="http://cacm.acm.org/magazines/2011/1/103200-cloud-computing-privacy-concerns-on-our-doorstep/fulltext" target="_blank">This CACM article</a> by Mark D. Ryan neatly summarizes the problem, but also points out that there may be benign uses of all this data. Hence, the scientist in me thinks that deleting all this data may be a bad idea for future historians. What we can and should do, though, is to <i>limit access </i>to it. As a starter, after the conference is over, <i>every program chair should take means that its submissions and reviews can only be accessed by a minimal set of people</i> – in particular, excluding past reviewers to extract all data. Here's how to do this for EasyChair:<br />
<br />
<ol>
<li>Log in as program chair.</li>
<li>In the top menu, go to "Administration" → "Configure".</li>
<li>Under "Access to Submissions", set</li>
<ul>
<li>"Are submissions anonymous?" to "Yes"</li>
<li>"Can non-chairs see information on submissions not assigned to them?" to "No"</li>
</ul>
<li>Under "Paper bidding and assignment", set</li>
<ul>
<li>"Is paper bidding enabled?" to "No"</li>
<li>"Is viewing bids of PC members by chairs enabled?" to "No"</li>
<li>"Is assignment of submitted papers to program committee enabled?" to "No"</li>
</ul>
<li>Under "Reviewing", set</li>
<ul>
<li>"Are reviewer's names visible to PC?" to "No"</li>
<li>"Status menu is" to "disabled"</li>
<li>"Review menu is" to "disabled"</li>
<li>"Permit PC members to enter reviews" to "No"</li>
<li>"Permit non-chairs see and discuss reviews" to "only their own reviews"</li>
</ul>
</ol>
<div>
To check the effect of these settings:</div>
<div>
<ol>
<li>In the top menu, go to "PC".</li>
<li>Select any regular PC member and click on "login as" (the rightmost item)</li>
<li>All the PC member should now be able to see are the submissions initially assigned to him or her.</li>
</ol>
<div>
You as a PC chair can still access all papers and reviews, and via the "Configure" menu, you can also re-enable access (for whatever reason). If you configure the settings as above, though, the risk of data leaks will be greatly diminished.</div>
<div>
<br /></div>
<div>
If you have similar instructions for other conference management systems, let me know and I will link or include them here. And if you as a reviewer find that your conference management system still grants you access to everything, tell your PC chair about this page. Thanks a lot!</div>
</div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com4tag:blogger.com,1999:blog-8747085902902510837.post-73474001521245815062014-10-07T11:51:00.002+02:002014-10-07T13:56:41.994+02:00Wie Universitäten die "Industrie vor Ort" unterstützen können: Firmen gründen, statt Firmen nachzueifern<i>Forscher stehen unter Druck, Probleme der Industrie vor Ort zu lösen. Dies verkennt jedoch die Rolle der Universitäten. Würden Länder und Universitäten aber Firmengründungen besser unterstützen, wäre allen geholfen, meint Andreas Zeller.</i><br />
<br />
Ein Kollege von mir hat sich unlängst an einer deutschen Universität auf eine Softwaretechnik-Professur beworben. Im Bewerbungsgespräch ging es neben zukünftiger Forschung und Lehre vor allem um die Frage, "wie er denn der lokalen Industrie helfen könnte". Die lokale Industrie: das ist ein Mix von mittelständischen Unternehmen, die vor Ort Produkte wie Bleistifte oder Autoteile produzierten. Mein Kollege gab wahrheitsgemäß zur Antwort, dass seine Forschung vorwiegend in der Software-Erstellung eingesetzt werde; er wäre dementsprechend international ausgerichtet. Diese Antwort schien der Kommission aber nicht zu gefallen; und so erhielt er eine freundliche Absage.<br />
<br />
Die Frage, "was man denn für die lokale Industrie tun könnte", ist mir als Forscher schon mehrfach begegnet. In Verhandlungen für eine Professur an einer norddeutschen Universität wurden mir stolz die fünf Büroräume präsentiert, die für die "vielen lokalen Projekte" reserviert waren, die ich sicherlich "bald akquirieren würde". Die Landesregierung des Saarlandes hat die hiesige Informatik durch die Blume wissen lassen, dass sie es trotz aller Erfolge der Saarbrücker Informatik gerne sähe, wenn es mehr Kooperationen "vor Ort" gäbe, von denen die "lokale Wirtschaft" profitierte.<br />
<br />
Die Forderung nach "Kooperation vor Ort" ist zunächst einmal nachvollziehbar. Hier bin ich, der Leiter eines mittelständischen Unternehmens, und ich habe große Probleme mit meiner IT. Zufälligerweise gibt es nur wenige Kilometer entfernt eine große Universität, voll mit Informatikern, die Ahnung haben. Da liegt es doch nahe, zu fragen, ob die nicht helfen können. Und da die Universität meistens aus Landesmitteln finanziert wird, die zudem aus meinen Steuern kommen, könnte das Land doch entsprechend Druck ausüben, richtig?<br />
<br />
Diese Denkweise verkennt jedoch die gesellschaftliche Rolle einer Universität. Universitäten haben zum einen die Aufgabe, <i>Forschung zu betreiben</i> – und sich mit Fragen zu beschäftigen, die in der Regel zu abseitig oder zu risikoreich sind, um damit unmittelbaren wirtschaftlichen Gewinn zu erzielen. Es kann Jahre, gar Jahrzehnte dauern, bis Forschungsergebnisse außerhalb der Universitäten aufgegriffen werden. Immer wieder aber gelingt der eine oder andere Durchbruch, der ohne langjährige freie Forschung gar nicht möglich gewesen wäre, und davon profitiert die Gesellschaft als Ganzes.<br />
<br />
Damit die Gesellschaft aber Nutzen von Forschung ziehen kann, müssen Forscher wissen, welche Probleme es zu lösen gibt. Als Informatiker muss ich mich damit auseinandersetzen, wie ich komplexe, vernetzte Systeme sicher und verlässlich mache, wie ich wichtige Daten schützen kann; und ich muss über kurzfristige wie langfristige Lösungen nachdenken. Im Gespräch mit der Industrie bekomme ich einen Eindruck davon, welche komplexen Nebenbedingungen es gibt, und welche Lösungen zumutbar sind; auch dies ist ein wichtiger und wertvoller Teil meiner Forschung. Geht es jedoch darum, die konkreten Probleme "vor Ort" zu lösen, muss ich mich fragen, inwiefern ich aus einer Lösung ein allgemeines Prinzip ableiten kann. <br />
<br />
Wäre ich Maschinenbauer in Braunschweig oder IT-Forscher im Silicon Valley, hätte ich vor Ort jede Menge "lokaler" Industriepartner, bei denen dies problemlos ginge – Firmen nämlich, die im Kernbereich meiner Forschung produktiv tätig sind, und die daher meine Interessen teilen, innovative, tragfähige, und möglichst allgemeine Lösungen zu finden. Kümmere ich mich jedoch um die konkreten Probleme der <i>Anwender</i>, etwa der Bleistiftfirma, die ihre IT neuorganisieren muss, geht Zeit und Energie für die allgemeinen Lösungen verloren – und das sind die, die zu finden ich bezahlt werde.<br />
<br />
Wenn ich es will, kann ich natürlich "vor Ort" auch mit IT-Anwendern arbeiten. Ich mache einen Schwung Beratungsverträge mit der lokalen Industrie, und stelle billige Doktoranden ein, die dann die Probleme vor Ort lösen (und "nebenher" irgendwie promovieren). Ich generiere jede Menge "Drittmittel" aus der Industrie. Ob dies zu einem allgemeinen Erkenntnisgewinn führt, bleibt hintenangestellt – Hauptsache, die Zahlen stimmen. Will ich das, und warum?<br />
<br />
Die zweite Aufgabe der Universitäten ist es, <i>junge Leute zu bilden</i> – oder weniger vornehm, auszubilden. Meine Kollegen und ich haben den Anspruch, Menschen zu produzieren, die die jetzigen und kommenden Herausforderungen der Informatik meistern können. Wir tun dafür, was wir können – und mit Erfolg: Unsere Absolventen können sich in der Regel aussuchen, wo sie arbeiten; und sie gehen dorthin, wo sie die beste Arbeit bekommen. Firmen wie Google oder Microsoft sitzen nicht nur an angesagten Orten, sie zahlen auch sehr gut; und sie sorgen dafür, dass sich unsere Absolventen um nichts kümmern müssen, einschließlich Wohnung, Mahlzeiten und Wifi-Shuttle. Unsere besten Absolventen sind weltweit begehrt, und sie wissen das.<br />
<br />
Dies wiederum macht der "lokalen Industrie" zu schaffen, die nicht mithalten kann oder möchte. Ein örtlicher Vertreter einer großen deutschen Telekommunikationsfirma hat mich vor kurzem besucht, und gefragt, wie es denn sein könne, dass so wenig unserer Absolventen sich bei ihnen vor Ort bewürben – schließlich sei es doch eine spannende Herausforderung, das Zusammenspiel von nicht weniger als zwölf Alt- und Neusystemen zu orchestrieren. Ich fragte zurück, ob er mit den Sozialleistungen etwa von Google mithalten könne. Nun ja, sie würden nach Tarif bezahlen, hat er geantwortet, und Essen und Massage – nein, das sei nicht drin. Und da ist das Problem.<br />
<br />
(Was das ganze noch pikanter macht, war die Forderung eines IHK-Vertreters, wir mögen unsere Ausbildung weniger international gestalten, damit die Firmen "vor Ort" mehr Chancen hätten, unsere Absolventen anzuwerben. Muss ich das kommentieren?)<br />
<br />
Was die lokale Industrie will, ist die Lösung ihrer Probleme. Das ist völlig legitim. Was sie vermeiden möchte, ist einen Marktpreis dafür zu zahlen. Es gibt sehr gute Absolventen, die ein Problem nach dem anderen lösen, doch die kosten Geld – und schlimmer noch, sie verlangen gute Arbeitsbedingungen und das Gefühl, etwas wichtiges zu tun. Es gibt hervorragende Beratungsfirmen, deren Problemlösung man einkaufen kann; doch die kosten noch mehr Geld. Wer fordert, Forscher an den Universitäten mögen doch bitte die Probleme "vor Ort" lösen, verlangt <i>staatlich subventionierte Dienstleistungen</i> – die in Konkurrenz zu Absolventen und IT-Industrie stehen.<br />
<br />
Ich habe nichts gegen Subventionen, wenn sie als Investition dienen, also langfristig Gewinn versprechen. Wenn ein Land, eine Universität, aber etwas langfristig Gutes für die Industrie "vor Ort" tun will, und dabei freie Forschung nicht nur nicht behindert, sondern unterstützt, gibt es einen anderen, weit besseren Weg, nämlich <i>aus der Universität heraus Firmen zu gründen.</i> Eine Neugründung kann einerseits spannende Forschungsergebnisse vermarkten, und so Teil der Industrie vor Ort werden. Sie kann aber auch Dienst- und Beratungsleistungen vor Ort anbieten, und so der Wirtschaft weiterhelfen. Eine Neugründung kann sich die Arbeitsumgebung schaffen, die die richtigen Absolventen anzieht. Das Land freut sich über Arbeitsplätze und erhöhtes Steueraufkommen; die Universität profitiert durch Lizenzeinnahmen und einfacherer Legitimation ihrer gesellschaftlichen Relevanz.<br />
<br />
Bin ich als Forscher mit einem Startup assoziiert, muss ich mich nicht fragen, wie ich Anfragen der Industrie im Universitätsalltag lösen kann; ich kann einfach auf die entsprechende Firma verweisen. Ich habe ein Ohr an den zu lösenden Problemen, und kann meine Forschung anwendungsnäher gestalten, ohne den Anspruch auf Allgemeinheit aufzugeben. (Wenn dabei eine neue Forschungs- oder Firmenidee abfällt, um so besser.) Und ich kann den Absolventen, die nach ihrem Abschluss weiterhin spannende Arbeit machen möchten, die Startups "vor Ort" empfehlen. Jeder kümmert sich um seine Aufgaben: Die Universität schafft Wissen, das Startup schafft Umsatz.<br />
<br />
Wer also "vor Ort" wirken will, soll <i>Firmengründungen unterstützen</i> – zum einen durch Fördergelder, vor allem aber durch das Schaffen von Strukturen, die das Gründen vereinfachen. Dazu gehören Anlaufstellen für Beratung und Information; Netzwerke und Veranstaltungen zum Finden von Mentoren, Mitstreitern und Investoren; bezahlbare Firmensitze in Uni-Nähe; und generelles Werben für die Fortführung der eigenen Ideen in einer eigenen Firma. Spezielle Kurse für Firmengründer. Mix-and-Match-Veranstaltungen zwischen Forschern und Betriebswirtschaftlern. Treffen mit erfolgreichen Gründern. Macht Eure eigene Firma auf – und tut was "vor Ort"!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY3s-hqqvkF-gr38JZbk8Ya6rLGta1pS9wTCfzhSk9zjFISsLXuWDPQRIWT9cZtk2wqR_ZKHqzoJBFRCiUO08iP52cCHC2A_FLsUOZWS-HpnHj2eBx0A3CiIdIJqGV4HQ4Tl250B41yTzk/s1600/Gruender-Campus-Saar-Newsletter-1-2014.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiY3s-hqqvkF-gr38JZbk8Ya6rLGta1pS9wTCfzhSk9zjFISsLXuWDPQRIWT9cZtk2wqR_ZKHqzoJBFRCiUO08iP52cCHC2A_FLsUOZWS-HpnHj2eBx0A3CiIdIJqGV4HQ4Tl250B41yTzk/s1600/Gruender-Campus-Saar-Newsletter-1-2014.jpg" height="221" width="400" /></a></div>
<br />
<b><br /></b>
<b>Eine Universität soll gründen, statt sich selbst als Firma aufzuspielen. Wenn sich dieser Gedanke in den Hochschulleitungen durchsetzt, werden zukünftige Bewerber vielleicht gefragt, ob und wie sie helfen können, Firmengründungen zu unterstützen. Und das wäre nicht das schlechteste.</b><br />
<br />
<br />
<span style="color: #021eaa; font-family: 'Times New Roman'; letter-spacing: 0px;"><a href="http://www.st.cs.uni-saarland.de/zeller/"><b><i>Andreas Zeller</i></b></a></span><span style="font-family: 'Times New Roman'; letter-spacing: 0px;"><i> ist seit 2001 Professor für Softwaretechnik an der </i></span><span style="font-family: 'Times New Roman'; letter-spacing: 0px;"><span style="color: #011993; letter-spacing: 0.0px;"><i><a href="http://www.st.cs.uni-saarland.de/">Universität des Saarlandes</a>, einer </i></span></span><i style="font-family: 'Times New Roman';"><a href="http://www.exist.de/exist-gruendungskultur/gruenderhochschule/" target="_blank">EXIST-Gründerhochschule</a></i><span style="font-family: 'Times New Roman'; letter-spacing: 0px;"><i> </i></span><span style="font-family: 'Times New Roman'; letter-spacing: 0px;"><i>in Saarbrücken</i></span><i style="font-family: 'Times New Roman';">. Seine mehrfach ausgezeichneten Test- und Analyseverfahren werden in zahlreichen Unternehmen eingesetzt. Die</i><i style="font-family: 'Times New Roman';"> von ihm </i><i><span style="font-family: Times New Roman;">2013 mitbegründete </span></i><a href="http://www.testfabrik.com/" style="font-family: 'Times New Roman';"><span style="color: #021eaa; letter-spacing: 0px;"><b><i>Testfabrik AG</i></b></span></a><i style="font-family: 'Times New Roman';">, ein Startup zum automatischen Testen von Web-Anwendungen, <a href="http://www.kwt-uni-saarland.de/de/bereiche/unternehmensgruendungen/starterfirmen/informatik-spin-off-der-saar-universitaet-siegt-beim-europaeischen-ideenwettbewerb-fuer-start-ups.html" target="_blank">siegte 2014 im E</a></i><span style="font-family: Times New Roman;"><i><a href="http://www.kwt-uni-saarland.de/de/bereiche/unternehmensgruendungen/starterfirmen/informatik-spin-off-der-saar-universitaet-siegt-beim-europaeischen-ideenwettbewerb-fuer-start-ups.html" target="_blank">uropäischen Ideenwettbewerb für Startups.</a> </i></span><br />
<i style="font-family: 'Times New Roman';"><span style="font-family: Times New Roman;"><br /></span></i>
<i style="font-family: 'Times New Roman';"><span style="font-family: Times New Roman;">Die Firmengründung wurde durch den </span><a href="http://www.kwt-uni-saarland.de/de/bereiche/unternehmensgruendungen/gruender-campus-saar.html" target="_blank">Gründer-Campus Saar</a><span style="font-family: Times New Roman;"> an der Universität des Saarlandes unterstützt und durch einen <a href="http://www.exist.de/exist-forschungstransfer/" target="_blank">EXIST-Forschungstransfer</a> mitfinanziert. Die Firma sitzt derzeit in einem der drei <a href="http://www.kwt-uni-saarland.de/de/bereiche/unternehmensgruendungen/starterzentrum.html" target="_blank">Starterzentren</a> der Universität. Neue</span></i><i style="font-family: 'Times New Roman';"> IT-Gründungen profitieren zudem vom <a href="http://itinkubator.de/" target="_blank">IT Inkubator</a> der Max-Planck-Gesellschaft und der Universität des Saarlandes.</i><br />
<br />
<br />
<br />Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-52117619040299190112014-07-07T12:09:00.001+02:002014-07-07T12:09:34.296+02:00This day in my history: The 1974 World Cup Final as seen from Beirut, LebanonToday exactly 40 years ago, on May 7, 1974, my family and I watched the Football World Cup Final between Netherlands and West Germany on a tiny TV in Beirut, Lebanon. 1974 was the last year before the civil war broke out, and Beirut was still a wonderful post-colonial middle-east city with clear Western (and mostly French) influence; it also was a major tourist destination for Westeners. I remember Beirut as chic, white and golden, the Beeqaa Valley as green and incredibly fertile, and the Roman temples of Baalbek as white and green – it was about the first time my then nine-year old self fully realized the beauty of landscapes and architecture. All the charms and beauty of Beirut would get lost in the 16 years of civil war that would ignite only a few months later, and well-armed soldiers on every major crossing were a sign of things to come.<br />
<br />
How would we get to Beirut, you may ask? At the time, my father was a German teacher in Bangui, Central African Republic, and we lived there in a nice house with a lavish garden; every Summer, we would return to Europe, though, and on some of these return trips, we would include stopovers in locations on the way. At the time, the Western and Eastern blocks were still fighting for dominance in Africa, and consequently, the Central African Republic was well-connected – both by Western Airlines (UTA and Air Afrique), as well by Soviet Airlines (Aeroflot). With Aeroflot being cheaper than the others, we would use the saved money for stopovers, which is how we ended up for short trips to Moscow, Russia; Cairo, Egypt; and, in 1974, Beirut, Lebanon.<br />
<br />
The flight crew and us were in the same hotel, and my father and the crew had already made acquaintances during the trip from Africa. It turned out that this very day was the finals of the 1974 Soccer World Cup in Munich, with the Netherlands playing against West Germany. The crew also knew that we were Germans – and one of their rooms had a TV! So at breakfast in the restaurant, they approached us and told us that they wanted to watch the finals later today. Would we like to join them? Of course, my parents gladly accepted.<br />
<br />
So my sister and I found ourselves in a stuffy Beirut hotel room with our parents and a Russian flight crew, in front of a tiny, blurry, black and white TV screen with the audio in a language I did not understand (Arabic? Russian?). It was even hard to distinguish the teams, although the light grey team seemed to be the Germans, and the darker grey team seemed to be the Dutch. And I occasionally got the words "Beckenbauer" and "Cruyff", whom I identified as the main protagonists of their respective teams. On every goal shot, vodka bottles were passed, and the crew's eyes got more and more watery. The game lasted for 90 minutes, and at the end, the Germans won.<br />
<br />
At this time, I was a nine-year old boy, and you might think I would have been super enthusiastic about watching the World Cup Final, captured by the game, and knowing each single player in the team. Unfortunately, this wasn't the case. Having spent the past year in Africa, I knew nothing about German football, its teams, or its players; nor did I know how the world cup went so far. All the news we had were a weekly news magazine, Der Spiegel, which normally would arrive one to three weeks late; occasional short-wave radio news; and French Newsreels, to be shown in the local cinemas.<br />
<br />
Still, I was glued to the screen, like my parents, like the crew with their wide open, bright blue eyes. But in this moment, what I found thrilling was not so much that Germany was just about to win the World Cup. It also wasn't that I was in a hotel room in Beirut with a bunch of Russians on my way back from Africa. It was that I was looking at real live TV.Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-69778447705301696722014-07-02T11:33:00.000+02:002014-10-07T11:54:03.117+02:00if you plan to invite me for your upcoming event, please avoid nightmares like this oneI frequently get invitations to give talks at conferences, summer schools, and other events. Normally, it is an easy thing: I check my availability, agree, book travel and hotel, prepare and give a talk, have fun, get reimbursed. Not so with this recent summer school in France, though.<br />
<br />
January. I get an invitation to lecture at a summer school in France. Sounds fun and exciting, but it is in our exam week. I go for a compromise: I accept, but will be able to join only for one day. As usual, I don't get paid for such events, but the trip will be taken care of.<br />
<br />
February. The organizers ask all speakers for a flight connection, such that they can book a flight for us. I spend an hour looking up possible flight connections on Expedia, only to find that the plane will be cumbersome and expensive. I send the quote, adding that going by train will be cheaper and faster, an assessment shared by the organizer.<br />
<br />
February. The summer school asks me for a quote of the train ticket. This is not possible yet, as the ticket can only be booked three months in advance. I give an approximate price based on next month's connections.<br />
<br />
March. I send title and abstract and state my availability for the lecture (Tuesday or Thursday). Everything is fine.<br />
<br />
April 17. The summer school produces a flight itinerary and asks me to confirm the booking. <i>Flight?</i> I reply stating that we already agreed to have me travel by train; I again retrieve and provide a quote for the train ticket. As the flight itinerary assumes that the talk be on Wednesday (which does not fit me), I ask that my talk be moved according to my availability. The organizers agree.<br />
<br />
April 17. In return, the summer school produces a train itinerary, again asking for confirmation. The itinerary also assumes my talk is on Wednesday. I cannot confirm the itinerary, repeating that the date does not suit me and that my talk date will be moved to either Tuesday or Thursday.<br />
<div>
<br /></div>
April 22. I get a tentative program, where my talk is scheduled on Wednesday morning. This is the day on which I am not available, and I curse between my teeth. I again ask for the talk to be rescheduled. The organizers apologize for their mistake and schedule my talk for Thursday.<br />
<br />
April 29. I get an electronic train ticket. This booking is still the same itinerary as on April 17, assuming my talk would be on Wednesday. I am pretty upset and state that the ticket neither fits my schedule nor the revised schedule of the summer school. Rather than exchanging more mails, I suggest I book the ticket myself, saving time and money.<br />
<br />
April 30. The organizers apologize for their confusion, and ask me to please book a ticket myself.<br />
<br />
April 30. I book a train ticket on the SNCF site. This takes me 10 minutes. I send the itinerary to the organizer, who again apologizes. All seems to be set.<br />
<br />
May. The administration asks all speakers for bank account information, address, and passport copy, which I all provide.<br />
<br />
June 5. I get a hotel voucher for my stay.<br />
<br />
June 27. The organizers ask all speakers for rechecking the information on their Web site. I am busy in a PC meeting and postpone the check for a few days.<br />
<br />
July 1. The organizers urgently ask me if it would be okay for me to chair a session on Wednesday morning. <i>Wednesday?</i> I will be traveling on Wednesday morning, so no. I remember the earlier message to recheck all information. It turns out that on the Web site, my talk <i>is still scheduled on Wednesday</i>, and my hotel is booked wrongly, too. As I realize this, I jump up and scream, literally banging my head against the wall; my secretary comes in and asks whether everything is okay.<br />
<br />
July 1. As I cool down, I write to the organizers. I repeat my travel details and for the last time, ask the organizers to reschedule my talk as confirmed multiple times. The organizers again deeply apologize for the confusion and promise that the schedule will be fixed.<br />
<br />
July 2. [Update] I cancel my participation, wishing all the best to the summer school, and bearing the ticket expenses myself. I feel relieved and relaxed. All is well that ends well.<br />
<br />
<b>Lesson learned #1: </b>While this trip has easily caused me more trouble than all my other trips combined, let me state that almost all of my business trips are painless, including all my trips to France so far. Also, other lecturers at the same event report that their booking all went well, so I guess I just had a long series of unfortunate events which is not typical for France at all.<br />
<br />
<b>Lesson learned #2:</b> If you organize an event, make sure you have a <i>single point of contact</i> – there should be precisely one person to talk to the invitees, who keeps track of all information and (possibly) interacts with the administration. Do not have your administration interact with the invitees directly, bypassing you; likewise, keep the administration updated at all times.Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com2tag:blogger.com,1999:blog-8747085902902510837.post-52870609484668083562014-05-13T09:30:00.001+02:002014-05-13T09:30:28.002+02:00Using Microcontrollers to teach Programming to EngineersThis Summer, I am exceptionally busy. I am teaching a course "Programming for Engineers" – a first-year course (twelve 90-minute lectures, plus exercises) for 120 engineering students who are to learn programming in C (and a bit of C++). In the past years, this has been one of the less popular courses both among its teachers and its students, so our Dean of Studies was very astonished and happy to have me volunteer for it.<br />
<br />
The reason I had volunteered is that earlier this year, my 12-year old daughter and I had toyed with an <a href="http://arduino.cc/" target="_blank">Arduino micro controller board</a>, writing simple programs that would cause LEDs to blink and buttons to be checked. Left on her own, my daughter (who had no C experience before) already had produced a full traffic light control for an intersection, together with a button to trigger a pedestrian crossing – and all of this within a few hours. The Arduino board served very well as a motivator: Controlling a set of LEDs and buttons is so much more concrete, tangible, and direct than printing "Hello, world" on a screen.<br />
<br />
As computer scientists, we tend to abstract away the concrete machinery – because computer science is the science of abstraction, getting rid of concrete details until all that remains is the pure beauty of computation. Unfortunately, this is not the mindset of engineers at all, who see programming not necessarily as an exercise in elegant design, but mainly as a tool to get their actual job done. A micro controller board is a great device to connect between the world of programming and the world of engineering; all of a sudden, everything you learn during programming becomes directly usable for engineering contraptions of any kind.<br />
<br />
So, I set off to build the programming course around microcontrollers. Intel donated us <a href="http://www.intel.com/content/www/us/en/intelligent-systems/galileo/galileo-overview.html" target="_blank">40 Galileo boards</a> (thanks so much!), which we fitted with electronic equipment such as LEDs, 16x2 LCD displays, photo resistors, microphones, speakers, breadboards and (many) cables – all of which would be used to motivate specific programming exercises. Each set would be shared by a group of four students. The process is that we give out a programming assignment each week, which each student solves and submits individually first (using only static checking, without executing them – welcome, <a href="http://en.wikipedia.org/wiki/Cleanroom_software_engineering" target="_blank">cleanroom software engineering</a>), followed by a group session in which the students jointly put together their programs to then test and demonstrate their final working solution on their wired board. Each student must be able to explain the group's program, so they get some exposure to teamwork and understanding programs, too.<br />
<br />
So far, it has been a great course. We set off with getting LEDs to blink, subsequently defined our own functions for sending Morse messages. Checking whether a button is pressed is surprisingly hard, as you have to introduce timer checks to handle possible contact bounce. Introducing multiple LEDs to display a voltage level led to arrays and loops; I used the final 20 minutes of the lecture to demonstrate how the Heartbleed bug works through buffer overflows. Later today, we will explore interactive automata, using the LCD display to select drinks from a soda machine. I don't know yet what will be up next week, but I reckon it is time to convey some algorithmic thinking, and by the end of the course, students will be left to their own devices to work on self-designed projects. From what I hear from tutors, students so far are very motivated and eager to learn. What else could you ask for?<br />
<br />
The only downside of this course is that it requires quite a lot of preparation. Before each lecture, I have to set up and wire my own microcontroller board, and it takes an extra ten minutes to set up and tear down the mounted camera to transmit live pictures of the board on my presentation; there's also plenty of live programming and interaction, which means I have to build and test all programs in advance. Giving out thousands of electronic pieces to students also was a logistic challenge (thanks to all tutors!). Furthermore, I have to build this course from scratch. Together with my other new course on "Generating Software Tests" (more on that in a later post), I am currently producing no less than 140 slides a week. So if you are eagerly waiting for my response, or wonder why I say "no" to almost everything these days – just keep in mind that this Summer, you are competing with 120 students who all await my latest and greatest.<br />
<br />
For those interested: <a href="http://www.st.cs.uni-saarland.de/edu/ping/" target="_blank">Here is the website of the course, with slides and all </a>(in German).Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com1tag:blogger.com,1999:blog-8747085902902510837.post-12337306068564517032014-01-20T08:15:00.000+01:002014-01-20T08:49:17.623+01:00Since I follow "Getting Things Done", I get more reminders, not lessAs a professor, I have many responsibilities and tasks to take care of, and many of these have deadlines. To keep track of them, I use an automated "To-Do"-Tracker, which allows me to enter new tasks (and ideas!) quickly, to assign categories and priorities to them, and to mark them as completed when I'm done. Most importantly, I can attach a "due date" to every task, and tell my tracker to automatically have a task pop up a few days in advance – typically the number of days I will need to complete it. So far, this works pretty well: When someone asks me whether I'd be available for a task, I can get a good assessment of my commitments (implying the answer is typically "no"); but if I commit to a task, it will <i>almost certainly be done</i> <i>by its deadline.</i><br />
<br />
Now, here's the fun thing. Since I use this approach, I get way <i>more</i> reminders than before. Journal editors tell me that my review is due in only five days. PhD students send me worried letters that I haven't sent out the referral letter yet, due on Friday. Conference chairs send me friendly reminders that I am an author, yet haven't registered for the conference, due in a week. Thank you for the reminders – actually,<i> I know all that,</i> as my to-do tracker nicely tells me so, and it is all factored in. Today, I need to work on this paper, this book chapter, this slide deck – all stuff which gets better the earlier it can be reviewed; and I know that the task you remind me off can still wait until due day, or the N days it will take me to complete it. In 2013, I completed 700+ tasks, and there was only one reminder which was effective (because I had mistyped the completion date).<br />
<br />
Overall, I believe that this "just in time" production makes me more effective, as it puts things in the right order (and the right priority). But then, it apparently strains the nerves of everyone else involved. Should I set up a rule to complete everything a week before it is due? Or should I set up a disclaimer of sorts?<br />
<br />
(And now back to my "To-Do"-tool: Five days left to complete this paper. Better start today...)<br />
<br />
<br />Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com2tag:blogger.com,1999:blog-8747085902902510837.post-31148230063498199302013-12-09T10:15:00.000+01:002014-01-20T07:42:37.810+01:00<h2>
Post-Doc Position with Andreas Zeller at Saarland University</h2>
<b>[Update 20-01-2014: The position is no longer available. Thank you! -- Andreas Zeller]</b><br />
<b><br /></b>
At the <a href="http://www.st.cs.uni-saarland.de/" target="_blank">Software Engineering Chair at Saarland University (Andreas Zeller)</a>, we have a post-doc position available for initially two years, starting March 1, 2014 or later. We are looking for applicants in all areas of Software Engineering, with a special focus on experience in<br />
<ul>
<li>Program analysis and testing</li>
<li>Specification and specification mining</li>
<li>Empirical software engineering</li>
<li>Security and privacy</li>
</ul>
<a href="http://www.st.cs.uni-saarland.de/" target="_blank">Our group</a> currently consists of one professor, two post-docs, and ten PhD students, all working on how to increase programmer productivity and program quality. We have a record of substantial impact at international venues, are well funded, and are fun to work with. We cooperate with researchers around the world, including companies like Google, Microsoft, or SAP. Our students are constantly looking for research challenges in their BSc and MSc theses.<br />
<br />
<a href="http://www.cs.uni-saarland.de/" target="_blank">Saarbrücken</a> is one of Europe's leading places in computer science research. It features Computer Science at Saarland University, two Max-Planck-Institutes for Informatics and Software Systems, the German Center for Artificial Intelligence, the Intel Visual Computing Institute, an Excellence Cluster in Multimodal Computing and Interaction, as well as the Saarbrücken Graduate School for Computer Science totaling more than 400 researchers. There are ample opportunities for local cooperation beyond the chair.<br />
<br />
English is the working and teaching language all over computer science. Germans generally have reasonable command of English; and in day-to-day life, you can easily get along with English as your only language. (You may wish to learn the German food names, though, to avoid surprises.)<br />
<br />
Mandatory conditions of employment are:<br />
<ul>
<li>PhD in Computer Science or Software Engineering</li>
<li>Fluency in English</li>
</ul>
Full-time positions are divisible in principle (§ 7 Abs. 1 TzBfG).<br />
<br />
Salary will be paid according to German TV-L.<br />
<br />
The university aims to increase the number of women in this field. Therefore, women are especially encouraged to apply for this position.<br />
<br />
Severely handicapped persons are preferentially employed in case of similar qualification.<br />
<br />
The position is available as of March 1, 2014. You should apply before December 22, 2013, although later applications may also be considered.<br />
<br />
Please send a detailed CV, including two referees, as a PDF document to:<br />
<br />
<a href="http://www.st.cs.uni-saarland.de/" target="_blank">Prof. Dr. Andreas Zeller</a><br />
<a href="http://www.cs.uni-saarland.de/" target="_blank">Saarland University – Computer Science</a><br />
<a href="mailto:office@st.cs.uni-saarland.de">office@st.cs.uni-saarland.de</a><br />
Tel.: +49 681 302-70970<br />
<br />
Applications have to include reference no. W793.Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-76578494407669970762013-10-22T03:42:00.000+02:002013-10-22T08:21:56.606+02:00Summarizing your presentation with miniature slides<div class="separator" style="clear: both; text-align: center;">
</div>
<h3>
The slide that drives me nuts</h3>
<div class="separator" style="clear: both; text-align: center;">
</div>
<span style="text-align: -webkit-auto;">As part of my job, I listen to talks a lot. There's good talks, and there's bad talks. But nothing can drive me as crazy as making a slide like <i>this</i> your final slide:</span><br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: -webkit-auto;">
</div>
<div style="text-align: -webkit-auto;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; font-weight: bold; margin-bottom: 1em; margin-left: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; font-weight: bold; margin-bottom: 1em; margin-left: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; font-weight: bold; margin-bottom: 1em; margin-left: 1em;"></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkJWUewJ6GExyOEnSB6m1TIzqxneUVRPOmsb1yeDUrCDC-KofcHMQ6RxluadQl75fjDcBbL3HmNvHsqd1Ku5mljefNg2FiCe0n_-R0KeTOBj6KTLfVpID8NDyRS0GstmA1ScKp6G4itiin/s1600/Bumblebee.001.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkJWUewJ6GExyOEnSB6m1TIzqxneUVRPOmsb1yeDUrCDC-KofcHMQ6RxluadQl75fjDcBbL3HmNvHsqd1Ku5mljefNg2FiCe0n_-R0KeTOBj6KTLfVpID8NDyRS0GstmA1ScKp6G4itiin/s320/Bumblebee.001.png" width="320" /></a></div>
<a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"></a><br />
<div style="margin-left: 1em; margin-right: 1em; text-align: left;">
<br /></div>
<div>
<div>
What's wrong with showing "Thank you!" at the end? Or "Questions?"? Or both? The problem is that most talks are followed by a discussion. With a slide like this showing up at the end, there's three problems:</div>
<div>
<ul>
<li>Parts of your audience will have lost contact during your talk, and wake up for the final slide only. With this showing up, they won't get the message.</li>
<li>The idea of the discussion is to get valuable feedback. But to ask questions, your audience will have to get the technical terms right. If they don't remember the exact name of your technique, they won't ask questions for fear of embarrassment.</li>
<li>If the discussion goes for five minutes, this is the single slide that shows up for the longest time. This is such a missed opportunity for delivering your message!</li>
</ul>
</div>
</div>
<h3>
</h3>
<h3>
The bullet summary style</h3>
<div>
A much better alternative is to simply <i>say</i> "Thank you" and to use the screen space for summarizing your talk. In standard bullet style, the slide would look something like this:</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="margin-left: 1em; margin-right: 1em;">
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
</div>
<div style="text-align: center;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdTEn6hmbtz3PCQnHsjuMEM8vcLgzzvXs42InRzcCR__Tt1IgMdVL7cNs_pic5y_Lwu3nb3ZVnT1G0kMzrh5Uaqy880pVHW3mgCXPTBkmY7A_Ttj2el29aQKE4-Xolqx9pe3nJ0VSdDAm3/s1600/Bumblebee.002.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdTEn6hmbtz3PCQnHsjuMEM8vcLgzzvXs42InRzcCR__Tt1IgMdVL7cNs_pic5y_Lwu3nb3ZVnT1G0kMzrh5Uaqy880pVHW3mgCXPTBkmY7A_Ttj2el29aQKE4-Xolqx9pe3nJ0VSdDAm3/s320/Bumblebee.002.png" width="320" /></a></div>
<br /></div>
</div>
<span style="text-align: left;">Your audience can now use the slide to ask questions ("Your... errr... </span><i style="text-align: left;">Bumblebee</i><span style="text-align: left;"> technique: If it is better for amblefuns, wouldn't it be worse for fumbleduns?"); people who have slept throughout your talk still get the message; and anyone who finds the discussion too special have all the time to enter the URL into the browser of choice to learn more about the project. Much better already.</span></div>
<div style="text-align: left;">
<div style="text-align: -webkit-auto;">
<div style="text-align: left;">
<div style="text-align: -webkit-auto;">
<h3>
</h3>
<h3>
Summarizing with miniature slides</h3>
</div>
</div>
<div>
<span style="text-align: left;">The "bullet style" summary still has issues, though. Many talks have diagrams that summarize processes or results. These won't show up on a bullet style slide, and as these invariably attract the most questions ("Why is it that this column is so high?"), you frequently have to switch back to earlier slides, which (a) takes time and (b) again takes the summary of your talk away from the audience.</span></div>
</div>
</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Here's my solution for the last slide. Take the four most important slides of your talk (say: problem, process, results, and discussion), and paste them together on one single slide. The result looks like this:</div>
<div style="text-align: left;">
<a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: left; float: left; font-weight: bold; margin-bottom: 1em; margin-right: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: left; float: left; font-weight: bold; margin-bottom: 1em; margin-right: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: left; float: left; font-weight: bold; margin-bottom: 1em; margin-right: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></a><br /></div>
<div style="text-align: center;">
</div>
<div style="text-align: center;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq2spta6xyTDwM4w1gpLx6UrqQLzou9xgESx1wSaa2TN7qc7y4VIUEJbYoQQfQxxaRiSXJqpSAb8wEfoekww86FptUaHCMFyPipWb9wP5vZe8NpPZn9-V620sP0s0Bqp5dXXh7kJQ_nIa0/s1600/App-Behavior.010.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjq2spta6xyTDwM4w1gpLx6UrqQLzou9xgESx1wSaa2TN7qc7y4VIUEJbYoQQfQxxaRiSXJqpSAb8wEfoekww86FptUaHCMFyPipWb9wP5vZe8NpPZn9-V620sP0s0Bqp5dXXh7kJQ_nIa0/s320/App-Behavior.010.png" width="320" /></a></div>
</div>
<div style="text-align: left;">
<br />
All these are slides that were shown earlier during the talk; hence, I can make it easy for my audience to recapitulate what I said. During discussion, I can simply point to the appropriate detail on the slide, and I <i>never</i> have to switch back to an earlier slide. Even if the text is hard to read at 50% of the original size, this does not matter much, as the audience will remember most from the talk anyway. I have been using this style for eight years in a row now, and it has worked ever since. Enjoy!<br />
<div>
<div style="text-align: center;">
<div style="text-align: right;">
<div style="text-align: center;">
</div>
</div>
</div>
</div>
<a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; font-weight: bold; margin-bottom: 1em; margin-left: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; font-weight: bold; margin-bottom: 1em; margin-left: 1em;"></a><a href="http://www.blogger.com/blogger.g?blogID=8747085902902510837" imageanchor="1" style="clear: right; float: right; font-weight: bold; margin-bottom: 1em; margin-left: 1em;"></a><br />
<b>Update 1:</b> How do I produce such miniatures? On a Mac, the process is straightforward. In Keynote or Powerpoint, select a slide and copy it to the clipboard. In Preview, use Command-N to open the clipboard, and Command-C, which copies it back again, but now as a bitmap. Go back to Keynote/Powerpoint, paste it in, resize and position appropriately. That's all!</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<b>Update 2:</b> Actually, it might have been me who invented this style of summary, or at least introduced it to the community. On December 6, 2005, I gave my first talk on "Model Mining", using miniatures on my final slide, and I don't think having seen this before. Anyone with earlier "miniature" slides? Please step up!<br />
<br />
<b>Update 3:</b> Here's my (or "the"?) first slide with miniatures from December 2005:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBuF5jnvzGfnRFCy_nQZQtLytTCq5Hb1-ONwMeTQ_ZYKLmwf6sDKPWzynDke6ZsrRuUX6NElvcq7dsdRMjlgx4aiv3DYYNFTcueD8gb4U1HIoJDO8S3qar5CoPfilBsc-sfwE_oXMXujs_/s1600/MiningModels.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBuF5jnvzGfnRFCy_nQZQtLytTCq5Hb1-ONwMeTQ_ZYKLmwf6sDKPWzynDke6ZsrRuUX6NElvcq7dsdRMjlgx4aiv3DYYNFTcueD8gb4U1HIoJDO8S3qar5CoPfilBsc-sfwE_oXMXujs_/s320/MiningModels.png" width="320" /></a></div>
<br /></div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com3tag:blogger.com,1999:blog-8747085902902510837.post-88816873226965601862013-09-11T11:49:00.002+02:002013-09-11T11:49:42.708+02:00Faster, better, stronger: The case for one-phase reviewsWhen running a scientific conference, one has to decide which submitted papers get published (and presented), and which ones are not. Each paper gets a number of reviews; and based on these, the program committee decides which papers get accepted.<br />
<br />
Flooded with submissions, several SE conferences went for a two-phase model to reduce workload: A paper first gets two reviews, and only if at least one of these favors acceptance, the paper gets a third review. As PC chairs of ASE 2013, we went back to a single-phase model, though. Several people have asked us why this was so. The answer is: time. With a one-phase review, you can cut the time required for reviews in half, thus allowing for more, better, and more recent papers.<br />
<br />
<br />
The math is as follows. Let's assume your conference gets 314 papers submitted (as we had for ASE):<br />
<br />
<ul>
<li>In a <i>two-phase</i> model,</li>
<ul>
<li> 314 papers would have gotten two reviews, totaling 618 reviews.</li>
<li>1/3 of papers are eliminated in the first round, leaving 210 which get a third review.</li>
<li>Total number of reviews: 618 + 210 = 828.</li>
</ul>
<li>In a <i>one-phase</i> model,</li>
<ul>
<li> 314 papers get three reviews, totaling 942.</li>
</ul>
</ul>
<br />
By having the one-phase model, your conference will thus have <i>14% more reviews</i> – in our case, 22–23 papers to review per PC member instead of 20. However, going for two phases requires a lot more administrative overhead – for the PC chairs, of course, but also for the reviewers. In particular, you need to allocate <i>two phases</i> in which to do the reviews – instead of, say, eight weeks, where reviewers would be free to allocate their load, you now have six + four weeks, each of which may be more easily blocked by travel, holidays, etc. Plus, as a PC chair, you need at least another week in between to reallocate and inform the new reviewers.<br />
<br />
All in all, you thus trade 14% more reviews against an extension of the review period by three weeks. Not necessarily a good thing. In addition, this assumes that papers are all equal; but they’re not: The 14% extra papers are mostly papers that would be quick rejections anyway. So, not much damage either.<br />
<br />
Assuming that <i>more</i> than a third of papers may be rejected in the first round favors the two-phase model. But then, there’s the concern that by having only two reviewers, you run a greater risk of one reviewer being incompetent, unwilling, lazy, etc. (It happens to anyone.) So you may be have to introduce <i>rebuttals</i> to compensate for potential misjudgments, and here go another two weeks of reviewing period. And then, someone will have to read these rebuttals, which adds more overhead.<br />
<br />
To illustrate how two-phase reviews and rebuttals eat up time in contrast to one-phase reviews, let's take a look at ICSE 2014, whose papers are due September 13, 2013. Notification is January 17 – that is, more than <i>four months</i> later. For ASE (one phase, no rebuttals), this was May 17 to July 25 – <i>two months</i> <i>and one week</i>, or roughly half the time ICSE takes. <br />
<br />
With its short reviewing period, ASE 2013 had more than three times the number of submissions compared to last year, which gave us a great choice of papers. ICSE will also have a record number of submissions; but if you miss the deadline, chances are you will find another conference which will publish your paper well before ICSE.<br />
<div>
<br /></div>
<div>
<div>
Summary: Get rid of frills, and allow authors six more weeks to prepare their submissions. You will get more, better, and more recent papers – and more, better, independent reviews at the same time.</div>
</div>
<div>
<br /></div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com3tag:blogger.com,1999:blog-8747085902902510837.post-46195996782497506322013-05-26T02:26:00.001+02:002013-05-26T02:26:04.139+02:00A Slide Generator for EasyChair<br />
If you chair a program committee with a physical meeting and use <i><a href="http://www.easychair.org/" target="_blank">EasyChair</a></i> as a conference management system, then the <i><a href="http://www.st.cs.uni-saarland.de/zeller/easyslide/" target="_blank">EasyChair Slide Generator</a></i> is your friend. This script<br />
<br />
<ul>
<li>generates slides structuring the discussion at the PC meeting</li>
<li>generates personalized discussion lists for each PC member</li>
<li>generates a conflict-free seating order for all PC members.</li>
</ul>
<br />
EasyChair Slide Generator comes as a Python script, tested on Mac and Linux systems. As input, it takes an EasyChair data export; see the included README.txt file for details. Enjoy!<br />
<br />
=> <i><a href="http://www.st.cs.uni-saarland.de/zeller/easyslide/" target="_blank">EasyChair Slide Generator</a></i><br />
<br />
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com0tag:blogger.com,1999:blog-8747085902902510837.post-91018653294494083092013-04-05T13:56:00.000+02:002013-04-05T14:07:38.364+02:00My top ten presentation issues in other's papers<span style="font-family: inherit;">I am a frequent reviewer of submissions for scientific journals and conferences. By far most of my reviews are specific to the individual paper, its contributions, and its issues. However, there is a small number of presentation problems which I have to report again and again. All these problem are very minor, and none of them will make me reject your paper. However, fixing them is a matter of minutes, and will make your paper an easier read for both readers and reviewers:</span><br />
<ol>
<li><b><span style="font-family: inherit;"><b>Abstracts without results</b><span style="font-weight: normal;">. Many abstracts end with the words "We conducted a study to show the effectiveness of our approach". There's three things wrong in here:</span></span></b></li>
<ol>
<li><b><span style="font-family: inherit; font-weight: normal;">You conduct a study to <i>evaluate</i> your approach – if you run your study with a predetermined outcome, I will suspect a flaw in your setup.</span></b></li>
<li><b><span style="font-family: inherit; font-weight: normal;">As a reader of your abstract, I am not so much interested in your process, but in your <i>results</i>; if you want cliffhangers, go and write TV shows instead. </span></b></li>
<li><b><span style="font-family: inherit; font-weight: normal;">You need results to sell your paper in the review process – in particular for outsiders who take only a brief glance at your paper.</span></b></li>
</ol>
<span style="font-family: inherit;"><span style="font-weight: normal;">So make this "Our study shows a 25% precision increase" or some other strong argument.</span>
</span>
<li><b>Multi-letter identifiers in <a href="http://en.wikipedia.org/wiki/LaTeX" target="_blank">LaTeX</a> math mode.</b> In LaTeX, <span style="font-family: Courier New, Courier, monospace;">$foo$</span> stands for the product of three variables f, o, and, o; and LaTeX typesets it as such. If you want one variable, use <span style="font-family: Courier New, Courier, monospace;">$\textit{foo}$</span>. (For a computer scientist, this is the single biggest design failure of TeX and LaTeX.)</li>
<li><span style="font-family: inherit;"><b>Confuse hyphens, minus signs, and dashes.</b> All these typeset into different characters: </span></li>
<ol>
<li><span style="font-family: inherit;">Hyphens: </span><span style="font-family: Courier New, Courier, monospace;">state-1 is blue, state-2 is red.</span></li>
<li><span style="font-family: inherit;">Minus signs: </span><span style="font-family: Courier New, Courier, monospace;">$\textit{bar} = \textit{foo} - 1$</span></li>
<li><span style="font-family: inherit;"><a href="http://en.wikipedia.org/wiki/Dash" target="_blank">En-dashes</a> (for ranges):</span><span style="font-family: Courier New, Courier, monospace;"> A car has 3--4~wheels</span></li>
<li><span style="font-family: inherit;"><a href="http://en.wikipedia.org/wiki/Dash" target="_blank">Em-Dashes</a> (for sentences): </span><span style="font-family: Courier New, Courier, monospace;">He said ``captain''---I said ``wot''. </span>(This can also be typeset using en-dashes with spaces, as in <span style="font-family: 'Courier New', Courier, monospace;">He said ``captain''~-- I said ``wot''</span><span style="font-family: Courier New, Courier, monospace;">.</span><span style="font-family: inherit;">)</span></li>
</ol>
<li><span style="font-family: inherit;"><b>Bad capitalization in <a href="http://en.wikipedia.org/wiki/Bibtex" target="_blank">BibTeX</a> references,</b> as in "[1] Jai: a javascript api ide" (should be "[1] JAI: A JavaScript API IDE") I could include this in all my reviews and be right 60% of the time.</span></li>
<li><span style="font-family: inherit;"><b>Omitting reference information.</b> I see papers omitting page numbers, conference names, editors, or even authors. I suppose that "[1] Smith et al., ICSE 2004" is something I should Google, and I hope there's just one Smith at ICSE. (This even occurs in papers whose venues have no page limit for references!)</span></li>
<li><b style="font-family: inherit;">Orphaned numbers,</b><span style="font-family: inherit;"> as in "I want 17<newline>foos." Tie numbers to items using non-breakable space (Word) or using a tilde (LaTeX): </span><span style="font-family: Courier New, Courier, monospace;">I want 17~foos.</span></li>
<li><span style="font-family: inherit;"><b>Orphaned lines and titles.</b> Section title at end of page 3, actual section on page 4. Better yet: a single line of text left under a table, such that you easily confuse content and caption. "LaTeX gets this wrong" is not an excuse; please fix this.</span></li>
<li><span style="font-family: inherit;"><b>Using monospacing with a non-monospaced font</b> (or vice versa). This seems to be a default setting of the LaTeX "<a href="http://en.wikibooks.org/wiki/LaTeX/Source_Code_Listings" target="_blank">listings</a>" environment – using Times fonts and arranging the letters at equal distance. Use monospaced fonts with monospacing, and proportional fonts otherwise.</span></li>
<li><span style="font-family: inherit;"><b>Omitting articles</b>, as in "Main feature is lack of articles". Now that spell checkers weed out the most blatant errors, the more subtle errors remain – and this is my #1 grammar issue. If your native language has no articles, take double care.</span></li>
<li><span style="font-family: inherit;"><b>Testing the limits.</b> Page limits are there for a reason – not to save trees, but to save the time of your readers by requiring you to focus on the main points. Cramming every little detail into the paper will decrease readability and alienate your readers; it may also lead to desk rejection if you alter the format. Stick to standard style, cut down your paper to one page less than the limit and have your paper proof read by others, then add whatever your proof readers missed.</span></li>
</ol>
<div>
<span style="font-family: inherit;">All these issues can be fixed through simple proofreading, and leave your readers and reviewers more time to enjoy the actual content. And now that the syntax is done, let's get back to semantics!</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">(As a reviewer or reader, what are your favorite gripes? Use the commenting function below.)</span></div>
Andreas Zellerhttp://www.blogger.com/profile/02016277079276068582noreply@blogger.com4