Downloaded 24 times
![TABLE II
AVICENNA DIAGNOSES VS. HUMAN DIAGNOSES
Bug AVICENNA diagnosis (using ISLa syntax [13]) Expert Bug Description
HeartBleed str.to.int(<length>) > str.len(<payload>)
“Attackers can send Heartbeat requests with the
value of the length field greater than the actual
length of the payload” [37]
grep.7aa698d3
exists <utf8> in <lc_all>:
<utf8> = "UTF-8" and
exists <ignore_case> in <general_options>:
<ignore_case> = "-i"
“If grep conducts a case-insensitive search (-i) on
an input that contains multibyte characters and the
locale is UTF8, then grep prints a match of incorrect
length.” [36]
grep.5fa8c7c9
exists <patterns> in <command>:
<patterns> = "’’" and
exists <utf8> in <lc_all>:
inside(<utf8>, <lc_all>) and
exists <fixed_string> in <cmd_1>:
inside(<fixed_string>, <cmd_1>)
“Searching with grep -F for an empty string in a
multibyte locals [sic] would freeze grep.” [36]
(Note: hfixed stringi expands to -F and
--fixed-strings)
grep.c96b0f2c
exists <regex_> in <patterns>:
<regex_> = "ˆ$" and
(exists <ignore_case> in <matching_control>:
inside(<ignore_case>, <matching_control>) or
exists <line_no> in <output_line_prefix_control>:
inside(<line_no>, <output_line_prefix_control>))
“Options -i and -n will not work when applied to an
empty line” [36] [in a UTF-8 locale]
(Note: hignore casei expands to -i and
--ignore-case; hline noi expands to -n
and --line-number)
“Core dump with pattern ’(ˆ| )*( |$)’” [36]
Some Benchmark Results
Subject AVICENNA Diagnosis Expert Diagnosis (from DBGBench)](https://image.slidesharecdn.com/language-basedtestinganddebugging-230717175442-3410a995/85/Language-Based-Testing-and-Debugging-pdf-35-320.jpg)
![grep.c96b0f2c
exists <regex_> in <patterns>:
<regex_> = "ˆ$" and
(exists <ignore_case> in <matching_control>:
inside(<ignore_case>, <matching_control>) or
exists <line_no> in <output_line_prefix_control>:
inside(<line_no>, <output_line_prefix_control>))
“Options -i and -n will not work when applied to an
empty line” [36] [in a UTF-8 locale]
(Note: hignore casei expands to -i and
--ignore-case; hline noi expands to -n
and --line-number)
grep.3c3bdace
exists <extended_regex> in <matcher_selection>:
inside(<extended_regex>, <matcher_selection>) and
exists <repetition> in <patterns>:
<repetition> = "*"
“Core dump with pattern ’(ˆ| )*( |$)’” [36]
[and -E option]
(Note: hextended regexi expands to -E and
--extended-regexp)
grep.3220317a
exists <bracket_expr> in <first_expression>:
inside(<bracket_expr>, <first_expression>) and
exists <utf_characters> in <bracket_char>:
inside(<utf_characters>, <bracket_char>)
“Segmentation fault on multibyte character
classes” [36]
(Note: hbracket expri expands to [...] in
a regular expression; hutf charactersi occur
within hbracket chari, i.e. the characters within
hbracket expri)
find.07b941b1
exists <match_opts> in <find_expression>:
<match_opts> = "-regex " and
exists <character_expr_no_minus> in <first_expression>:
<character_expr_no_minus> = "."
“find segfaults when using -regex, for instance
./find -regex ’.*’” [36]
find.091557f6
exists <file_properties> elem in <find_expression>:
<file_properties> = "-type f" and
exists <directory_name> elem_0 in <starting_dir_list>:
inside(<directory_name>, <starting_dir_list>)
“assertion failure on symbolic link loop:
Let’s say we accidentally create a symlink
loop $ mkdir tmp; cd tmp and
$ ln -s a b; ln -s b a and use find
to find files and follow symlinks inside the tmp-
folder: ../find -L -type f” [36]
Some Benchmark Results
Subject AVICENNA Diagnosis Expert Diagnosis (from DBGBench)](https://image.slidesharecdn.com/language-basedtestinganddebugging-230717175442-3410a995/85/Language-Based-Testing-and-Debugging-pdf-36-320.jpg)
![find.091557f6
exists <file_properties> elem in <find_expression>:
<file_properties> = "-type f" and
exists <directory_name> elem_0 in <starting_dir_list>:
inside(<directory_name>, <starting_dir_list>)
“assertion failure on symbolic link loop:
Let’s say we accidentally create a symlink
loop $ mkdir tmp; cd tmp and
$ ln -s a b; ln -s b a and use find
to find files and follow symlinks inside the tmp-
folder: ../find -L -type f” [36]
find.dbcb10e9
exists <digit> in <last_modified>:
inside(<digit>, <last_modified>) and
exists <numeric_arg> in <find_command>:
inside(<numeric_arg>, <find_command>)
“-mtime produces segmentation fault, e.g., ./find
-mtime 2” [36]
(Note: hlast modifiedi expands to -mtime)
find.ff248a20
exists <ln_file> in <ln>:
<ln_file> = "al_ln -s . link" and
exists <find_expression_or_empty> in <command>:
<find_expression_or_empty> = " -follow"
“infinite loop with -follow; e.g.,
$ mkdir testingfindagain;
ln -s . testingfindagain/symlink;
./find testingfindagain -follow” [36]
For details on all DBGBench bugs (including expert bug descriptions), visit https://dbgbench.github.io/.
For the HeartBleed description, see https://www.synopsys.com/blogs/software-security/heartbleed-bug/.
Some Benchmark Results
Subject AVICENNA Diagnosis Expert Diagnosis (from DBGBench)
At FSE 2023](https://image.slidesharecdn.com/language-basedtestinganddebugging-230717175442-3410a995/85/Language-Based-Testing-and-Debugging-pdf-37-320.jpg)
![Andreas Zeller • APR Launch Event • National University of Singapore • 2022-11-11
Joint work with Dominic Steinhöfel • Martin Eberlein • Lars Grunske
Semantic Debugging
Why and When Does My Program Fail? TABLE II
AVICENNA DIAGNOSES VS. HUMAN DIAGNOSES
Bug AVICENNA diagnosis (using ISLa syntax [13]) Expert Bug Description
HeartBleed str.to.int(<length>) > str.len(<payload>)
“Attackers can send Heartbeat requests with the
value of the length field greater than the actual
length of the payload” [37]
grep.7aa698d3
exists <utf8> in <lc_all>:
<utf8> = "UTF-8" and
exists <ignore_case> in <general_options>:
<ignore_case> = "-i"
“If grep conducts a case-insensitive search (-i) on
an input that contains multibyte characters and the
locale is UTF8, then grep prints a match of incorrect
length.” [36]
grep.5fa8c7c9
exists <patterns> in <command>:
<patterns> = "’’" and
exists <utf8> in <lc_all>:
inside(<utf8>, <lc_all>) and
exists <fixed_string> in <cmd_1>:
inside(<fixed_string>, <cmd_1>)
“Searching with grep -F for an empty string in a
multibyte locals [sic] would freeze grep.” [36]
(Note: hfixed stringi expands to -F and
--fixed-strings)
grep.c96b0f2c
exists <regex_> in <patterns>:
<regex_> = "ˆ$" and
(exists <ignore_case> in <matching_control>:
inside(<ignore_case>, <matching_control>) or
exists <line_no> in <output_line_prefix_control>:
inside(<line_no>, <output_line_prefix_control>))
“Options -i and -n will not work when applied to an
empty line” [36] [in a UTF-8 locale]
(Note: hignore casei expands to -i and
--ignore-case; hline noi expands to -n
and --line-number)
“Core dump with pattern ’(ˆ| )*( |$)’” [36]
Some Benchmark Results
Subject AVICENNA Diagnosis Expert Diagnosis (from DBGBench)
Features on Demand
"hello" ...
<length> <payload> <length> <payload>
<client-request> <server-response>
<exchange>
0x1 0x2000 ... 0x2 0x2000 "hello0secret0password0"
"37h%$"
0x1 0x5 ...
"0"
0x1 0x1 ...
"37h%$"
0x2 0x5 ...
"0"
0x2 0x1 ...
Learner
Test
Generator
Perfect conditions for automated learners
"${}"
0x1 0x3 ... "${}"
0x2 0x3 ...
""
0x1 0x0 ... ""
0x2 0x0 ...
"'; --"
0x1 0x5 ... "'; --"
0x2 0x5 ...
Test
generation
problem
/ / / / / / / / / / / / / / / / /
/ / / / / / / / / / / / / / / / /
Oracle
problem
/ / / / / / / / / / / / / / / / /
/ / / / / / / / / / / / / / / / /
Testing with Language Specs
<length> ::= <uint16>
<payload> ::= <byte>*
<padding> ::= <byte>*
<exchange> ::= <client-request> <server-response>
uint16(<length>) = len(<payload>)
<client-request>.<payload> = <server-response>.<payload>
Syntax
I/O Grammar
Semantics
Constraints
Testing Mocking
0x1 <length> <payload> <padding>
<client-request> ::=
0x2 <length> <payload> <padding>
<server-response> ::=
Monitoring
AVICENNA
Learning Failure Circumstances
Input
Language Spec
Failure
Circumstances
<client-request> ::=
0x1 <length> <payload>
uint16(<length>)
> len(<payload>)
Failing Inputs
"hello"
0x1 0x2000
Determine relevant
input elements
Determine failure-
related properties
Generate new inputs
to refine theory
Negate constraints
<length>
<payload>
<length> = 0x2000
<payload> = "hello"
<length> ≠ 0x2000
<payload> ≠ "hello"
"xyz"
0x1 0x0003
@AndreasZeller](https://image.slidesharecdn.com/language-basedtestinganddebugging-230717175442-3410a995/85/Language-Based-Testing-and-Debugging-pdf-41-320.jpg)
Language-Based Testing and Debugging.pdf
The document discusses the challenges and methodologies involved in language-based testing and debugging of programs, particularly focusing on test generation and oracle problems. It introduces a language to specify inputs and outputs, alongside a fuzzer and checker to ensure that inputs are valid and conform to specified constraints. Additionally, it highlights the importance of synthesizing language specifications from existing interactions to refine testing processes and improve program behavior understanding.
![Inside Python [OSCON 2012]](https://cdn.slidesharecdn.com/ss_thumbnails/insidepythonpresentation-120724001527-phpapp02-thumbnail.jpg?width=640&height=640&fit=bounds)
