Discussion:
Bug#886120: doxygen = scapegoat ?
Paolo Greppi
2018-03-14 20:34:36 UTC
Permalink
@kilobyte, let me see if I can understand your purpose with this bug report.

What happened is:
1) you find a serious bug in ctpp2 (popcon: 14)
2) you decide it's RC
3) it's doxygen fault so you reassign it to doxygen (popcon: 8381), and it stays RC

Now let's look at the effect of all this:
1) doxygen is currently orphaned (https://bugs.debian.org/888580)
2) therefore nobody will take care of this unless someone adopts it
3) the average Johanna trying to decide whether she should adopt it will be repelled by this RC bug and the perspective that anybody anytime can file RC bugs because any obscure package fails to build and it's doxygen fault
4) consequently doxygen will not make it to buster
4) ctpp2-doc is also dropped from stable, and all other packages that depend on doxygen for building, which if I am not mistaken are 443:

grep-dctrl -FBuild-Depends doxygen -sPackage /var/lib/apt/lists/*Sources | wc
443 886 8612

Is that your wish ?

Paolo
Adam Borowski
2018-03-14 21:41:53 UTC
Permalink
Post by Paolo Greppi
@kilobyte, let me see if I can understand your purpose with this bug report.
1) you find a serious bug in ctpp2 (popcon: 14)
2) you decide it's RC
3) it's doxygen fault so you reassign it to doxygen (popcon: 8381), and it stays RC
1) doxygen is currently orphaned (https://bugs.debian.org/888580)
2) therefore nobody will take care of this unless someone adopts it
3) the average Johanna trying to decide whether she should adopt it will
be repelled by this RC bug and the perspective that anybody anytime can
file RC bugs because any obscure package fails to build and it's
doxygen fault
4) consequently doxygen will not make it to buster
4) ctpp2-doc is also dropped from stable, and all other packages that
grep-dctrl -FBuild-Depends doxygen -sPackage /var/lib/apt/lists/*Sources | wc
443 886 8612
Is that your wish ?
Your reasoning is: https://en.wikipedia.org/wiki/Appeal_to_consequences
which is not a reason for dismissing a bug. It _may_ be a valid reason
for the Release Team to mark the bug as buster-ignore, or to temporarily
reduce its severity, but this won't make it disappear.

The problem is not that random packages sometimes fail to build (not just
ctpp2, but obscure races are pretty hard to reproduce and/or debug). The
problem is that packages _succeed_ where they shouldn't.

Doxygen turns bugs that usually would be obvious, easy to fix breakage into
random rare failures that appear to build the vast majority of the time.

And, packages that have enough impact (such as doxygen) are already immune
to autoremoval, which addresses your 4) and 4).

Because of ignoring the failures, we don't know how widespread problems like
misgenerated output that's then (correctly) rejected by graphviz, are.

Only then we'd be able to correctly assess reasonable severities to
individual bugs. Here, doxygen causes at least one package to FTBFS which
might or might not be RC (we'd know more only once it's no longer leakily
papered over).


Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄⠀⠀⠀⠀ A master species delegates.
Paolo Greppi
2018-03-16 10:16:41 UTC
Permalink
With my previous message I was trying to understand your purpose with your bug report, not to demonstrate anything nor to dismiss it.

Adding the new info you provided, the chain of events triggered by your actions would be:
1) = as above
2) = as above
3) = as above
4) since doxygen is a “key package”, when the times are ripe the Release Team will likely mark this bug as buster-ignore
4) (let’s stick to this peculiar numbering system) but at that point ctpp2-doc is still FTBFSing so its maintainer will get back the hot potato, last minute.
Please confirm I got it right this time.

Additionally, I would appreciate your comments on this:
https://salsa.debian.org/paolog-guest/doxygen/wikis/home

P.S. FYI I intend to adopt this package, but I haven't yet.
Agustin Henze
2018-12-01 15:17:13 UTC
Permalink
Control: tag -1 +moreinfo +unreproducible
Control: severity -1 normal

Hi Adam,

I have been trying to reproduce the FTBFS of the cttp2 package (more than 20
times) you are saying without success. I've used the following script to do that

#!/bin/bash

set -e
count=0

while true; do
sbuild -A -s --force-orig-source -j4 -d sid-amd64 ./ctpp2-2.8.3
count=$((count + 1))
done

function finish {
echo $count
}
trap finish EXIT

For this reason I'm changing the severity, feel free to provide more
information about the FTBFS and what environment are you using to get them.

Also, please find more answer below

On Fri, 2 Feb 2018 02:45:00 +0100 Adam Borowski <***@angband.pl> wrote:
[...]
As I don't see anything obvious in the way ctpp2 calls doxygen + graphviz,
there's no point in having the bug stay here. The culprit here is doxygen
as it's where the syntax errors come from.
Generating calError: /<<PKGBUILDDIR>>/doc/html/a00332_a3c04138a5bfe5d72780bb7e82a18e627_cgraph.dot: syntax error in line 67 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a00332_a3c04138a5bfe5d72780bb7e82a18e627_cgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a00332_a3c04138a5bfe5d72780bb7e82a18e627_cgraph.svg"'
Running Error: /<<PKGBUILDDIR>>/doc/html/a00603_a6824947e6525f4afddb3bfa083f950d9_icgraph.dot: syntax error in line 8 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a00603_a6824947e6525f4afddb3bfa083f950d9_icgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a00603_a6824947e6525f4afddb3bfa083f950d9_icgraph.svg"'
... and many others.
I could reproduce them, and I think it's not a doxygen job ensure that you
write correct `dot` syntax in your comments. Doxygen process its commands
(that's all) and for the strings you have to ensure they are accepted by dot
later. At least, I couldn't find anything about Doxygen will escape the strings
for you in the documentation. But maybe it was me reading too quickly. For this
reason I think we should close this bug, because you have to forward these
issues to upstream to get the strings fixed in the documentation. I've tried
fixing one of them and it worked, so...
Please if you think we should need digging into it, provide more information
about why do you believe that Doxygen should be the responsible for sanitizing
the strings in the code. Otherwise I'll close this bug soon.

Cheers,
--
TiN
Debian Bug Tracking System
2018-12-01 15:27:05 UTC
Permalink
Post by Agustin Henze
tag -1 +moreinfo +unreproducible
Bug #886120 [doxygen] makes ctpp2 randomly FTBFS, syntax errors, hides problems
Added tag(s) moreinfo.
Bug #886120 [doxygen] makes ctpp2 randomly FTBFS, syntax errors, hides problems
Added tag(s) unreproducible.
Post by Agustin Henze
severity -1 normal
Bug #886120 [doxygen] makes ctpp2 randomly FTBFS, syntax errors, hides problems
Severity set to 'normal' from 'serious'
--
886120: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=886120
Debian Bug Tracking System
Contact ***@bugs.debian.org with problems
Adam Borowski
2018-12-02 03:02:47 UTC
Permalink
Post by Agustin Henze
Control: tag -1 +moreinfo +unreproducible
Control: severity -1 normal
Hi Adam,
I have been trying to reproduce the FTBFS of the cttp2 package (more than 20
times) you are saying without success. I've used the following script to do that
That's the whole problem! The FTBFS is random -- comes and goes; on current
unstable I can't seem to reproduce the build failure on either the amd64
machine nor the armhf one; when something else changes the issue will come
back. Whether it ends up in a build failure or not depends on _something_
that might be phase of the moon, some race inside the build that
semi-reliably gets won by one of the sides for a given state of the
toolchain, etc.

But that's only a symptom, not the cause.

The cause reproduces reliably:

Running dot for graph 763/1263
Running dot for graphError: /<<PKGBUILDDIR>>/doc/html/a01347_a236ee5e58731c059affd0e50b4d0aee4_cgraph.dot: syntax error in line 18 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a01347_a236ee5e58731c059affd0e50b4d0aee4_cgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a01347_a236ee5e58731c059affd0e50b4d0aee4_cgraph.svg"'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a01347_ab353c212082bbd71bd9c3c69f136f3bf_cgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a01347_ab353c212082bbd71bd9c3c69f136f3bf_cgraph.svg"'
Error: /<<PKGBUILDDIR>>/doc/html/a01347_a39915107bcf5ac712b6a28163a2653e2_cgraph.dot: syntax error in line 14 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a01347_a39915107bcf5ac712b6a28163a2653e2_cgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a01347_a39915107bcf5ac712b6a28163a2653e2_cgraph.svg"'
Error: /<<PKGBUILDDIR>>/doc/html/a01347_ad9290c6979f4d804ed33483beb548c14_cgraph.dot: syntax error in line 34 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a01347_ad9290c6979f4d804ed33483beb548c14_cgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a01347_ad9290c6979f4d804ed33483beb548c14_cgraph.svg"'
Error: /<<PKGBUILDDIR>>/doc/html/a01359_ae99959d42c4f7f886dd420aa119b29e2_cgraph.dot: syntax error in line 75 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a01359_ae99959d42c4f7f886dd420aa119b29e2_cgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a01359_ae99959d42c4f7f886dd420aa119b29e2_cgraph.svg"'
Error: /<<PKGBUILDDIR>>/doc/html/a01359_a98233ab4534754a671fae8ab588ef5e7_cgraph.dot: syntax error in line 49 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a01359_a98233ab4534754a671fae8ab588ef5e7_cgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a01359_a98233ab4534754a671fae8ab588ef5e7_cgraph.svg"'
Error: /<<PKGBUILDDIR>>/doc/html/a01359_aa0295dae085e2be96737ed0fd19ad714_cgraph.dot: syntax error in line 49 near '-'
(and so on)
Post by Agustin Henze
For this reason I'm changing the severity, feel free to provide more
information about the FTBFS and what environment are you using to get them.
The proper severity is RC: silently ignoring errors here:
* is a violation of a "must" clause of the Policy
* notoriously wastes maintainers' and QA's time dealing with heisenbugs like
this FTBFS of ctpp2
Post by Agustin Henze
Also, please find more answer below
[...]
As I don't see anything obvious in the way ctpp2 calls doxygen + graphviz,
there's no point in having the bug stay here. The culprit here is doxygen
as it's where the syntax errors come from.
Generating calError: /<<PKGBUILDDIR>>/doc/html/a00332_a3c04138a5bfe5d72780bb7e82a18e627_cgraph.dot: syntax error in line 67 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a00332_a3c04138a5bfe5d72780bb7e82a18e627_cgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a00332_a3c04138a5bfe5d72780bb7e82a18e627_cgraph.svg"'
Running Error: /<<PKGBUILDDIR>>/doc/html/a00603_a6824947e6525f4afddb3bfa083f950d9_icgraph.dot: syntax error in line 8 near '-'
error: Problems running dot: exit code=1, command='dot', arguments='"/<<PKGBUILDDIR>>/doc/html/a00603_a6824947e6525f4afddb3bfa083f950d9_icgraph.dot" -Tsvg -o "/<<PKGBUILDDIR>>/doc/html/a00603_a6824947e6525f4afddb3bfa083f950d9_icgraph.svg"'
... and many others.
I could reproduce them, and I think it's not a doxygen job ensure that you
write correct `dot` syntax in your comments. Doxygen process its commands
(that's all) and for the strings you have to ensure they are accepted by dot
later. At least, I couldn't find anything about Doxygen will escape the strings
for you in the documentation. But maybe it was me reading too quickly. For this
reason I think we should close this bug, because you have to forward these
issues to upstream to get the strings fixed in the documentation. I've tried
fixing one of them and it worked, so...
As far as I know (but I don't know doxygen), no other tool produces these
strings, thus they come from doxygen and thus need to be sane. If I'm wrong
and it's some other package what's the culprit, please reassign.
Post by Agustin Henze
Please if you think we should need digging into it, provide more information
about why do you believe that Doxygen should be the responsible for sanitizing
the strings in the code. Otherwise I'll close this bug soon.
Closing this bug (or keeping it at an ignorable severity) means we'll run
into this very problem again for some other random package. And waste time
once again.


Meow!
--
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ Ivan was a wordly man: born in St. Petersburg, raised in
⢿⡄⠘⠷⠚⠋⠀ Petrograd, lived most of his life in Leningrad, then returned
⠈⠳⣄⠀⠀⠀⠀ to the city of his birth to die.
Paolo Greppi
2018-12-02 11:13:01 UTC
Permalink
Control: reassign 886120 ctpp2
Control: tags 886120 patch

Here is a patch that fixes the root cause i.e. the wrongly escaped docstrings.
This avoids the intermittent, unreproducible build failure.

Bye bye,

Paolo
Debian Bug Tracking System
2018-12-02 11:21:07 UTC
Permalink
Post by Paolo Greppi
reassign 886120 ctpp2
Bug #886120 [doxygen] makes ctpp2 randomly FTBFS, syntax errors, hides problems
Bug reassigned from package 'doxygen' to 'ctpp2'.
Ignoring request to alter found versions of bug #886120 to the same values previously set
Ignoring request to alter fixed versions of bug #886120 to the same values previously set
Post by Paolo Greppi
tags 886120 patch
Bug #886120 [ctpp2] makes ctpp2 randomly FTBFS, syntax errors, hides problems
Added tag(s) patch.
--
886120: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=886120
Debian Bug Tracking System
Contact ***@bugs.debian.org with problems
Loading...