Author Topic: Potential vs Confirmed Bugs (Read 20068 times)

Steve Walmsley · « **on:** May 02, 2020, 04:39:14 PM »

This is a point for discussion, rather than anything set in stone.

I'm reaching the point where a significant number of bug posts aren't bugs, but rather working as intended or decimal separator related or other known issues, plus reports on what could be real bugs but have very little information in the post. While I am working through these and providing feedback, it is taking time that could be spent on 'real' bugs or adding new functions, plus generating some frustration on my part

I am wondering whether to setup two separate bug threads; a 'Potential Bug' thread and a 'Confirmed Bug' thread. Confirmed bugs would be those that need me to focus on finding and fixing them, so I would only work through the 'Confirmed' thread.

The tricky part would be the mechanics of moving bugs from Potential to Confirmed. One option would be locking the Confirmed thread except for a set of 'Bug Moderators', which would be volunteers. Their role would be filtering out WAI, separators, asking for more detail/clarification, etc. and only passing on bugs that I need to work on. Another option is trusting people to only post in Confirmed after they posted in Potential and received confirmation from others that is was a bug. Option 2 is more flexible, but based on content of the current bug thread I can see a lot of people not reading the header post and simply posting everything in Confirmed.

I am open to other options for Potential vs Confirmed or other completely different options that solve the same problem.

Caplin

For your sanity, I favor option 1.

As a QA person, I definitely recognize the problem with bug reports which don't provide sufficient information. Some kind of triage to ensure that bugs you work on have undergone a minimum of vetting sounds like a good idea.

Lightning · « **Reply #2 on:** May 02, 2020, 05:35:20 PM »

I agree with option 1 as well

Black · « **Reply #3 on:** May 02, 2020, 05:46:25 PM »

I myself got confused with mechanics that was changed from how it worked in earlier version of C# Aurora and reported it as a bug. I didn't notice that it changed and I am frequenting the forum a lot.

So in current situation I believe you should go for option 1 and select some people that can filter through bug reports and try to replicate reported bugs.

Zincat · « **Reply #4 on:** May 02, 2020, 05:54:18 PM »

Absolutely option 1 Steve.
You cannot possibly expect people on the internet to be disciplined with bug reporting. While a large part will be, just as many will not and that can be disastrous for your time AND motivation.

Make two threads, appoint some volunteers, and never look into the potential bug reports thread ever again. Let the volunteers post verified bug reports into the confirmed bugs thread.

I say this as a person who has beta tested a lot for an open souce project. We used to do something like this in that project, and the developers' sanity was saved.
This way you can see and work only on confirmed and/or reproductible bugs, and adding new features. And of course, play the game for yourself! You said time and time again how much that's important to you. We don't want you to burn out you know

If someone reports a "bug" but cannot provide information, or guarantee that he's on the latest version etc, it's better if you don't see it. If it's a real bug someone else will report it in more detail sooner or later. If not, a volunteer can deal with the likely mistaken report.

EDIT: I would also forbid users to make new threads in the bug reporting sub-forum. Else some people will just make new threads for their bugs, which once again you don't need. Let those two threads created by you be the only ones used.

Froggiest1982 · « **Reply #5 on:** May 02, 2020, 06:13:38 PM »

Quote from: Steve Walmsley on May 02, 2020, 04:39:14 PM

One option would be locking the Confirmed thread except for a set of 'Bug Moderators', which would be volunteers. Their role would be filtering out WAI, separators, asking for more detail/clarification, etc. and only passing on bugs that I need to work on.

+1

Quote from: Steve Walmsley on May 02, 2020, 04:39:14 PM

that could be spent on 'real' bugs or adding new functions

I would't spend time on more features until the known bugs are fixed, to be honest. A couple of them are really annoying.

pwhk · « **Reply #6 on:** May 02, 2020, 06:58:28 PM »

Agreed to do option 1. As a software developer myself I know how frustrating non-bug reports are

Omnivore · « **Reply #7 on:** May 02, 2020, 07:20:12 PM »

Number one please. Otherwise current trend will probably worsen as Aurora is received by a wider audience.

Thanks!

Iceranger · « **Reply #8 on:** May 02, 2020, 07:37:09 PM »

I personally think the first is the better approach to streamline the bug fixing process.

Steve Walmsley · « **Reply #9 on:** May 02, 2020, 07:46:15 PM »

Thanks for the unanimous feedback. I will implement this for the next release onward to avoid confusion with the current release.

If you would like to be a bug-thread moderator please PM me. As a minimum you would need plenty of experience with the game and to have read the changes log all the way through

If you are an experienced tester that would be ideal. I appreciate the good intentions behind all bug reports, but if I am often asking for more information when you post reports, or I often reply that it isn't a bug, please wait until you have more experience before applying to be a moderator.

The role of the bug moderators will be to review posts in the 'potential thread', ask for more information if needed, seek confirmatory reports from other players if required and then make a post in the confirmed thread.

I will concentrate on working my way through the confirmed thread.

Erik, if you are watching, I'm not sure how to restrict posters to a thread so please let me know

Erik L · « **Reply #10 on:** May 02, 2020, 09:37:56 PM »

I can create a Bug Moderator group and play around with the permissions.

I don't believe the permissions can be locked to a specific thread or not, but rather to a forum. If that is the case, I can create a subforum for confirmed bugs.

Erik L · « **Reply #11 on:** May 02, 2020, 09:51:51 PM »

There is a new group. And a new board for Confirmed bugs. At this moment, it is only visible to me, Steve, moderators, and the as-yet-unpopulated Bug Moderator group. So bug reports would continue as normal, and the bug moderators will copy confirmed bugs into the confirmed bug forum, whether this is a single thread per version or one thread per bug. I'd suggest one per bug, then it can be locked and put into a changelog thread (by either Steve or a Bug Moderator).

Yezarul · « **Reply #12 on:** May 03, 2020, 12:09:51 AM »

It is a good idea. By having approved people move things from potential to confirmed, they can also conform it to a standard template that makes it easier for you to look through so things are faster/tidier. This would not only improve the likelihood of fixing things but your own mental state as well.

I'm not sure how hard it would be to implement via this forum but there could also be a template that people type the information into that is already set-up, rather than relying on people to type it in their forum post.

I would suggest too that the approved moderators have access to delete from the potential bug threads, that way duplicates could be removed. (IE Say I don't have the time to verify the bug myself in game but it has been posted multiple times. I could then remove it and someone else that has the time could look into it).

Spreading out the workload of verifying or gathering additional data benefits everyone.

v/r
Yez

Cosinus · « **Reply #13 on:** May 03, 2020, 06:01:06 AM »

May I suggest using a bug tracker? There are some open source versions out there and they have multiple advantages:
For example Dwarf Fortress uses the Mantis Bug tracker https://www.mantisbt.org/, which is amazing and has many features a simple forum thread does not have:

Ability to search for bugs easily.
Ability to mark bugs as confirmed, solved, duplicate, WAI, etc.
Ability to easily categorise bugs into Typos, crashes, display bugs, gameplay bugs, unintuitive behaviour, missing features, etc.
They provide multiple fields that need to be filled when reporting the bug, such as version number, reproducible status, etc. This standardises bug reports.
Ability to have comments on a specific bug, without having to search for them across multiple pages.

(see their bug tracker in action here: http://www.bay12games.com/dwarves/mantisbt/my_view_page.php)
This simplifies reporting and fixing bugs immensely:

Bug reporters can search for existing duplicate bug reports without looking though dozens of pages.
Moderators can close WAI bug reports, duplicates, stuff that is caused by decimal separator, so steve doesn't have to deal with it.
Steve can concentrate on bugs that are flagged as confirmed.
Players can see persisting bugs easily and can work around them.

Everybody wins.

Froggiest1982 · « **Reply #14 on:** May 03, 2020, 06:13:27 AM »

Quote from: Yezarul on May 03, 2020, 12:09:51 AM

I would suggest too that the approved moderators have access to delete from the potential bug threads, that way duplicates could be removed. (IE Say I don't have the time to verify the bug myself in game but it has been posted multiple times. I could then remove it and someone else that has the time could look into it).

I agree with this; actually, they could start from the reports from 1.0 at today as I am sure few bugs have not been sorted or may gone unnoticed. I am also sure Steve is keeping a list but while we are at it better do a full and complete report.

Re: Potential vs Confirmed Bugs