What Nodes Go In Debate Trees? And Other Debate Tree Questions

This article answers several common questions about debate trees. It discusses mechanics of how trees work, addressing all criticism instead of excluding arguments from discussions, and how to protect your time.

Adding Nodes

When making debate trees, what nodes should be added? (Suppose you're the one with the tree document on your computer, and you're adding nodes for everyone.)

You should add any node that any participant in the debate wants to add. If one person wants to add a node, you should add it.

(Note: I have in mind primarily the kind of tree where you have a debate or critical discussion, and you speak freely, and you put your important points in the tree so they can be highlighted and organized. That way there's no need to answer every mistake or even argument that anyone says. If something you think is important isn't answered, your recourse is to add it to the tree. This two-tiered discussion system allows people to speak without stressing about it, and then be more careful only when adding nodes to the tree. It's an attempt to get the benefits of a free discussion without limiting how much you can say or trying to make every comment high quality, while also getting the benefits, via the tree, of having an organized discussion where every statement aims at high quality. Then the conclusion or outcome of the debate can be judged just from the tree by looking at what's refuted there.)

If you think a node is a bad idea and the person adding it would want to change their mind if they knew your criticism, then you can object, explain the problem, and suggest they don't add it. But you should say something like "If you disagree, we'll add the node. You don't have to debate whether or not to add it. I'm just trying to be helpful."

The most common reason to object to a node is you see an error that you think the person will immediately agree is an error after you point it out. And it's a boring error, like a typo, not an error that someone else might make. If you think there's an error that's more complicated, interesting or relevant to anyone else, then he (or she) can add it to the tree and you can add criticism to the tree.

The important concept here is there should be no gate keeping about what nodes are added. They don't have to pass peer review or get approval from anyone other than the person adding the node. The goal is to gather relevant arguments and refute many of them, not to find reasons to exclude arguments from the debate.

If there are conflicts about what nodes to add, then you're doing it wrong. If you're the person who wants to add a node and there's resistance from the person doing the tree, then I'd suggest you meta-discuss their debate policies or start making your own debate tree in your own software on your own computer.

You could also make trees with a website or app that lets multiple people edit the same document. Then everyone could add their own nodes themselves. I think this is a good idea if people are getting along well enough. If there's more conflict, then each person (who wants to) having full control over a tree on their own computer makes sense instead of trying to share a document.

State of the Debate Trees

What if you're reading the literature and making a tree about the state of the debate on some topic, rather than having a debate with one or more other people? Then what nodes should you add to the tree?

The short answer is: add any node that anyone suggests. The general concept here is you should not gate keep what arguments are included in a debate. Don't look for reasons or excuses to exclude arguments. Any argument, thought of by anyone, should be addressed. If you think it's wrong, put it in the tree then add a node refuting it. It's good for the tree to cover lots of mistaken arguments, and refute all of them, rather than leaving all that information out.

I'll discuss two potential problems here, clutter and high volume. These tend to be problems primarily when a large number of people are involved: either a debate with many participants or a large audience making suggestions. They shouldn't come up much in a more typical debate with only one other participant or a few participants.

Clutter

When dealing with lots of node suggestions, your tree could get cluttered. Good tree software will offer features like collapsing sections, and zooming in and out, so even huge trees can be reasonably manageable. But you may also need to make multiple tree documents. You can put what you think are the main points in one tree, and keep a separate tree where you answer a bunch of detail objections. That way each criticism can go in a tree, but not in every tree. Make as many trees as you need to cover all the arguments people are making without too much clutter in any one tree. For example, you can make a separate tree for each pattern of errors that comes up. Then in the main tree, you explain the pattern and why it's wrong, and give a link, cite, footnote or reference to the subtree with more details. Then the subtree can focus on just examples of that one pattern. Hopefully by the time it has 20+ examples, most of your audience will start to see the pattern better and stop suggesting more things that fit the same pattern.

Splitting a large tree into multiple tree documents doesn't change the tree conceptually. The documents together still contain the same nodes and connections (plus a few extra nodes specifying connections at document change points). You can still conceive of it as one big tree, while also organizing it into separate documents for convenience. You can imagine that future software would offer better organizational tools than multiple documents in one document.

If you're discussing in a small group, then the few people who disagree with you should hopefully say something new. If they don't, and they're still making the same few errors over and over, while you fill up subtrees, then you should consider that you might be doing something wrong and you're fallible. And you should try some meta discussion about what's going on. You should be patient and do quick, efficient ways to address many arguments instead of trying to keep the number of arguments low. But at some point you'll also want to consider not talking with some people directly anymore, if they're very repetitive about stuff you already answered, unless they can meet some conditions that you have in a written debate or paths forward policy. Also, if you are having these kinds of issues with a few people out of many that you talk with, maybe that's OK. But if you're having severe issues with the majority of people you talk with, that means you're likely doing something wrong.

Volume

What if there are suggestions to add thousands of nodes? This could happen if you're famous. In that case, you can't add them all. You'll need some policies for what to spend time on which do a reasonable job of still having paths forward, and not ignoring ideas due to bias, while also not paying attention to everyone. For example, you might respond to some famous people, some highly upvoted people, some random people, and some people of your choice. You might do one hour per week for each of those, or whatever other time budget works for your situation. That way you have only partial control over which points to answer, so your ability to avoid difficult arguments you're biased about is limited. I made some more suggestions in my essay Using Intellectual Processes to Combat Bias.

Note that I mentioned responding to randomly selected people, not randomly selected arguments. Why? You don't want an incentive for a person who wants attention to submit dozens of arguments to have more chances to get your attention. Basically, one lottery ticket per person is less prone to abuse than letting someone buy more tickets by making more arguments. You could also consider giving people more chances for attention based on effort they put in, e.g. each book they read and write a blog post about gets them an extra chance.

Also, basically, do your best to answer a wide range of arguments, and then ask people to do their best to not submit duplicative or repetitive points. And ask them to upvote or otherwise bring to your attention whatever points you haven't answered yet. With some good faith from the community, you can do a reasonable job of finding every unique issue and adding it to your tree. You do want to answer every argument that actually hasn't been refuted yet. The problem is if people keep bringing up things you already addressed, repetitively, then that would take too long. But as long as you're answering new points, then you should keep answering every criticism that anyone can come up with.

If you don't have refutations of all criticism, then you should not be confident in your claim. Admit that you don't know. Consider the issue inconclusive for now. It's unreasonable to reach a conclusion that X is true when you know there is a refutation of X which you have no refutation of.

If someone criticism has already been answered by someone else, you can use a citation instead of writing a new answer. You don't need to rewrite things that you're happy with. The important thing is to take responsibility for the correctness of each argument you use (and for them to exist in writing), not to personally write them all down.

Note that I'm discussing decisive refutations – arguments that actually contradict the thing they're criticizing. So if the argument is correct, then the thing it's refuting must be incorrect. It doesn't just try to weaken it; it isn't compatible with it being correct. Using indecisive arguments creates a lot more additional difficulties for organized and conclusive debate, which I won't address here. Actually, I won't address that anywhere: I'm not offering solutions for how to deal with indecisive arguments; I think they cause problems and people should stop using them. Decisive arguments are better and should be used exclusively. (Indecisive arguments are OK to mention informally. E.g. you might not see how to make a decisive argument related to an indecisive argument you thought of. You could explain your reasoning and ask for help formulating a good, decisive argument. And someone might be able to either help you do that or give a criticism that helps you see what's wrong with your reasoning.)

If you aren't famous and aren't talking on a forum with thousands of members trying to post on a topic, then you shouldn't be flooded with too many things people want to say. In that case, you should address every (decisive) argument that criticizes your claims – even if there are dozens of arguments. (Decisive arguments are ones that, if they are true, mean you're wrong. So they should be answered. Indecisive arguments are compatible with your ideas being true, so they don't require an answer. People may claim indecisive arguments make your ideas less likely to be true, which gets into complex epistemology issues – like credences or judging ideas by amounts of goodness – that I've discussed elsewhere.)

What if you can't do that? What if it's too hard, you don't know enough, it takes too long, etc? Then don't reach a conclusion. Don't claim you have a correct conclusion, which is knowledge that everyone should believe. Don't claim you're ready and able to debate doubters and win. Instead, just acknowledge that you don't know, don't have all the answers, are not done working through all the issues, etc. And if figuring a topic out is a work in progress where you know you aren't done, then you should be neutral, not biased towards a particular conclusion.

What if people make repetitive, incorrect arguments? Then find the pattern – the similarity between each argument – and give a criticism that applies to any argument fitting the pattern.

Criticizing types of arguments, instead of criticizing each argument one by one, is one of the most important debate skills. If debate takes too long because you aren't able to do this effectively, the solution is to improve your skills. Refuting patterns is crucial to being able to deal with lots of criticism effectively and efficiently. It's a huge time saver. Being able to think about and deal with patterns is necessary to conceptual thinking in general, and being able to do it well with debate arguments is necessary to making debates productive. (You can also think of it in terms of making an argument that refutes a whole category of ideas. Dealing with categories or patterns is fundamentally the same thing, but people often think of them a bit differently, so knowing both concepts is useful.)

Sometimes you'll criticize a pattern and then someone will make an argument that fits the pattern anyway. (They aren't arguing with your criticism. They're just saying things already covered by it.) They didn't realize their idea fit the pattern, or they were sloppy, or maybe it was even in bad faith. What do you do? Point out it fits the pattern. But don't rewrite your criticism of the pattern. Just give a link to the argument you already made (or with a tree, you can write a reply node giving a reference/citation/footnote to the refutation, which happens to be in another node in the same tree). Giving a link to where you already wrote down a refutation that applies to what someone is saying is short and fast.

If you have a really big audience, then even giving links or references to already-arguments could be too time consuming, but most people can just do this every single time and it still won't take very long. Have some patience and try it for a while before you start trying to avoid it. If you do have a big audience, you should encourage your audience to start doing this for you. You can't trust your audience to debate for you, but you could ask them to link to arguments you already made when they're relevant, and they should be able to do a decent job of that. If they can't do that, then maybe you're not explaining things well enough and you have all these fans for the wrong reasons. If they don't understand your arguments enough to even link the right ones in response to common, repetitive errors – when you explicitly and directly explained the pattern the error fits and why it's wrong – then you're not being very educational for them. Educating people is hard and it's totally understandable for it not to go well, but if you're super popular while your audience isn't even learning your points enough to use links competently in repetitive cases, then your popularity is coming from some other cause besides education.

If you have a big audience, there will be people who just don't get it, and I won't blame you for that. If 20% of your audience is clueless, that's reasonable (whereas 50% would be a sign you're doing something wrong). With a big enough audience, even 1% clueless people could be thousands of people – plenty to make a ton of bad, repetitive arguments. But the best 20% of your audience is also a lot of people, and they should be able to provide a lot of links and answer the most basic questions and criticisms, so the stuff you personally have to deal with is reduced a lot.

If there's still too much for you to deal with, a good solution is some written policies that allocate some time, on a regular basis, as discussed in my essay Using Intellectual Processes to Combat Bias. Do the best you can with your available time but don't spend more time than that. And be transparent about what's going on and why. This is a rare situation to be in, but you may see some public intellectuals with large audiences, and notice they handle it very differently than these suggestions; you could consider why they do that and what the consequences are.

Removing Nodes

Should I ever remove nodes from a debate tree?

The short answer is no. We're trying to document arguments and their refutations. More is better. Why would you remove an argument? If it's wrong, put a refutation under it. Having the wrong argument plus the refutation in your tree helps prevent other people making the same wrong argument. It's educational for readers and it preemptively prevents that incorrect argument from being made. And if someone does make the same wrong argument anyway, you already have the refutation in the tree, so you won't have to rewrite it. Removing bad ideas means removing the refutations of them too, since it wouldn't make sense to keep the refutation without the thing it refutes. If you delete a node, you'd delete all its children, grandchildren and further descendants, because they're no longer attached to the tree. That will lead to more rewriting.

Similarly, you shouldn't edit tree nodes in general.

However, you could remove or edit a node with unanimous consent of everyone in the debate.

Adding a node requires one person to want to, but removing or editing a node requires that everyone wants to.

What you can do more often is move nodes. Suppose there are five arguments, and you refute them all, recognize a pattern, and then also refute the pattern. Then it could be reasonable to move the five instances of the pattern to go under the node explaining the pattern (which didn't exist yet when they were first added to the tree, so they can't already be under it). If the tree is getting too cluttered, you can also move some stuff into additional trees in separate documents. Moving nodes should also be done with unanimous consent.

If you're making a state of the debate tree by yourself – and allowing comments from the public but not specifically discussing or debating with some particular people – then you get more control over the tree. You can't please everyone in the public and you do need to organize. So you might e.g. move stuff as long as 10% or more of your audience doesn't object. You can't use unanimous consent as a standard when dealing with big groups of people. You should usually move not delete things, and keep edits minimal (like fixing a typo is fine, but don't change the meaning of a node – if you want to a new meaning then make a new node). Don't use edits or deletes to try to hide ideas you think are dumb, nor to try to remove or fix your own past mistakes. Leave your mistakes in the tree and, when you learn they're mistaken, add refutation nodes below them. Also, it's good to keep version history of trees. Before changing stuff, save a copy. And leave old versions of trees on the internet, available for anyone who prefers them, rather than getting rid of them. The point of making organizational changes is to have a useful tree that's easier to work with, not to hide any discussion history, any past errors, etc. Basically, it can be OK to organize and archive stuff that you think is no longer actively useful, but it's bad to fully delete stuff or try to rewrite or hide history.

Don't try to hide your mistakes, and don't try to hide other people's mistakes or prevent other people's mistakes from getting into trees.

Last Word

What if someone keeps adding new low-quality nodes to try to get the last word?

If they keep making new arguments that aren't already refuted by text already in your tree (or cited/linked/referenced by your tree), that's good and helpful. Even if their arguments are dumb and wrong in some sense, they're showing you ways that your arguments are incomplete. They're helping you make a more complete case. When the quantity of arguments is high enough to be problematic, the solution is finding and responding to patterns instead of answering every argument individually. Getting more examples that push you to realize your current pattern-based arguments are incomplete can help you improve the patterns by making them broader, or sometimes help you find a whole new pattern that you were missing.

What if they repetitively make arguments that are already refuted? That might be bad faith. It might be bias, incompetence, ignorance, irrationality, etc. One thing you can do is answer five arguments in a row by pointing out they're already refuted in the tree. (This might still work with fewer arguments or with not all of them being consecutive.) Next, point out the pattern in their contributions to the tree. Instead of answering their new arguments individually, you can speak to the pattern of how they keep creating pre-refuted arguments. You can suggest ways they can make progress and avoid that mistake, e.g. by studying and practicing. Or maybe they need some introspection and post mortem effort to understand what's causing this. You can discuss what methods you think they're using, and criticize those, and discuss what methods you think would be productive. You can argue about methodology instead of continuing the original debate. Another way to view this is when the object level of the debate is at an impasse, then you can move to a meta level, and then it won't be keep repeating the same points because the topic has changed. Methodology errors are a type of pattern that can be criticized.

I advise having lots of patience with arguments you think are dumb. But you can watch out specifically for someone repeatedly making arguments that are already refuted in the tree – that one specific type of being dumb can be differentiated from other types of (alleged) dumbness and dealt with merely patiently instead of extremely patiently. Keep in mind it's common that people say "I already answered that" when they didn't. Even if they give a link/reference to something specific (which they often don't), it's often wrong: they answered something different and failed to notice the difference between it and what you said. Keep in mind your fallibilism and don't be surprised if someone points out an error of yours after you claim you already answered something. But if they say pre-answered stuff over and over, while never disputing that it really was pre-answered, then you'll need some way to limit repetition.

It's fine if someone makes pre-refuted arguments a couple times and you point them in the right direction (with bigger trees, it's reasonable for someone to ask about an issue without fully reading the entire tree). This especially comes up with citations: people can't be expected to read every citation and know what's already answered in them (when you give a cite you should say what it's for so people know the main point and maybe a brief summary without having to go read it, but they won't know every detail covered in each cite). So if someone brings up an argument that's already answered in a book you cited, that's fine: just let them know which book answers it, and give some specifics like the chapter, section or page. If they go read the book and then try to criticize it, that's great. If they don't read the book and make another argument which is answered in a different book you cited, that might be OK, but if they keep making more pre-refuted arguments five times without reading any of your literature that sounds like a methodology problem. (If they read the first book and concede that point, then make a new argument addressed in other literature, that's good again. They aren't just throwing arguments at you carelessly. Progress is happening.) If they are making arguments that you think are already addressed directly in your tree, and they do that several times, then you need to consider that your arguments are less clear than you thought. Or, in the alternative, something is going wrong with their reasoning skills, effort, good faith, literacy or something else.

Please try really hard not to tell people to go read some book, which allegedly answers them, when it doesn't. That is a common. I've often been directed to literature and, after searching through it, found that it was irrelevant. It never tries to address the point I was making. A way to avoid this problem is referencing a specific section of text (say both start and end points for your reference), including a quote, and giving a very short summary of what the cite says and how that refutes the idea the person brought up. With references like this, if you get it wrong, they can often point out the problem immediately based on the summary or quote; and otherwise they'll have a shorter section of text to look through and more information about what specifically to look for.

When you talk with a small number of people, these issues are usually fine, although if someone is really repetitive and maybe discussing in bad faith, then you'll have to bring up that pattern and take it from there. (And if that meta level discussion goes very poorly, you can go to another meta level. And if that doesn't work, my article about impasse chains may help.)

If you're dealing with a lot of people, then these issues can get more problematic and time consuming. So you may need to use techniques I described earlier like getting some of your audience to help answer the easy points (like answering arguments that are already answered in the tree, by pointing out where they're already answered). Or you can have some clear, written policies about how you spend a limited time budget which have anti-bias design features like answering some randomly selected people. Or you can have some clear, written policies about who you'll talk to which do some quality filtering but still give a way anyone with something important to tell you could predictably (from his point of view) put in effort, meet your criteria, and then get your attention.