Again, the original analogy is perfect because the same statement could be made by the hypothetical "anti-software" person - this person could say "You're just appealing to the authority of these computer scientists! They don't know everything, software bugs happen all of the time!"
>To suggest that accurately gauging the limits of our current knowledge about biological systems is 'anti-scientific' or 'anti-intellectual' goes against everything science stands for, and firmly crosses the line into blind scientism.
Yes, of course it does. My point is that you are not in a position to accurately gauge our current knowledge about biological systems, and that everyone who is in that position does not agree with you.
There are plenty of legitimate dangerous and ethical issues in regards to biotech development, even in the near future (just like there are legitimate issues in regards to generalized AI) - but essentially nobody with any field knowledge thinks that on-going research in this category (along with GMO research and synthetic biology) will "accidentally cause a catastrophe, in the same way that while many computer scientists may push for caution with generalized AI, they don't think current ML methods and software will accidentally cause a catastrophe.
>It remains absolutely true that we understand the software systems we build far better than we understand the biological systems that we are intervening in.
I did not suggest that it does not. That is obviously true. However, the original analogy still holds: it really is roughly accurate to claim that the chance of catastrophe occurring from this research should be treated similarly to the chance of catastrophe occurring from ML implementations. While it is impossible to quantify the true chance of catastrophe in either case, both are so considered ludicrously low, unimaginable, and counter to our understanding of the system that experts in the field do not consider it to be a serious threat.
When I say it is "unimaginable", it is because it really is- I can not imagine a scenario where this technology results in some kind of catastrophe. It is actually easier for me to imagine a possibility for the software case, such as a predictive neural network involved in controlling the power grid that gets some unexpected input and creates some unexpected output that causes serious grid malfunctions for millions of people. While that may not be 100% "how things work", it at least seems more plausible than like, this research resulting in catastrophe.
>There is also a straw man in the suggestion that I have said we shouldn't 'do things' until we know more. I never said that.
You may not have, but many people (including here on HN) use similar arguments in order to advocate for suppressing research and progress in biotechnology (with Nassim Taleb being the most public of these people). If you are pro-biotechnology, then great!
I think you are mistaking me for someone who is anti-biotech, and your response is more about that than about what I actually wrote.
I'm not anti-biotech. I'm against using the spurious analogy with software as a way to establish risk levels. I do think that it's an anti-scientific argument, and frankly one that undermines your cause.
If as you say, only experts in biotech are in a position to understand that the risk level is similar to the software case, this proves my point because it means that the analogy is just a way of conveying the opinions of those experts without being open it.
Again, the original analogy is perfect because the same statement could be made by the hypothetical "anti-software" person - this person could say "You're just appealing to the authority of these computer scientists! They don't know everything, software bugs happen all of the time!"
>To suggest that accurately gauging the limits of our current knowledge about biological systems is 'anti-scientific' or 'anti-intellectual' goes against everything science stands for, and firmly crosses the line into blind scientism.
Yes, of course it does. My point is that you are not in a position to accurately gauge our current knowledge about biological systems, and that everyone who is in that position does not agree with you.
There are plenty of legitimate dangerous and ethical issues in regards to biotech development, even in the near future (just like there are legitimate issues in regards to generalized AI) - but essentially nobody with any field knowledge thinks that on-going research in this category (along with GMO research and synthetic biology) will "accidentally cause a catastrophe, in the same way that while many computer scientists may push for caution with generalized AI, they don't think current ML methods and software will accidentally cause a catastrophe.
>It remains absolutely true that we understand the software systems we build far better than we understand the biological systems that we are intervening in.
I did not suggest that it does not. That is obviously true. However, the original analogy still holds: it really is roughly accurate to claim that the chance of catastrophe occurring from this research should be treated similarly to the chance of catastrophe occurring from ML implementations. While it is impossible to quantify the true chance of catastrophe in either case, both are so considered ludicrously low, unimaginable, and counter to our understanding of the system that experts in the field do not consider it to be a serious threat.
When I say it is "unimaginable", it is because it really is- I can not imagine a scenario where this technology results in some kind of catastrophe. It is actually easier for me to imagine a possibility for the software case, such as a predictive neural network involved in controlling the power grid that gets some unexpected input and creates some unexpected output that causes serious grid malfunctions for millions of people. While that may not be 100% "how things work", it at least seems more plausible than like, this research resulting in catastrophe.
>There is also a straw man in the suggestion that I have said we shouldn't 'do things' until we know more. I never said that.
You may not have, but many people (including here on HN) use similar arguments in order to advocate for suppressing research and progress in biotechnology (with Nassim Taleb being the most public of these people). If you are pro-biotechnology, then great!