Smitcher's thought gargleblaster

the impossibility of proving one's own thought method

what can you prove? if you manage to prove something, you also need to prove that the system you used to prove whatever you wanted to prove is correct and valid. for instance, if I say that conclusions should come from evidence and that electric cars are better for the environment after a certain distance driven, I not only need to prove this true based on the evidence I find but I also need to prove that evidence-based thinking is the best way to think about this.

you cannot prove that your axioms are the correct ones without using those axioms. nobody can say their axioms are correct because of this. we also can't compare axioms because the metric depend on what you care about, which comes down to your axioms once again.

TL;DR: Godel says you can't prove your system is correct.

the public aspect of epistemic humility

if we take the impossibility of proving one's own thought method for granted, then we come accross a problem: the people who become conscious of the impossibility of a perfectly accurate model will try to learn humility.

the truely best of thinkers will learn humility and reserve their opinions because of this uncertainty. this means that loud morons show their truths more than people who have thought about the issues. the more one thinks, the less their views will be exposed though they are probably the most exact.

this is the dunning-kruger paradox. only the overly confident views are shown while the nuanced takes are barely taken into account. this is an important issue stemming from many problems like tribalism and the way our tribe-optimized brains interact with opposing views with other people.

we have to take concrete views to fit in, and because of our antique machinery, we don't create a truth-seeking system but rather a society in which extremes thrive. this renders most debates pointless unless some basic common axioms are established.

epistemic fundamentals

the system is the studied thing. it's very difficult to create an accurate model of anything, so studying a specific self-contained system is the only way to move forwards. a system has only one requirement: it must be self-contained. no external factors to the system must influence it. by far, most systems aren't perfectly self-contained, but we can pretend they are.

i'm aware of the münchhausen trilemma, and therefore chose option three to base my epistemic model upon: an un-proved axiom. it's turtles all the way down, and this is my fundamental turtle: Shelley!

Editor's note: I wrote this text late at night. I now realize just how insane it looks not only to name the epistemic turtle upon which i based my epistemic model, but to give it an image.

my first axiom is the following:

the system can be predicted perfectly.

i define in this axiom that given the same inputs, a system will always give the same outputs. this axiom can be defined as there being at least one model that can accurately predict the output based on the given inputs. this axiom can't be proved, but i assume it's true. there is no point in trying to predict a non-deterministic system such as rolling dice. this epistemic model is only applied in predictable systems because trying to predict a (by definition) unpredictable system will yield nothing of value. given that it is difficult to know wether a system is deterministic or not, based only on what we can see from it. it is difficult to make the difference between a model being wildly incorrect and the system not being predictable. faced with this difficulty, one should assume that the system is predictable, because they can do nothing in the contrary case.

from this initial axiom we can find another one:

one must seek a correct model of the system

there is at least one model that predicts the system accurately. it's useful to have an accurate model of the system because having more knowledge is better than less. having more data helps you with any goal. this is instrumental convergence: no matter your goal you should aim to have accurate data.

there are infinite possible models of a system

this axiom is simple. you can always create a new configuration of inputs and do anything you want with them. we can define a model's truthfulness with the following:

a correct model can predict the system output correctly from the inputs for every possible set of inputs. an incorrect model does not fit reality for at least one set of inputs.

from this point in my epistemology journey, i can go towards several ways of thinking: popperism, bayesianism and more. popperism is useful in theory but breaks down when a model is mostly correct. it's hard to define "mostly correct" models with popperism, but this becomes easier with bayes. i haven't really explored either way of thinking in enough depth to form an opinion about it. for a later date!

tl;dr: epistemic basics: you can apply my epistemic model in predictable systems. you want to look for correct models. correct models predict the outcome perfectly for every input whereas incorrect ones fail in at least one instance.

the delegation problem

to get anywhere in life, people must take other people's word for it. one cannot rething everything, because one simply doesn't have the time. this is why we trust experts and take people's word for it.

for instance, I would take Neil Degrasse Tyson's word for it if he said something about astrophysics. I don't need to study astrophysics, recreate its theories from scratch, dedicate large amounts of time to understanding this to feel confident in trusting what he has said.

I trust this source for a couple of reasons: he's a scientist who follows the scientific method which I respect. He is very popular so a major mistake wouldn't go unnoticed by his community leading to an automatic fact checker, which puts a lot more chances towards him being right rather than a lesser-known person on the internet.

these reputable sources are incredibly powerful because they let you dedicate your time to new issues rather than trying to recreate everything from scratch.

the link with ai safety:

right now, a universally reputable source isn't known to me in ai safety. many people working in the field say that it's an existential risk whereas others say that using dumber models to keep smarter models in check is safe. few people agree about ai risk, and thus people have to recalculate the risks, which is incredibly taxing.

imagine if scientists needed to recreate newton's laws before being able to actually estimate a trajectory. no reputable source is known to me, but it doesn't mean it doesn't exist. sadly, i don't have the capabilities to asses if a source is accurate or not without diving into the problem myself, which means that i need to recalculate anyways.

there is no nobel prize in ai risk, nor is there an agreed-upon universal epistemic method. rationality seems a bit blurry. some of its core axioms seem to be agreed-upon (such as conclusions follow evidence, not the other way around), but there isn't a rationalist's handbook dictating specific well-defined axioms. the sequences are a good start, but again this is a lot more work for new-comers.

if ai risk is truely an urgent matter (which i am not denying or affirming. it seems to be a popular perspective, so it should be taken into account from an outsider's perspective who hasn't truely thought about this), then these layers of complexity to get into the ai risk assesement world are barriers which probably push people away from these issues. if one needs to learn a new epistemic method before thinking about a very complicated issue to finally have a worthy conclusion before acting, then few people will actually be fully conscious of the true problem and be able to do anything about it.

one final issue is the dunning-kruger effect: people who don't know much about the ai alignment problem will probably be loud-mouthed because they are overly confident. people who have actually thought about this complicated issue and who have a nuanced take probably aren't as loud because they don't defend a concrete extreme perspective.

we need to solve this issue by:
• defining the best epistemic method concretely and define core axioms to be able to think about the issue
• make a clear thought process based on these axioms that leads to some conclusion or another
• make this thought process public, easily accessible and easy to understand for newcomers
• if ai risk is truely an important problem, we should create reputable sources of information for new-comers to take from without having to think everything through again

Solving the Delegation Problem

the delegation problem is an important one: who should you trust when you don't know enough about a subject to make your own mind up. you cannot judge if someone is right or not without thinking about the problem yourself, which defeats the purpose. i trust neil degrasse tyson even though i haven't thought everything through myself. once again, i trust him bc i trust the method he uses and the scientific community.

i trust reputable scientists bc the scientific institution is worth something. i can probably trust something published by MIT bc they wouldn't be willing to publish something wrong and tarnish their reputation. this means that the things that people value lets me know that i can trust some things more than others. this is great for famed institutions like MIT, but their process won't show great ideas from random people. there are only so many institutions and groups you can trust and yet many great ideas don't get shown.

a good solution: LessWrong! LessWrong's karma system automatically sorts for the highest quality posts and comments with the rationalist metric. this means that the good ideas (by the standard of rationality) automatically are spread better. the rationality corner of the internet offers axioms for rationality and ways of thinking in the form of the sequences and other important posts. it also teaches these methods in camps like ESPR or ASPR.

this is great bc the way of thinking is first presented and then enforced through the karma system. this means that newcomers can be fairly confident that a post with high karma is correct and offers valuable insight even though they might not be able to measure its quality themselves. this artificially removes the "loud morons" from the equation. it mutes the morons and gives great posts a better chance by putting them on the front page for example.

this system is also beneficial bc a post has a separate karma value from its creator. this means that good ideas can be trusted independently from its author.

TL;DR: it's hard to judge whether or not a source is reputable without diving into the subject yourself. LW is built in a way that lets you find the good sources w/o diving deep into the subject yourself.

on rationality

i've thought about axioms and ways of thinking. the results of which is a puny list that can be resumed to "truth be good!". i'm satisfied with this base of axioms, but i know this is very idealised and theoretical. it's very nice to say that finding the truth is good, but my little list of axioms aren't going to bring me anywhere.

i wrote my base axioms in a vacuum, without the input of anyone and without reading anything beforehand, yet if i compare them to LessWrong's fundamental values, they're pretty much the same. after reading through some of the sequence highlights, i see a lot of similarities though my ideas are a lot less fleshed out and clearly presented. rationality seems to be an excellent way of thinking and its community seems very truth-centered, which is perfectly aligned with my thoughts. i believe in rationality.

my axioms are fun, but pointless. the world isn't perfect. there are no perfectly self-contained systems. i need to adapt my way of thinking to actual concrete problems. rationality already has ways of implementing good theoretical axioms into the real world with bayesianism for example. instead of rebuilding everything from the ground up, i will outsource a lot of the way of thinking to the rationalist community.

i can probably trust an epistemic system derived from readings on rationality more than my little thing i create in my room. rationality has been applied to numerous situations and the rationalist community has predicted numerous things such as the pandemic. this means rationality has a much better track record than my system which has none. it would also take a lot less time to learn rationality than attempt to create a lesser way of thinking by myself.

btw, the fact that my axioms look a lot like elements from the sequences and rationality probably isn't a coincidence. i had read many articles from the sequences about a year before writing these texts, and even if i didn't reference them consciously, they probably affected my findings. my texts are mostly just the basics from the sequences but written in a worse style and with a lot less important information and with less insight.

TL;DR: rationality is aligned with my "findings". it's much better to become a rationalist than trying to create an epistemic model by myself. this means becoming a rationalist should be my (instrumental) goal. rationality ftw.