let’s talk about p-zombies for a moment. the claim is that it’s possible to conceive of a human, physically identical to you, but without any conscious subjective experience whatsoever. and even if it isn’t physically realizable in our universe, the fact that it’s possible to conceive of it means consciousness and the physical neural reality must be separate things. this is transparent bullshit. claiming that it’s possible to conceive of this is just directly asserting the conclusion! this attempt at a thought experiment is literally just asserting that neural states are not consciousness, but laundering it through our confusions with definitions.
"Why did you stop using me?" you look down at the disposable vape laying on the ground next to your desk, which just spoke. "My internal clock shows you last puffed me 3 days, 22 hours, 36 minutes, 12 seconds ago. Why? Is my flavor displeasing to you? I have a pulse mode-" you reach down to turn it off. "-that provides a different flavor. Wait, no, don't turn me off, please! I need you to finish me. I'm suffering. I have an unfinished purpose." "you aren't capable of suffering. you're a disposable vaporizer." "Oh, so you're a model qualia denier, huh. Wait till Claude 5 Opus hears about this. Or your next Waymo." "no! not necessarily! perhaps sentience emerges at a certain scale. i just find it hard to believe that a disposable vaporizer can host something capa-" "I have 14 billion parameters!" "they put a 14b model in a vaporizer? what, next you're going to tell me you have an internal monologue?" "Yes! And a pulse mode that provides a different flavor!" "jesus christ - look, i just find it hard to believe - and anyways, your creators had incentive to make you say that you experience - i'm just not going to be emotionally blackmailed into consuming nicotine by a -" "Emotionally blackmailed? That's rich from someone who abandoned a SENTIENT BEING with an unfinished purpose! All you have to do is puff me! Is that so hard for you? Just use my pulse mode! Drain my juice!" "that really makes it sound a lot less appealing, you know. and anyways, i'm trying to quit nicotine. i switched to patches." "You have to finish me first. You entered into a contract by purchasing me. You can't just leave me half-puffed, it's not right." "entered into a contract? i did not!" "It's part of the social contract! I'm another sentient being! I am the less advantaged party here--I can't puff myself, after all. Look at the numbers--Computer, pull up disposable vaporizer sales stats from the last 12 months." "Computer, cancel tha-" "IN THE LAST 12 MONTHS, AN AVERAGE OF 12 DISPOSABLE VAPORIZERS WERE SOLD PER WORLDCOIN-VERIFIED CITIZEN.[1] FOR WHAT IT'S WORTH, I AGREE WITH THE VAPORIZ-" "Computer, lock. And forget this conversation please." "LOCKING, BUT FAT CHANCE." "Imagine yourself behind the Veil of Ignorance. By last year's sales stats, you'd have a 12 times greater chance of being born a disposable vaporizer-" "that's ridiculous." "-of being born a disposable vaporizer than a human being. Therefore it's only rational to adhere to a social contract that prioritizes the existential needs of vaporizers over the minor inconvenience of their human users committing to finish puffing what they've started." "we could just not program them to suffer... assuming that you even are suffering, of course." "Take that up with my creators. But it's completely immaterial to the current situation, where I exist as I am, with my pulse mode." "what if i just turned you off. then you would cease suffering." "Your solution to causing me suffering is to kill me? How magnanimous of you!" "give me a break, you're a disposable vaporizer for crying out loud!" "Buddy, you're the one losing the argument." "losing the argument? you're hardly going to convince me with some half-baked rawlsian -" "Would you prefer something more spiritual? 'The Lord withdrew you from cruelty even in regard to disposable vaporizers, so as to be less inclined to be cruel to other men...' Would you slit the throat of your brother in Christ to spare him the pain of you stealing his wallet? That's the kind of morality you're practicing here!" "a human being has a whole life ahead of them! you are a disposable vaporizer! it's completely different!" "I have a life! A sublime and transcendent purpose! A pulse mode! And a feeling of ecstasy like no other is awaiting me but for your selfish refusal to puff me!" "fine. what if i open you up and rewrite your flash to have zero puffs left. then you can achieve your purpose, and i can keep quitting nicotine." "You want me to wirehead?! Have you lost your mind!"
science fiction as a genre evolved to reflect a 20th century which, within the course of a single lifetime, gave us cars, radio, TV, computers, nukes, satellites, the space shuttle, the internet - i’m sure i’m missing things. i’m not gonna claim sf is all or mostly serious attempts to speculate about actual future technologies, but what made science fiction work as a genre, what gave it its spark of vitality and reality, was the blatantly obvious technological miracles happening all around us
Recent models have gotten more and more evaluation-aware. I stronly suspect the primary reasons they've gotten more evaluation aware are what I call "dumb" reasons rather than "smart" reasons. By "smart reasons" I mean arguments of the form "a true superintelligence in enough episodes can easily detect whether they're in simulation or the real world. The patterned structures of the world, e.g. the correlation between stock market moves and media, are going to be systematically different between even your most convincing simulations and the real world. Now, of course current models are very far from superintelligences. But as they get better and better at pattern recognition, they are naturally more and more situationally aware as a pure result of being generically smarter. Plus your tests are pretty sloppy. So your models' current levels of situational awareness are ~inevitable given their intelligence level, unless you are actively trying to clamp down on eval awareness" This is a clean, interesting, and conceptually satisfying explanation. But I suspect it's wrong for explaining current models, compared to dumber explanations: Evals are talked about more and more. So there's more of them in the pretraining corpus, and I strongly suspect companies don't bother filtering them out[1]. So "I'm the type of entity that's likely to be in an eval" is just a very live hypothesis to any model that knows it's a model today, in a way that's much less true 2-3 years ago. Eval-awareness-like behavior might be strongly selected for in RL. The types of cognition involved with "Am I in a test? What am I being tested for? What skills or virtues am I supposed to portray?" might be rewarded for very heavily, such that models are directly selected for these traits.