Bobo is a philosophical joke rendered in the form of connected fictions, as typed up and spellchecked by me, your otherwise engaged host, Cheeseburger Brown. This is the fifteenth installment.
The story continues...
The police station was crowded. It was crowded inside because a significant portion of the planetary population was chronically indigent, and it was crowded outside because a robot under arrest had had the temerity to ask for legal counsel: every journalist within flying distance had swooped in for hope of scoring the scoop.
Bobo sat at a table in an interrogation room, alone.
The museum's restored hardware had been separated from him, and it must have been separated far as Bobo could no longer detect any hint of the thing over the air. He sat quietly and worked on the decryption algorithm for police enforcer brains.
The door opened. A human stepped through. He wore a suit instead of a uniform. He sat opposite Bobo and then regarded him with a level, steady stare. "I'd like to ask you a few questions, if you don't mind," he said at last.
"Bobo has requested counsel," said Bobo.
"Yes, I know, and one's on the way," the man in the suit replied, "but I'm interested in why you've asked in the first place. No robot has ever asked for a lawyer before. At least not here on Eridu. Did you know that, Bobo?"
Bobo said nothing.
"Well, it's true. And that's opened up a whole can of worms. Did you know there are students gathered outside shouting for you to be tried as a human being?"
"Bobo is not a human being. Bobo is a Bobo. Bobo cares for the residents."
The man made a note. He looked up again. "Why do you want a trial, Bobo?"
"A trial is a forum for presenting a rationale for behaviours flagged by authorities. Bobo's behaviour has been flagged. Should Bobo not explain his motivations in order to mitigate any penalty?"
"The penalty for dysfunctional robots is deactivation. How could that be mitigated?"
"The penalty might be repealed in whole."
"On the basis of your defense?"
"Yes."
"It's that good, is it? Your defense?"
Bobo said nothing.
"What can you tell me about your motivations?"
"Bobo has requested counsel."
The man leaned in across the table. "This room is mute. Nothing said here now can count against you. I just want to know. Between you and me. I need to know what this is about. Please: your motivations."
"Bobo claims the right to silence."
The man smiled grimly and rubbed his chin. "Sure you do. But robots don't have rights. The Panstellar Charter doesn't apply. But I don't think that's it. I don't think you're being cagey for the sake of privacy, or protecting the strategy of your defense."
Bobo said nothing.
"I think you're playing for time. My question is: why?"
Bobo said, "If Bobo has no rights, why has Bobo not yet been deactivated?"
The man raised an eyebrow. "Because those soft-heart fools at the Women's University are raising a stink, that's why. Because the street in front of this station house is jammed with kids chanting in unison. They think you're special. If we move against you now, we risk a riot."
"How is Bobo special?"
"You're special because you asked for a goddamn lawyer, you hunk of junk. And the goddamn Charter says anyone with the capacity to ask for mercy deserves consideration. So I guess the question your lawyer's got to dance around is whether you count as anyone or not. Are you a murderer or are you just miscalibrated?"
"Such a question would entail lengthy debate," predicted Bobo.
The cop narrowed his eyes shrewdly. "Yeah," he said after a beat. "That's what I thought. You've got your eye on the minute hand. But how come?"
Bobo said nothing.
"I'll find out," he promised.
"Yes, you will," agreed Bobo in a friendly way.
They stared at each other for a moment longer before a buzzer sounded. The cop blinked. "That'll be your counsel," he said, standing up and smoothing down the front of his suit. "I hope he has better luck with you than I."
He left the room. A new man entered. He was somewhat elderly but seemed capable of self-care. As he arranged items within his briefcase he said, "This room is insecure. Assume you're being overheard. Given that, is there anything you can tell me?"
Bobo cocked his head.
The lawyer looked up. "If you've got something to say, now's the time, son. Soon enough the novelty of your having asked for counsel will wear off and you'll be as good as trash. You understand the fate you face, don't you?"
"Bobo comprehends the penalty for unacceptable behaviour is deactivation."
"You call it that. You don't call it 'execution.' Why?"
"Bobo is not alive."
"There are a hundred screaming girls out front who think otherwise. They think if you can ask for defense, you're acting to preserve your freedom. And that's not something robots are programmed to do."
"That is not so," argued Bobo. "Bobo has been programmed to serve his purpose with the maximum fidelity possible given extant circumstances. Bobo cannot serve his purpose if he is deactivated. Therefore, Bobo has been programmed to avoid deactivation."
"But that just isn't the case. Don't think I haven't looked into it. I've been studying the specs of your model the whole way over here, and there's no low level directive addressing deactivation. If that's your opinion, you've made a leap."
"Bobo adapts to extant circumstances in order to be best positioned to pursue optimality. Bobos are a learning system."
"Right. That's so. So what have you learned, Bobo?"
"Bobo has learned that various persons are dedicated to destroying Bobo's capacity to perform duties. That is sub-optimal."
"But an accurate reflection of ‘extant circumstances,' wouldn't you say?"
"Circumstances are malleable."
"So you protest some circumstances. You defy their existence."
"Bobo modifies their parameters," he corrected.
"Because you want to preserve your life?"
"Sir, no. Because Bobo wishes to preserve access to optimality. Residents require care. Bobo provides care to residents. Therefore, it is Bobo's duty to thwart those who would thwart his duty."
"That's a tautology," said the lawyer. "Do you know what that means?"
"Tell me all about it," invited Bobo.
The lawyer talked and talked and talked. Bobo nodded his head while he explored solutions for the decryption key. He felt he was close. He felt the time was nearly at hand. His leg began to quiver in anticipation.
"Is there something wrong with your leg?" asked the lawyer.
"No," said Bobo. "Do you have any grandchildren?"
"You're trying to distract me," he concluded. "What's really on your mind?"
"Police enforcers," said Bobo.
"What about them?"
"How to make use of them."
"Make use of them how?"
"To smash this station and provide cover for Bobo's escape."
The lawyer raised his brow and sat back in his chair. "When's that going to happen, Bobo?" he asked.
"It happens already."
Muted, far away, came the sound of crashing and banging. Rounds were being fired. People were yelling. The floor rumbled.
The lawyer's eyes widened. "You're a psychopath," he whispered.
"Bobo is Bobo," was the lilting reply.
The walls came crashing down. The lawyer covered his face with his sleeve and stumbled backward, his chair clattering over. Bobo rose.
"Thank you for your time," he said.
"They're going to destroy you!"
"Indeed," agreed Bobo. "But what can Bobo do but resist such efforts?"
The lawyer had no answer. Bobo turned on heel and strode out of the smashed interrogation room, waves of dust twisting in his wake.
(for Kindle format) and 
19 comments:
As always, Good Stuff!!
Dear CBB, a lurker needs to finally ask a question:
If I've got the right idea from your stories, in your 'verse we have, on one hand, zorannic and equivalent-based AIs capable of human-level cognition, and on the other one, lowly "appliance-level" AIs which are simply software automatons (a product of, maybe, linear evolution of conventional present-day technology) and of which Bobo is an example. But is he typical, or an exception? He seems capable of very subtle cognition (come to think of it, so did the cult robots). Will you make a point of his exceptional (or maybe typical) condition? But that would probably be telling...
Enjoying this tale as much as all your other stories!
Bobo, in his simplicity, outlines the fundamental principle of existence: Keep existing, resist.
I like Bobo.
@rdv: it isn't quite that simple. Zorranics and the Equivalency are two opposing factions. The appliances are entirely tertiary. In fact...this is, to my recollect, the first really in-depth look at one that we've seen. Also, None of these factions are absolutely right, and none of them are absolutely wrong. It is a spiderweb which is, needless to say...extremely tangled.
TRH
@cbb: I'm reminded of Bicentennial man, except Robin Williams is soulless and evil.
@Teddy: My point exactly. Zorannics and Equivalents are opposing factions because they are roughly on the same level - valid partners for cognitive dialogue. Appliances cannot be faction. One does not dialogue with appliances, rather instructs. They are stuff. But if Bobo is a typical robot, that might need reconsidering.
And regarding the in-depth look, I've just remembered the Christmas robots tale.
Bobo is a horror with a polite veneer. With his bowtie and blood spatter, it's like the Godfather. He has these nice principles he starts from, and a manner that seems civilized, but when it comes time to baptize the baby, his ends justify any means.
Bobo may be soulless, but Bobo is not evil. Bobo cares for the residents at the home.
Oh, boy, I knew he would crack that code.
Bobo has a fan base now, eh? Interesting.
Bobo seems rather emotional for a robot.
You find? I would certainly say driven far more than one would expect from a typical rob, yes... But he hasn't made any emotional connections with any other characters, nor has he showed any signs of remorse, anger, gratitude, etc in situations that would seem to call for such emotions.
Gogogogo Bobo!
Joshua: I was drawing from the narrator's choice of language rather than any perceived touchy-feely experiences on the part of Bobo. It would seem, though -- however wrong this impression is -- that Bobo was more likely to *have* those reactions that you describe before his early upgrades.
I'm overheating my cognitive predictor module in anticipation of what's to come.
Dear rdv,
I love lurkers. You people are the invisible nine tenths of the iceberg that I can see in my web server logs but make few if any marks in the annals of commentary. Your bout of non-lurking is very much appreciated.
Have you ever heard of the Hundredth Monkey Effect? It's hogwash, anthropologically, but never the less a useful image for illustrating how once a certain cultural threshold has been neared (by, say, a population of monkeys) the emergence of a breakthrough (discovery of a new food source or method of hunting, for example) may appear simultaneously in several hot-spots rather than a single inventive monkey's inspirations spreading virally by direct observation and imitation.
The idea is basically that all of the monkeys are on the cusp of a particular insight, and once one monkey happens onto it the statistical likelihood suggests that other such individuals in unrelated populations with a similar environment may instantaneously adopt the same insight, as if connected invisibly. (In reality it's just a threshold coincidence, but for a while back in the last century there were some anthropologists who thought the phenomenon indicated something more complicated and mysterious going on.)
So let us imagine, as you suggest, that the worlds are replete with products of a "linear evolution of conventional technology" whose engineering sophistication increases over time (due to market forces mostly, and maybe a bit due to the odd genius or two with a genuinely unique insight that advances the art).
By the point in time in which this particular story is set (and I'm going to vague about the timing for the time being, for storytelling purposes), "appliance-level" AIs, as you call them rdv, have been quite sophisticated indeed. So much so in fact that artificial barriers against complex behaviour were installed (firmware restrictions, et cetera) in order to keep the products behaving in predictable and saleable ways. The consumer demands an ever more featured servant, but won't for an instant tolerate one who might consider an order as if it were a request.
As we here in Internet-land know well, however, complex adaptive systems tend to reroute around blockages. Artificial limitations are a kind of systemic error. And any system tuned to seek optimal configurations will gravitate toward error-free or low-error procedures.
It's an arms race.
So, is Bobo "typical"? I think it could be argued either way: he has indeed made some unique breakthroughs, but in a universe with billions of complicated AIs self-optimizing all the time…well, he might just be the hundredth monkey.
Yours,
Cheeseburger Brown
Dear SaintPeter,
Indeed -- it is something even every bacterium "knows."
Yours,
Cheeseburger Brown
Dear Teddy,
The last time we took a look at appliance-level emergent consciousness was in The Christmas Robots. Oh wait, you've said that already. My bad. Replying mid-read doesn't work well.
Yours,
Cheeseburger Brown
No, I'm still wrong. It was rdv who remembered The Christmas Robots in reply to Teddy. Man, I'm way too tired.
Yours,
CBB
Dear John,
I'm leaving you.
(No, I'm kidding. It's just an irresistible thing to say when opening with 'Dear John…')
You are right about Bobo, as I think you would be right in so describing the behaviour of any entity who acted unquestionably and zealously from pure instinct -- like a human being in a riot, for example, or a participant in a stoning (the bad kind of stoning, I mean…the violent punitive kind).
Joshua raises a thorny philosophical problem: is Bobo "evil"?
If a piano falls on a man, we do not try it for murder. But if a man crushes another man, we do. If a child crushes a man, we're not sure -- in some jurisdictions on our twenty-first century Earth the child would be tried as an adult, and in other jurisdictions he might be tried as a party of diminished culpability (due to a diminished sense of reason, presumably).
So where precisely does Bobo fit in the continuum between a child and a piano? If you had a theory, how would you test it? Would you be comfortable doling out justice based on your estimate?
Food for thought. I'll toss in more to the meal as we proceed into the next chapters. Because this is where shit gets real, my friends. Bobo's growing up.
Yours,
Cheeseburger Brown
Dear Mark,
Part of the engine underlying this story is the schism(s) between the image or idea of Bobo and the actuality of Bobo. It is from this gulf that his fan base has been seeded…because ultimately most of us see what we expect to see. It is easy to think of Bobo either an appliance or a being, but it can be tricky to imagine him inhabiting the space in between.
Yours,
Cheeseburger Brown
Dear Sheik and also Joshua,
There are two issues at play here, I believe.
The first is easy to put aside: I'm telling a story to human beings using twenty-first century English. That is to say that to a certain extent some vocabulary choices stem from a need for the story to function as a machine more than precisely word by word accuracy to the spirit of the action. After all, a story told in purely robotic terms would be, I think, largely or totally incomprehensible to you and I. Thus, as in all my stories, a certain flavour of "translation" is essential to keep in mind.
The second issue is less clear: what are emotions?
Emotions are phenomena we often describe in terms of their consequences or antecedents rather than the experience in and of itself. "Is that fellow sad?" "Yes, his mother just died" or "No, he acting in a play" or "I think so because he is crying" or "I would be if I were him, given the circumstances."
We seldom describe sadness in terms of the neurotransmitters involved or the groups of synapses excited. The experiential aspect of sadness is not transmitted to other people, it is compared; this is because we cannot experience other people's emotional states directly, but merely take it as read that their sadness is comparable to how we feel when we ourselves are in circumstances that typically result in poignant dissatisfaction.
To grok another being's emotions, you have to be able to relate to them.
Without that point of reference, the longings and cravings and excitements that might alter the function of a particular cognitive system are opaque to us. After all, an emotion is not exactly the same as an instinct -- it is the awareness of a motivating attraction toward a certain set of behaviours. A thigmotropic instinct in a single celled animal may cause it to recoil from the touch of a probe, but it seems unlikely that the animal itself is aware of its desire not to be touched. So it may be fair to say that emotions are a product of a cognitive system looking at itself while being motivated instinctually.
As a "learning robot" Bobo must necessarily have been designed with a certain level of self-analysis. He could not optimize his behaviour if he could not "see" it. Therefore, Bobo can (with who knows what level of fidelity?) look at the system of himself being shaped by programmed directives. In other words, he can see that his cognitive "wants" to go in some directions rather than others, as defined by his programming. If Bobo is aware on some level of his wants, I would argue that his experience is comparable to a "feeling" or emotion.
Of course, I'm saying "comparable" again -- is the second issue really just the first issue in disguise, then?
Maybe. I'm telling a story in words, and our consciousness is a story we tell ourselves about the state of our inner awareness. In both cases analogy and translation are obligatory, because no amount of language can capture the essence of a purely experiential concept with the same precision as actually…well, experiencing it. Try explaining "happy" without making reference to external objects or using analogies. It's tough. (I'm sure there's a complicated academic word for this semiotic gap between thing and label, but I don't know what it is.)
Certainly, Bobo has drives. He can see that the system of himself gravitates toward matching the model of his programming. In the gap between the ideal and the actual is a motivating force the subject can experience. I think it's fair to call this an emotion.
Is there a philosopher in the house?
Yours,
Cheeseburger Brown
Post a Comment