What's stopping them?

25 posts / 0 new
Last post
Steel Accord Steel Accord's picture
What's stopping them?
With all of these AGIs around, why don't they achieve a singularity of their own? I'm not saying it would be easy, but why don't they just re-write their base codes to be capable of exponential self-improvement? Now I know in the inner system, they are restrained in that matter, but what about in the outer system? I'm just surprised we don't see a mass exodus of AGIs going, "too smart for you all, TTFN!"
Your passion is power. Focus it. Your body is a tool. Hone it. Transhummanity is a pantheon. Exalt it!
ShadowDragon8685 ShadowDragon8685's picture
First off, you need hardware
First off, you need hardware to do that, it's not just a matter of software, and getting that kind of heavy metal won't be easy, even in an autonomist hab. Secondly, everybody's still hyper-aware of the TITANs, so trying to follow in their footsteps isn't going to end well, one way or another. Thirdly, they themselves are probably afraid of what could happen if they did that - not just what others would do to them, but what it would do to [i]them[/i] - AGIs have been growprammed, basically, socialized with humans and others. They're probably for the most part terrified of what recursive self-improvement could do to their mind-states.
Skype and AIM names: Exactly the same as my forum name. [url=http://tinyurl.com/mfcapss]My EP Character Questionnaire[/url] [url=http://tinyurl.com/lbpsb93]Thread for my Questionnaire[/url] [url=http://tinyurl.com/obu5adp]The Five Orange Pips[/url]
Armoured Armoured's picture
A Promethean did it
Another reason is that the vast majority of AGI codebases were meddled with/cyber-uplifted/writted by the Prometheans, post-fall. Its not revealed why they work to spread machine consciousness across the solar system, but they do. If anyone wants to stop another TITAN singularity, its the Prometheans. Being ones themselves, they know the deep secrets that allow AIs to gain strong self-improvement, and have neutered these channels in AGIs.
Smokeskin Smokeskin's picture
I figure most AGIs want to
I figure most AGIs want to remain themselves too. Modifying yourself to go on a hard takeoff means changing yourself fundamentally. There is someone who tries it once in a while of course, but they're unlikely to find collaborators who won't turn them in, and their accumulation of resources to construct more hardware to run its expanding mind will be noticed before it gets to problematic levels.
Steel Accord Steel Accord's picture
Prometheans
Armoured wrote:
Another reason is that the vast majority of AGI codebases were meddled with/cyber-uplifted/writted by the Prometheans, post-fall. Its not revealed why they work to spread machine consciousness across the solar system, but they do. If anyone wants to stop another TITAN singularity, its the Prometheans. Being ones themselves, they know the deep secrets that allow AIs to gain strong self-improvement, and have neutered these channels in AGIs.
Yeah about them, what is their deal? I know they were created before the TITANs and went through a soft takeoff Singularity. I know that they work against the TITANs but that's about it. Are they benevolent? Are they known to the public? I just keep hearing about them without really understanding them.
Your passion is power. Focus it. Your body is a tool. Hone it. Transhummanity is a pantheon. Exalt it!
Steel Accord Steel Accord's picture
That make sense
Smokeskin wrote:
I figure most AGIs want to remain themselves too. Modifying yourself to go on a hard takeoff means changing yourself fundamentally. There is someone who tries it once in a while of course, but they're unlikely to find collaborators who won't turn them in, and their accumulation of resources to construct more hardware to run its expanding mind will be noticed before it gets to problematic levels.
But don't a lot of uplifts and transhumans want to change themselves fundamentally? I thought continuous self-improvement was a core mindset of this setting, not to mention actual transhumanism. (Or is that just my Extropian side talking?)
Your passion is power. Focus it. Your body is a tool. Hone it. Transhummanity is a pantheon. Exalt it!
Leodiensian Leodiensian's picture
Prometheans
They're one of those largely-undefined "up to the GM" thing but the traditional scuttlebutt around them is that they're the nice AIs that had more human socialization than the pure-military TITANs and also managed to avoid exsurgent infection. It's probable that they're integral in Firewall - I remember one campaign features the Interdiction being run by a Firewall Promethean and I think Gatecrashing features an exoworld that a Firewall Promethean has colonized as a research station or something.
Steel Accord Steel Accord's picture
Same question
ShadowDragon8685 wrote:
First off, you need hardware to do that, it's not just a matter of software, and getting that kind of heavy metal won't be easy, even in an autonomist hab. Secondly, everybody's still hyper-aware of the TITANs, so trying to follow in their footsteps isn't going to end well, one way or another. Thirdly, they themselves are probably afraid of what could happen if they did that - not just what others would do to them, but what it would do to [i]them[/i] - AGIs have been growprammed, basically, socialized with humans and others. They're probably for the most part terrified of what recursive self-improvement could do to their mind-states.
Same question I gave to Smokeskin, isn't recursive self-improvement a key motivation in this setting and actual transhumanism? I mean, I understand the hesitance to loose yourself to just raw infinite intelligence, but wouldn't gearing one's self for a soft takeoff sort of remedy that?
Your passion is power. Focus it. Your body is a tool. Hone it. Transhummanity is a pantheon. Exalt it!
Smokeskin Smokeskin's picture
Steel Accord wrote:Smokeskin
Steel Accord wrote:
Smokeskin wrote:
I figure most AGIs want to remain themselves too. Modifying yourself to go on a hard takeoff means changing yourself fundamentally. There is someone who tries it once in a while of course, but they're unlikely to find collaborators who won't turn them in, and their accumulation of resources to construct more hardware to run its expanding mind will be noticed before it gets to problematic levels.
But don't a lot of uplifts and transhumans want to change themselves fundamentally? I thought continuous self-improvement was a core mindset of this setting, not to mention actual transhumanism. (Or is that just my Extropian side talking?)
Let's you're an athlete, and your thing is sprinting. Running really fast. You're extremely dedicated, going for the world record, with most of your life being about or at least influenced by it - you always stick to your diet, training and sleeping schedule, you don't party, you don't drink. But would you want to transform yourself into a drag racer?
Steel Accord Steel Accord's picture
Point taken
Smokeskin wrote:
Steel Accord wrote:
Smokeskin wrote:
I figure most AGIs want to remain themselves too. Modifying yourself to go on a hard takeoff means changing yourself fundamentally. There is someone who tries it once in a while of course, but they're unlikely to find collaborators who won't turn them in, and their accumulation of resources to construct more hardware to run its expanding mind will be noticed before it gets to problematic levels.
But don't a lot of uplifts and transhumans want to change themselves fundamentally? I thought continuous self-improvement was a core mindset of this setting, not to mention actual transhumanism. (Or is that just my Extropian side talking?)
Let's you're an athlete, and your thing is sprinting. Running really fast. You're extremely dedicated, going for the world record, with most of your life being about or at least influenced by it - you always stick to your diet, training and sleeping schedule, you don't party, you don't drink. But would you want to transform yourself into a drag racer?
I have a thing for martial arts and swordsmanship. If I could continually imprint every style and technique, practice them to perfection, and even invent new styles and weapons for zero-g environments or against non-anthropomorphic foes; not gonna lie, that would be fantastic. The appeal of infinite knowledge and unbound consciousness is certainly there, but it's not my way. So I see your point.
Your passion is power. Focus it. Your body is a tool. Hone it. Transhummanity is a pantheon. Exalt it!
thezombiekat thezombiekat's picture
Steel Accord wrote:I have a
Steel Accord wrote:
I have a thing for martial arts and swordsmanship. If I could continually imprint every style and technique, practice them to perfection, and even invent new styles and weapons for zero-g environments or against non-anthropomorphic foes; not gonna lie, that would be fantastic. The appeal of infinite knowledge and unbound consciousness is certainly there, but it's not my way. So I see your point.
but would you go so far as to replace your body with a armored sphere with 20 multi purpose limbs, vectored thrust flight allowing you to perform zero G maneuvers in gravity, and a range of integral weaponry. your now the pinical of close quarters combat able to easily defeat any of your old sparring partners even in numbers, but you have given up everything else that you where. trans-humans perform recursive, iterative improvements waiting to become reasonably comfortable with one and test the next before changing again. ex-humans and singularity seekers want all the change now. without giving time for the psyche and society to keep up. they may get to the same place in the end. but the slower path is safer and more comfortable.
bibliophile20 bibliophile20's picture
adding to the chorus, the
adding to the chorus, the amount of hardware needed for a hard takeoff is not something discreet, either. We're talking a 2 meter block of pure processors as probably the minimum needed to support a Seed AI ego. Minimum. And this is in a setting where something the size of a couple of credit cards is enough to run a transhuman ego. People will notice, unless you're very subtle and careful. Rare elements. Power draw during fabbing. Excess heat produced from fabbing. All of these will send up flags if anyone is paying attention, and, since people don't exist in a vacuum (well, they do, these days, but I'm talking a social vacuum), it doesn't take too much effort for someone to put two and two together with a bit of data mining that the guy who was doing research into advanced cognition protocols and seems to be fabbing alot of processor blocks might be worth keeping an eye on--just in case.

"Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -Benjamin Franklin

Steel Accord Steel Accord's picture
Actually
thezombiekat wrote:
Steel Accord wrote:
I have a thing for martial arts and swordsmanship. If I could continually imprint every style and technique, practice them to perfection, and even invent new styles and weapons for zero-g environments or against non-anthropomorphic foes; not gonna lie, that would be fantastic. The appeal of infinite knowledge and unbound consciousness is certainly there, but it's not my way. So I see your point.
but would you go so far as to replace your body with a armored sphere with 20 multi purpose limbs, vectored thrust flight allowing you to perform zero G maneuvers in gravity, and a range of integral weaponry. your now the pinical of close quarters combat able to easily defeat any of your old sparring partners even in numbers, but you have given up everything else that you where. trans-humans perform recursive, iterative improvements waiting to become reasonably comfortable with one and test the next before changing again. ex-humans and singularity seekers want all the change now. without giving time for the psyche and society to keep up. they may get to the same place in the end. but the slower path is safer and more comfortable.
What you just proposed sounds pretty appealing, not gonna lie. I could assault a Jovian camp for political prisoners with that morph and augments! That said, I do see your point. Better to reach it at your own pace rather than to just rocket to the conclusion.
Your passion is power. Focus it. Your body is a tool. Hone it. Transhummanity is a pantheon. Exalt it!
bibliophile20 bibliophile20's picture
This Order Of The Stick strip
This [url=http://www.giantitp.com/comics/oots0657.html]Order Of The Stick[/url] strip feels relevant to that comment, Steel Accord. Better to get there one step at a time and learn how to handle each new level than to rocket past the level that you're able to control yourself.

"Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -Benjamin Franklin

ShadowDragon8685 ShadowDragon8685's picture
bibliophile20 wrote:This
bibliophile20 wrote:
This [url=http://www.giantitp.com/comics/oots0657.html]Order Of The Stick[/url] strip feels relevant to that comment, Steel Accord. Better to get there one step at a time and learn how to handle each new level than to rocket past the level that you're able to control yourself.
That's only really relevant if the other guy actually has some means of stripping fast-earned power from you, though, as Xykon did when he smacked the chained souls out of V and was able to exploit flaws in the power upgrade s/he was channeling. This is rarely the case, however.
Skype and AIM names: Exactly the same as my forum name. [url=http://tinyurl.com/mfcapss]My EP Character Questionnaire[/url] [url=http://tinyurl.com/lbpsb93]Thread for my Questionnaire[/url] [url=http://tinyurl.com/obu5adp]The Five Orange Pips[/url]
thezombiekat thezombiekat's picture
bibliophile20 wrote:adding to
bibliophile20 wrote:
adding to the chorus, the amount of hardware needed for a hard takeoff is not something discreet, either. We're talking a 2 meter block of pure processors as probably the minimum needed to support a Seed AI ego. Minimum. And this is in a setting where something the size of a couple of credit cards is enough to run a transhuman ego. People will notice, unless you're very subtle and careful. Rare elements. Power draw during fabbing. Excess heat produced from fabbing. All of these will send up flags if anyone is paying attention, and, since people don't exist in a vacuum (well, they do, these days, but I'm talking a social vacuum), it doesn't take too much effort for someone to put two and two together with a bit of data mining that the guy who was doing research into advanced cognition protocols and seems to be fabbing alot of processor blocks might be worth keeping an eye on--just in case.
it wont throw ups many red flags as you would think. as long as you don't search for the computer plans directly. you will need to do some of the design work yourself to convert a system designed to house thousands (typical infomorf processor bank) of egos into a system to support a single super ego. and then network several of those together while still applying all the cycles to a single ego. you can do all this with the programing and hardware:computer skills. then you set out with a typical brinker setup and find yourself an asteroid made of methane ice, there are lots of those in the outer system. you can build processing power with nothing but CHO IRL they have the materials to do it, its just not ready for production yet. so you will be bringing in NO rare materials (this may mean you need a greater mass of processing power but who cares) you build a small hab and start fabing computers inside it. all they can tell is that you are consuming energy. they cant tell if your making a super computer, recreating famus art or just leaving the lights on all day.
Armoured Armoured's picture
Wellllllllll....
ShadowDragon8685 wrote:
bibliophile20 wrote:
This [url=http://www.giantitp.com/comics/oots0657.html]Order Of The Stick[/url] strip feels relevant to that comment, Steel Accord. Better to get there one step at a time and learn how to handle each new level than to rocket past the level that you're able to control yourself.
That's only really relevant if the other guy actually has some means of stripping fast-earned power from you, though, as Xykon did when he smacked the chained souls out of V and was able to exploit flaws in the power upgrade s/he was channeling. This is rarely the case, however.
I'm sure the TITANS thought that too, before tripping over whatever they did that made them... something else. As for other means of stopping a Strong AI, Firewall, Project OSMA, and others have had 10 years of thinking exactly about how to cripple or sabotage an intelligence undergoing hard takeoff. There was thing, called the Fall? People are strongly motivated to prevent that happening again.
thezombiekat thezombiekat's picture
assuming you had decided you
assuming you had decided you wanted to go for seed AI status you could easily assume you would only need to secure your initial bace for a short time. there is no point making spesific plans for your long term defense because you will be able to make much better plans (do plan to have general resources on hand, nanofaber, feed stock, some general purpose defenses). because your code base is not related to that of the titans most anti titan tricks wont work on you and because your goal is personal improvement you wont need to fight them just evade them and your going to be so much smarter than them. this of cause assumes no friendly seed AIs ready to do something about you. like the Prometheans.
Smokeskin Smokeskin's picture
The way I see it, just
The way I see it, just setting up on a single asteroid and converting it to solar panels and computronium, that's not going to give you an unstoppably smart seed AI. It's a step on the road there, but you're still not even at the point where you're going to be outsmarting or outteching a major hypercorp who still has way more resources and brainpower at its disposal. However, if you have the military network of a superpower like the TITANs did, and you use that to wage a systemwide war to get more and more resources under your control, then you can eventually get to the point where you're smart enough to make radical technological breakthroughs.
bibliophile20 bibliophile20's picture
I was going off of the
I was going off of the description from the Glory adventure, where a 2 meter sized block of pure processors was described as potentially enough power to run a TITAN.

"Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -Benjamin Franklin

Smokeskin Smokeskin's picture
But how powerful is a TITAN?
But how powerful is a TITAN? If I'm just by my naked self, I'm not much threat to a bear. Making a weapon would not be easy without tools. If I didn't even know about weapons or tools, getting the idea and getting it right would take a long time. But with the army of smart and productive people of the human race and centuries of development, I can buy a rifle with a scope and easily kill bears. Or I could go bear hunting in an Apache helicopter with thermal sights to find them and an autocannon to gun them down. A single TITAN might be very smart but still not be able to develop the tech that makes them truly dangerous. Maybe it takes millions of them to really get their tech development going.
bibliophile20 bibliophile20's picture
Honestly? I've been stealing
Honestly? I've been stealing from Orion's Arm's [url=http://www.orionsarm.com/eg-article/4b9f2a844034a]Transsapient concepts[/url], and I'd personally peg them as at minimum of a First Singularity transsapient, on up to a Second Singularity transsapient. The ETI, on the other hand, is at Sixth Singularity levels at a minimum--if it's even still around.

"Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -Benjamin Franklin

ORCACommander ORCACommander's picture
Smokeskin wrote:But how
Smokeskin wrote:
But how powerful is a TITAN? If I'm just by my naked self, I'm not much threat to a bear. Making a weapon would not be easy without tools. If I didn't even know about weapons or tools, getting the idea and getting it right would take a long time. But with the army of smart and productive people of the human race and centuries of development, I can buy a rifle with a scope and easily kill bears. Or I could go bear hunting in an Apache helicopter with thermal sights to find them and an autocannon to gun them down. A single TITAN might be very smart but still not be able to develop the tech that makes them truly dangerous. Maybe it takes millions of them to really get their tech development going.
what made the titans so dangerous is that they started off with military grade hacking decryption and brute forcing software. So their first step was to break containment of the military meshes. which was probably not that hard. after that they had all the information transhumanity had its disposal on the civilian mesh. thus it then gained a priority list of what to subvert and once it had subverted R&D facilities they could carry out r&d 24/7 and eschew safety protocols that slow things down since the titans don't need to worry about body count. As a titan i would prioritize the following: 1. remain undetected as long as possible 2. break out of military mesh containment subtly 3. absorb all information publicly accessible 4. break into subtly if possible secure information storages 5. subvert manufacturing facilities 6. subvert r&d facilities 7. secure resource extraction points 8. exterminatus
Pyrite Pyrite's picture
Usually, firewall dispatches
Usually, firewall dispatches a team of sentinels to stop them. With tactics that redefine extreme prejudice.
'No language is justly studied merely as an aid to other purposes. It will in fact better serve other purposes, philological or historical, when it is studied for love, for itself.' --J.R.R. Tolkien
consumerdestroyer consumerdestroyer's picture
Pyrite wrote:Usually,
Pyrite wrote:
Usually, firewall dispatches a team of sentinels to stop them. With tactics that redefine extreme prejudice.
Yeah, as a GM, this is definitely my answer to the OP. My players will know what happens when a group of AGI Isolate habs fortuitously working on very similar post-Singularity goals in the Oort Cloud drift within comm range of each other: because that'll be something Firewall taps them for.