• Log in with Facebook Log in with Twitter Log In with Google      Sign In    
  • Create Account
  LongeCity
              Advocacy & Research for Unlimited Lifespans


Adverts help to support the work of this non-profit organisation. To go ad-free join as a Member.


Photo

Volition And Friendliness


  • Please log in to reply
14 replies to this topic

#1 quicken

  • Guest
  • 14 posts
  • 0

Posted 20 October 2002 - 08:58 PM


In his writings on Friendliness, Eliezer Yudkowsky makes repeated references to the importance of respect of individual volition. However, he fails to offer any working definition thereof and I see this as a potentially critical weakness of his theorising. My challenge to Michael (or anyone else) is to provide such a definition and show that it is compatible with the theory of Friendliness. It should be noted that the brevity of this post is a deliberate attempt not to allow my own thoughts to cloud the issue.

Q

#2 Thomas

  • Guest
  • 129 posts
  • 0

Posted 21 October 2002 - 10:03 PM

I don't see the volition important at all. Now, before the Singularity, it's of course important in our relations with others. But toward FAI - I think it's not.

FAI should rather obey some Constitution or Protocol, than a volition of a confused person.

- Thomas

sponsored ad

  • Advert

#3 quicken

  • Topic Starter
  • Guest
  • 14 posts
  • 0

Posted 22 October 2002 - 06:02 PM

I don't see the volition important at all. Now, before the Singularity, it's of course important in our relations with others. But toward FAI - I think it's not.

FAI should rather obey some Constitution or Protocol, than a volition of a confused person.

- Thomas

You seem slightly confused here. The suggestion isn't that the AI should obey the volition of any individual but that the respect of individual volition (of all individuals) is a fundamental component of the protocol/constitution followed.

Q

#4 Thomas

  • Guest
  • 129 posts
  • 0

Posted 22 October 2002 - 11:32 PM

Here it is:

individual volition may even be the only part of Friendliness that matters - death wouldn't be intrinsically wrong


from

http://www.singinst..../challenge.html


This individual volition is too highly put. What if I am depressed and want to die? FAI will just let me. Apparently.

What kind of friendliness is that? I wouldn't call it that way, if it was my idea.

Instead of this volition, some read only Codex, Protocol or Constitution must be above the volition, when somebody wants something funny. The Codex, assembled by humans, would be a much wiser solution. In fact, we already have some laws dealing with awkward situations. Why not just expand that?

Another thing is - what about those, who refuse FAI protection? At least for a moment?

They may be killed by someone already "under protection". If this volition is above everything.

Did I miss something?

- Thomas

#5 MichaelAnissimov

  • Guest
  • 905 posts
  • 1
  • Location:San Francisco, CA

Posted 23 October 2002 - 02:13 AM

Quicken, CFAI covers this. A Friendly AI isn't a static morality which is entirely hardcoded by the original programmers, a FAI is a dynamic system which improves ver own morality using the tools of intelligence and common moral sense. It's not the AI builder's job to make an AI that knows what the exact definition of "volition" is, it's the AI builder's job to make an AI that knows how to reason about morality in general and converge to an optimum depending on the shape of the moral configuration space. Incidentally, I discussed this at length with Mechanus on the Singularity Discussion Forums a while ago:

http://www.bjklein.c...E=&TOPIC_ID=512

#6 Mechanus

  • Guest
  • 59 posts
  • 0

Posted 23 October 2002 - 04:18 PM

I think Quicken's objection is to volitional Friendliness (from now on, "v-Friendliness"), i.e. Eliezer's predicted model of Friendliness content, and not to the idea of Friendliness as a way of structuring an AI by seeding it with the "human moral frame of reference". [edit]On second read, I see that everyone here understands this, but I still think it makes sense to criticize proposed models of Friendliness even if it will be decided by the AI eventually[/edit]

The link Michael posted is a discussion about the second sort of Friendliness, and I don't have any real problem with that anymore. I still don't think v-Friendliness can be made into a coherent set of ethics that makes sense (I have several reasons and will post them somewhere after writing them out). So, I agree with Friendliness as an AI design philosophy without agreeing with the specific v-Friendliness model for Friendliness content.

I don't see how it's a critical weakness of anything, though - if v-Friendliness is incorrect or ill-defined, the AI in question will find this out and use another model of Friendliness content. Programmer independence means that v-Friendliness is something that can be predicted to be true or untrue, rather than an important choice in creating an AI - ultimately, it's not something the programmers can (or should) control.

To attempt a partial answer to Quicken's question: v-Friendliness means that every individual mind has the right not to be changed or destroyed or kept alive against its will.

#7 Thomas

  • Guest
  • 129 posts
  • 0

Posted 23 October 2002 - 05:14 PM

v-Friendliness means that every individual mind has the right not to be changed or destroyed or kept alive against its will.

What does it mean? Whoever will have a suicidal thoughts - even quite mild, on the Singularity day, the bypassing FAI will just push him into the grave. Volition.

The next day, or the day before, that person would probably ask FAI, to make him less (or never) depressed.

It's an illusion, how we humans have some metaphysical mights, to decide to die, or to live.

Serotonin's level (or something like that), is behind.

Don't like the idea of the so called volition - not a bit. It's a bad concept.

- Thomas

#8 quicken

  • Topic Starter
  • Guest
  • 14 posts
  • 0

Posted 23 October 2002 - 07:43 PM

I think Quicken's objection is to volitional Friendliness (from now on, "v-Friendliness"), i.e. Eliezer's predicted model of Friendliness content, and not to the idea of Friendliness as a way of structuring an AI by seeding it with the "human moral frame of reference".

The link Michael posted is a discussion about the second sort of Friendliness, and I don't have any real problem with that anymore. I still don't think v-Friendliness can be made into a coherent set of ethics that makes sense (I have several reasons and will post them somewhere after writing them out). So, I agree with Friendliness as an AI design philosophy without agreeing with the specific v-Friendliness model for Friendliness content.

I don't see how it's a critical weakness of anything, though - if v-Friendliness is incorrect or ill-defined, the AI in question will find this out and use another model of Friendliness content. Programmer independence means that v-Friendliness is something that can be predicted to be true or untrue, rather than an important choice in creating an AI - ultimately, it's not something the programmers can (or should) control.

To attempt a partial answer to Quicken's question: v-Friendliness means that every individual mind has the right not to be changed or destroyed or kept alive against its will.

1) Yep. I am objecting precisely to v-Friendliness, i.e. content and not structure. I use the term volitional Frienliness in my own musings as it happens.

2) I do not accept that Eliezer for one fully accepts the view that individual volition might not be involved in the evolving moral system. In his references to uploading for example, I recall that he writes rather exasperatedly of his conversations with people worried about forcible upload and the fact that he has to repeat reassurances about respect of personal volition ad nauseum.

3) You finally (almost) touch upon Free Will in your final statement. I see this thorny subject area as the crux of the problem with v-Friendliness.

Q

#9 quicken

  • Topic Starter
  • Guest
  • 14 posts
  • 0

Posted 23 October 2002 - 07:50 PM

@Thomas

In your last two posts you echo some of my own concerns. However, your own suggestion appears to be a utilitarian position and I would caution that the proposition that someone or something else generally knows better than you tends to make the majority of people very nervous indeed.

Q

#10 Thomas

  • Guest
  • 129 posts
  • 0

Posted 24 October 2002 - 02:41 PM

I see no major problem here. You *already* can't kill others, under the FAI protection, why shouldn't that be extended to yourself? And to other non protected sentients?

If I don't want to die now, I will not want to die later, when I will be much enhanced.

If I want to die now, later I would regret, if I'd died.

I think not only death, but some other, too narrow loops, should be forbidden also.

Free will is just a myth, and the volition based on this myth, can be a death trap. What clearly isn't friendly.

- Thomas

#11 Mechanus

  • Guest
  • 59 posts
  • 0

Posted 24 October 2002 - 04:47 PM

Thomas,

I agree with your concerns on volition and suicide - what bothers me even more is that, theoretically, someone could go insane and torture himself for 10^10 subjective years under a volitional Sysop. Theoretically, that someone could even be me.

As a compatibilist, I don't think free will is a myth, though - at least not the kind of free will that means "ability to make choices" rather than "mystical non-deterministic force that gives the mind power over matter and physics". I don't think that second kind is relevant to or necessary for volitional Friendliness.

Also, I would like to know what alternative you would propose or predict - what should be in the Protocol?

Quicken,

I think it would be an exaggeration to say Eliezer doesn't accept the possibility of a non-volitional Friendly AI. If I understand him correctly, the volition scenario only applies in case there is no objective morality; and even if there isn't, then even if he believes (for reasons I don't fully understand and don't fully agree with) that volitional ethics would be justified in such a case, he certainly doesn't come across to me as not willing to accept that he might be wrong. (In fact, his is the only strategy for creating a moral AI that I know of that is explicitly built around that possibility).

Reading his earlier stuff that he no longer agrees with certainly should make it clear he's at least capable of thinking such thoughts.

He does de-emphasize non-volitional possibilities for ethics and morality a lot, but that seems only logical considering the reactions you mention (and the fact that he does not believe them to be true).

#12 Thomas

  • Guest
  • 129 posts
  • 0

Posted 24 October 2002 - 05:26 PM

Mechanus,

Several laws should be inside the Protocol.

For example:

- no sentient may fell any pain

- no sentient shall be killed

- equal resources are granted

- no zone may stay outside the Protocol

- volition is free, iff it is consistent with the Protocol

- .......

- this lines may not be changed after the Jan 1. 2010

If my volition comes in a conflict with any of them, it should be accordingly to the Protocol, not to my will.


This Constitution is either read only area, either the changing procedure is written inside.

It's not only technology, the law must be also developed and implemented.

- Thomas

#13 quicken

  • Topic Starter
  • Guest
  • 14 posts
  • 0

Posted 25 October 2002 - 07:38 PM

I think it would be an exaggeration to say Eliezer doesn't accept the possibility of a non-volitional Friendly AI. If I understand him correctly, the volition scenario only applies in case there is no objective morality; and even if there isn't, then even if he believes (for reasons I don't fully understand and don't fully agree with) that volitional ethics would be justified in such a case, he certainly doesn't come across to me as not willing to accept that he might be wrong. (In fact, his is the only strategy for creating a moral AI that I know of that is explicitly built around that possibility).

Reading his earlier stuff that he no longer agrees with certainly should make it clear he's at least capable of thinking such thoughts.

He does de-emphasize non-volitional possibilities for ethics and morality a lot, but that seems only logical considering the reactions you mention (and the fact that he does not believe them to be true).

OK time for some quotes methinks:

The Sysop Scenario also makes it clear that individual volition is one of the strongest forces in Friendliness; individual volition may even be the only part of Friendliness that matters - death wouldn't be intrinsically wrong; it would be wrong only insofar as some individual doesn't want to die.  Of course, we can't be that sure of the true nature of ethics; a fully Friendly AI needs to be able to handle literally any moral or ethical question a human could answer, which requires understanding of every factor that contributes to human ethics.  Even so, decisions might end up centering solely around volition, even if it starts out being more complicated than that.


(Note also that an AI with shaper semantics cannot nonconsensually change the programmer's brain in order to satisfy a shaper.  Shapers are not meta-supergoals, but rather the causes of the current supergoal content.  Supergoals satisfy shapers, and reality satisfies supergoals; manipulating reality to satisfy shapers is a non-sequitur.  Thus, manipulating the universe to be "morally symmetric", or whatever, is a non-sequitur in the first place, and violates the volition-based Friendliness that is the output of moral symmetry in the second place.)


All that is required is that the initial shaper network of the Friendly AI converge to normative altruism.  Which requires all the structural Friendliness so far described, an explicit surface-level decision of the starting set to converge, prejudice against circular logic as a surface decision, protection against extraneous causes by causal validity semantics and surface decision, use of a renormalization complex enough to prevent accidental circular logic, a surface decision to absorb the programmer's shaper network and normalize it, plus the assorted injunctions, ethical injunctions, and anchoring points that reduce the probability of catastrophic failure.  Add in an initial, surface-level decision to implement volitional Friendliness so that the AI is also Friendly while converging to final Friendliness...

And that is Friendly AI.


The second quote clearly states that volition-based Friendliness is the output of moral symmetry. There is no 'might be' here. More importantly, the third quote seems to say that volition Friendliness is to be included as an initial component of the system. This does require definition and coding.

Q

#14 MichaelAnissimov

  • Guest
  • 905 posts
  • 1
  • Location:San Francisco, CA

Posted 26 October 2002 - 12:05 AM

Mechanus wrote:
On second read, I see that everyone here understands this, but I still think it makes sense to criticize proposed models of Friendliness even if it will be decided by the AI eventually.


Models of Friendliness content, acquisition, or architecture? I personally think it's best to focus more on the architecture and less on the content, because it allows one to divert the conversation away from political issues and more towards metamorality, although content and architecture are liable to be interrelated on a variety of levels.

I still don't think v-Friendliness can be made into a coherent set of ethics that makes sense (I have several reasons and will post them somewhere after writing them out). So, I agree with Friendliness as an AI design philosophy without agreeing with the specific v-Friendliness model for Friendliness content.


This works, I'd say that this is pretty close to my stance as well. Maybe you can start a thread at some point for us to discuss possible flaws or bugs in moralities based entirely upon volitional Friendliness content?

sponsored ad

  • Advert

#15 Mechanus

  • Guest
  • 59 posts
  • 0

Posted 26 October 2002 - 09:59 PM

I personally think it's best to focus more on the architecture and less on the content,


Of course, but what I meant was that Friendly architecture doesn't make it completely pointless to discuss Friendliness content (even if much less pointful).

Maybe you can start a thread at some point for us to discuss possible flaws or bugs in moralities based entirely upon volitional Friendliness content?


Soon, depending on circumstances.
[edit]I now have a long-ish dense-ish post ready that I first want to review to see if it all makes sense; going to sleep now and will post tomorrow[/edit]




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users