Subject: Ambiguities in section 3.1 of the keyboard-interactive draft

From: [email-masked] (Peter Gutmann)
Date: Wed, 29 Sep 2004 01:04:59 +1200

This draft currently contains ambiguities that make it more or less impossible
to create interoperable implementations:

  The submethods field is included so the user can give a hint of which actual
  methods he wants to use.  It is a comma-separated list of authentication
  submethods (software or hardware) which the user prefers.  If the client has
  knowledge of the submethods preferred by the user, presumably through a
  configuration setting, it MAY use the submethods field to pass this
  information to the server.  Otherwise it MUST send the empty string.

  The actual names of the submethods is something which the user and the
  server need to agree upon.

  Server interpretation of the submethods field is implementation-dependent.

So there's a pile of undefined methods that are handled in an implementation-
dependent manner.  Let's say the client wants to use the standard lowest-
common-denominator password authentication.  So it sends the submethod
"password" to let the server know this.  This is compliant with both the
apparent intent of the spec ("password" is the submethod "preferred by the
user"), and the wording, which really allows more or less any sort of
behaviour.  The server sees this submethod, decides that it doesn't like it,
and immediately sends back a SSH_MSG_USERAUTH_FAILURE (it could also
execl("/usr/games/hack", "submethods", 0) as its implementation-dependent
interpretation, I guess).

Both of these interpretations are fully compliant with the spec, but they
result in non-interoperable implementations.

As I see it, the intent is that the client provide a hint to the server as to
which method(s) it'd prefer to use.  In this case the server behaviour (the
second two paragraphs above) should be specified as:

  The names of the submethods are client- and server-dependant.  If the server
  encounters an unrecognised submethod, it MUST ignore that submethod.
  Otherwise, it MUST process the submethods in the order specified by the
  client.  In other words if the client specifies submethods
  "method1,unknown,method2" the server should first try method1, skip the
  unknown method, and then try method2.

This avoids the problem given in the earlier example, and nails down the exact
behaviour of client and server during the exchange.

Peter.

---

From: [email-masked] (Niels Möller)
Date: 28 Sep 2004 16:49:06 +0200

> to create interoperable implementations:

As I started implementing this draft some week ago, I can add some
more comments

Section 2, Rationale

   Currently defined authentication methods for SSH are tightly
   coupled with the underlying authentication mechanism. [...]

   Challenge-response and One Time Password mechanisms are also easily
   supported with this authentication method.

I don't think the given examples support this authentication method.
One-time passwords are easily supported using plain "password"
authentication. Some forms of challenge response authentication can
also be done by combining using "password" authentication and
USERAUTH_BANNER. A more correct rational consists of two points:

1. It makes it easier to support challenge response systems with
   hardware tokens such as securid. (This was the motivation given
   when the method was introduced, iirc).

2. It matches the PAM way of doing user authentication. To some people
   this is a huge advantage. (Personally, I believe the PAM way is pretty
   broken, and not at all suitable for network authentication).

The default openssh configuration for some linux distributions have
disabled plain "password" authentication in favour of
keyboard-interact, my guess is that they do this in order to work
better with PAM. I think this is bad for interoperability, and think
it would make sense to make a strong recommendation that if a
particular system supports old fashioned authentication with a
username and password, then it SHOULD also support the
USERAUTH_REQUEST "password" method when speaking the ssh protocol.

Section 3, Protocol Exchanges

I think a client should be allowed to abort a sequence of
SSH_MSG_USERAUTH_INFO_REQUEST, SSH_MSG_USERAUTH_INFO_RESPONSE by
sending a new SSH_MSG_USERAUTH_REQUEST.

This can happen for example if the client thinks that it can
authenticate using keyboard-interactive, but later has to change its
mind (the used forgot his securid at home, presses the cancel button
on the keyboard-interactive dialog, and next tries to remember his
long passphrase for publickey authentication).

Section 3.1, Initial Exchange

      byte      SSH_MSG_USERAUTH_REQUEST
      string    user name (ISO-10646 UTF-8, as defined in [RFC-3629])
      string    service name (US-ASCII)
      string    "keyboard-interactive" (US-ASCII)
      string    language tag (as defined in [RFC-3066])
      string    submethods (ISO-10646 UTF-8)

The "submethods" field is problematic, as already pointed out by
Peter. The entries are supposedly to be used as identifiers. I'd
suggest that this field be defined as a "name-list" (defined in the
architecture draft), where the entries are ASCII names using the same
conventions as for other negotiated methods (i.e. "blahonga-token" is
a standardized method, "[email-masked]" is an extension).

Section 3.2, Information Requests
      
      byte      SSH_MSG_USERAUTH_INFO_REQUEST
      string    name (ISO-10646 UTF-8)
      string    instruction (ISO-10646 UTF-8)
      string    language tag (as defined in [RFC-3066])
      int       num-prompts
      string    prompt[1] (ISO-10646 UTF-8)
      boolean   echo[1]
      ...
      string    prompt[num-prompts] (ISO-10646 UTF-8)
      boolean   echo[num-prompts]

The "name" field doesn't make much sense for me. First I assumed it
was a user name, which makes no sense (what is a client supposed to do
if it differs from the user name given in the USERAUTH_REQUEST
message?). But in the example in the end of the draft, it seems to
rather be part of the instruction, perhaps intended for a window title
or some such. I think this needs some clarification.

It may also be desirable to have one field where the server can put
the name of its selected submethod (i.e. an ssh name, not a free form
utf8 string).

The "language tag" is already deprecated, if we make changes, it
should be deleted. I don't remember the discussion leading to its
introduction or its deprecation.

The "num-prompts" is of type "int", which is not defined in the
architecture draft. I guess "uint32" is the intended type.

I also want to note that this and SSH_MSG_USERAUTH_INFO_RESPONSE are
the only request so far that uses a variable number of entries in this
way, which requires some additional complexity of the code for
formatting and parsing these messages (at least in my implementation,
which uses a general formatting function controlled by a format
string; I'd prefer not having to introduce loop constructs in the
format...). I think this is unfortunate, by I don't see any obviously
better way to do it.

As for processing of this field, Peter's suggestion makes sense to me:

>   If the server encounters an unrecognised submethod, it MUST ignore
>   that submethod. Otherwise, it MUST process the submethods in the
>   order specified by the client. In other words if the client
>   specifies submethods "method1,unknown,method2" the server should
>   first try method1, skip the unknown method, and then try method2.

I think we should also say that after the server has tried all methods
on the list that it recognizes and supports, it is free to try any
other submethods of its choice. This is what happens in the common
case that the client sends an empty submethods list.

Section 3.4, Information Responses

      byte      SSH_MSG_USERAUTH_INFO_RESPONSE
      int       num-responses
      string    response[1] (ISO-10646 UTF-8)
      ...
      string    response[num-responses] (ISO-10646 UTF-8)

Looks ok.

Regards,
/Niels

---

From: [email-masked] (Peter Gutmann)
Date: Wed, 29 Sep 2004 03:47:16 +1200

[email-masked] (=?iso-8859-1?q?Niels_M=F6ller?=) writes:

>The default openssh configuration for some linux distributions have disabled
>plain "password" authentication in favour of keyboard-interact, my guess is
>that they do this in order to work better with PAM.

That was where the example I used in my message came from: Standard "password"
is disabled, and if you send "password" as the submethod OpenSSH immediately
drops the connection with a userauth-failed response.  OTOH if you leave
submethods empty, the same "password" authentication succeeds without any
problems.  This behaviour is compliant with the spec, but only because
execl( "/usr/games/hack", ... ) would also be compliant with the spec.

>I think this is bad for interoperability, and think it would make sense to
>make a strong recommendation that if a particular system supports old
>fashioned authentication with a username and password, then it SHOULD also
>support the USERAUTH_REQUEST "password" method when speaking the ssh
>protocol.

It does actually support standard passwords, you just have to be very careful
not to tell it that that's what you're doing.

(That's pretty weird behaviour: You can't send it a standard password, but you
 can send it the password dressed up as keyboard-interactive auth provided you
 don't tell it that it's a password).

>The "name" field doesn't make much sense for me. First I assumed it was a
>user name, which makes no sense (what is a client supposed to do if it
>differs from the user name given in the USERAUTH_REQUEST message?).

Yeah, I've got problems with those fields as well:

/* Exactly whose name is supplied or what the instruction field is for is left
   unspecified by the RFC (and they may indeed be left empty), so we just skip
   it.  Many implementations feel similarly about this and leave the fields
   empty */

I really wasn't sure what to do with these fields.  Maybe they should be
marked as deprecated or extremely optional if no-one else can figure out what
to put in them either.

>I also want to note that this and SSH_MSG_USERAUTH_INFO_RESPONSE are the only
>request so far that uses a variable number of entries in this way, which
>requires some additional complexity of the code for formatting and parsing
>these messages (at least in my implementation, which uses a general
>formatting function controlled by a format string; I'd prefer not having to
>introduce loop constructs in the format...).

It also makes it extremely difficult to implement in any app that doesn't have
a UI, i.e. where you can't just keep asking the user for input until the
server is satisfied.  Fortunately for standard password auth I've never found
anything that sends more than one prompt, but I'd like to see a bit more
consideration given to non-interactive apps.  Currently the draft seems to
assume that the client-side is tied to a live user in front of a terminal who
can read, interpret, and respond to each prompt, which isn't always the case.

>I think we should also say that after the server has tried all methods on the
>list that it recognizes and supports, it is free to try any other submethods
>of its choice. This is what happens in the common case that the client sends
>an empty submethods list.

Yup, good idea.

Peter.

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Tue, 28 Sep 2004 13:19:27 -0400

On Wednesday, September 29, 2004 03:47:16 +1200 Peter Gutmann 
<[email-masked]> wrote:

>> disabled plain "password" authentication in favour of keyboard-interact,
>> my guess is that they do this in order to work better with PAM.
>
> That was where the example I used in my message came from: Standard
> "password" is disabled, and if you send "password" as the submethod
> OpenSSH immediately drops the connection with a userauth-failed response.
> OTOH if you leave submethods empty, the same "password" authentication
> succeeds without any problems.  This behaviour is compliant with the
> spec, but only because execl( "/usr/games/hack", ... ) would also be
> compliant with the spec.

OK; I'm confused.  In your last message, you seemed to be saying that the 
user had entered "password" as the submethod list because they were just 
randomly making something up instead of actually using the name of a 
submethod the server supports.  Now you seem to be saying that the user is 
trying to stuff the name of an ssh userauth method into the submethod 
field, which is _not_ a list of ssh userauth methods.  Which is it?

Either way, I have no sympathy.  The point of this knob is to allow for 
cases where the user knows what submethods the server supports and has a 
reason for requesting a particular submethod or list of submethods, rather 
than using the server's default behaviour (which is presumably the result 
of configuration by the server's administrator, possibly of something not 
specific to ssh).

If the user does not know what submethods the server supports, they should 
leave this field blank.

> (That's pretty weird behaviour: You can't send it a standard password,
> but you  can send it the password dressed up as keyboard-interactive auth
> provided you  don't tell it that it's a password).

You mean, that you don't randomly make up a submethod name that the server 
has never heard of?  Well, yes.

> It also makes it extremely difficult to implement in any app that doesn't
> have a UI, i.e. where you can't just keep asking the user for input until
> the server is satisfied.  Fortunately for standard password auth I've
> never found anything that sends more than one prompt, but I'd like to see
> a bit more consideration given to non-interactive apps.  Currently the
> draft seems to assume that the client-side is tied to a live user in
> front of a terminal who can read, interpret, and respond to each prompt,
> which isn't always the case.

Well, the name of the method is "keyboard-interactive", which pretty much 
implies that there _is_ a live user you can interact with.  For some of the 
underlying authentication schemes there is nothing you can do about this.

It is worth noting that many of the Linux distributions of which Niels 
speaks use PAM for all of their "initial authentication" needs as well as 
for accounting and session management.  OpenSSH implements 
keyboard-interactive using PAM, but it does _not_ implement "password" that 
way.  This means that if you expect to use PAM to authenticate user logins, 
then the server must be configured not to support "password".

-- Jeff

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Tue, 28 Sep 2004 13:27:36 -0400

> sending a new SSH_MSG_USERAUTH_REQUEST.

Yes, of course the client can do that.  This is a standard part of the 
userauth process, as specified in the ssh-userauth document:

3.1.1:

   An authentication request MAY result in a further exchange of
   messages.  All such messages depend on the authentication method
   used, and the client MAY at any time continue with a new
   SSH_MSG_USERAUTH_REQUEST message, in which case the server MUST
   abandon the previous authentication attempt and continue with the new
   one.

---

From: [email-masked] (Niels Möller)
Date: 28 Sep 2004 21:37:26 +0200

Jeffrey Hutzelman <[email-masked]> writes:

> as for accounting and session management.  OpenSSH implements
> keyboard-interactive using PAM, but it does _not_ implement "password"
> that way.  This means that if you expect to use PAM to authenticate
> user logins, then the server must be configured not to support
> "password".

If I remember the PAM docs right, it's fully possible to use the
accounting and session setup stuff in PAM, without using the PAM
authentication functions. Assuming I have this right, then behaviour
you describe is just a stupid implementation quirk in openssh. And if
I'm wrong, and it isn't possible to use PAM session and accounting
management without also using PAM authentication, that just shows that
the PAM API is badly designed.

I'm quite uncomfortable with this strong coupling between
keyboard-interactive and PAM. The way it is used on these PAM systems
implies that there are two different flavors of the protocol: PAM-less
systems implement userauthentication according to the userauth draft,
PAM systems do it according to the keyboard-interactive draft. And
then clients implementing the userauth draft (but not
keyboard-interactive, which I'd consider more experimental and less
mature), won't interoperate with the latter type of servers.

Regards,
/Niels

---

From: Damien Miller <[email-masked]>
Date: Wed, 29 Sep 2004 10:51:43 +1000

> the PAM API is badly designed.

The PAM API is terribly designed, poorly implemented and a very bad fit
for the SSH protocols. It really shouldn't be used as an example for
kbd-int.

BTW It is possible to use PAM authorization and session management
functions without the authentication functions (OpenSSH does this for
pubkey authentication, for example). In fact, most of the horror of the
PAM API is in its blocking conversation function, which is primarily
used in authentication. This conversation function is the worst
"impedance mismatch" between PAM and SSH.

-d

---

From: Damien Miller <[email-masked]>
Date: Wed, 29 Sep 2004 10:45:24 +1000

> then the server must be configured not to support "password".

Offtopic, but no longer true (as of 3.9p1)

-d

---

From: [email-masked] (Peter Gutmann)
Date: Thu, 30 Sep 2004 17:09:52 +1200

>has never heard of?  Well, yes.

So OpenSSH's behaviour is as follows:

1. It immediately rejects attempts to auth.using any method it hasn't heard
   of.
2. The methods it doesn't reject are undocumented.

This means that the only way to talk to an OpenSSH server is to act as if
there was an implicit requirement that clients MUST NOT set the submethods
field.  With this behaviour OpenSSH isn't even compatible with itself, if a
future version of OpenSSH adds a new submethod, all current versions will
reject attempts to connect from the new version because they'll see a
submethod they don't recognise.

Peter.

---

From: [email-masked] (Peter Gutmann)
Date: Thu, 30 Sep 2004 17:13:57 +1200

[email-masked] (=?iso-8859-1?q?Niels_M=F6ller?=) writes:

>I'm quite uncomfortable with this strong coupling between keyboard-
>interactive and PAM. The way it is used on these PAM systems implies that
>there are two different flavors of the protocol: PAM-less systems implement
>userauthentication according to the userauth draft, PAM systems do it
>according to the keyboard-interactive draft. And then clients implementing
>the userauth draft (but not keyboard-interactive, which I'd consider more
>experimental and less mature), won't interoperate with the latter type of
>servers.

That's exactly the problem I ran into.  There are Linux systems now shipping
that have OpenSSH set up to only allow keyboard-interactive auth, but the auth
they're tunnelling through keyboard-interactive is standard password auth.
Maybe the spec should state that where ambiguities exist (i.e. there are
several ways to do the same thing), the simplest method and/or the one in the
main RFC drafts should take precedence.

Peter.

---

From: Damien Miller <[email-masked]>
Date: Thu, 30 Sep 2004 15:34:08 +1000

> main RFC drafts should take precedence.

That is silly. It would require a SSH server implementation to somehow
peek into what authentication methods PAM is using so that it could
ensure that is isn't inadvertantly offering PAM password authentication
in "keyboard-interactive" instead of PAM auth via "password".

It is a moot point anyway, PAM doesn't provide any standard API for an
application to determine what authentication modules are in use.

-d

---

From: Damien Miller <[email-masked]>
Date: Thu, 30 Sep 2004 17:24:17 +1000

> 1. It immediately rejects attempts to auth.using any method it hasn't heard

This isn't entirely true: you can specify "method1,method2,method3" and
sshd will allow authentication using method3 if the method1 and method2
don't exist.

I'm not sure what you would have us do: kbdint doesn't seem to provide a
way for a server to report supported methods to a client and I don't
think it is correct to just ignore what a client has specified and
continue with a random method that the server picks. Especially since
the protocol isn't required to report exactly what method the server
*has* actually picked in the SSH_MSG_USERAUTH_INFO_REQUEST packets.

> 2. The methods it doesn't reject are undocumented.

Well, they are documented in the source :)

-d

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 30 Sep 2004 11:55:46 -0400

On Thursday, September 30, 2004 15:34:08 +1000 Damien Miller 
<[email-masked]> wrote:

> Peter Gutmann wrote:
>> That's exactly the problem I ran into.  There are Linux systems now
>> shipping that have OpenSSH set up to only allow keyboard-interactive
>> auth, but the auth they're tunnelling through keyboard-interactive is
>> standard password auth. Maybe the spec should state that where
>> ambiguities exist (i.e. there are several ways to do the same thing),
>> the simplest method and/or the one in the main RFC drafts should take
>> precedence.
>

Of course, the administrator knows how PAM is configured, and if the only 
method they will be using is checking passwords against the UNIX passwd 
file, then they can configure "password" support.  But if they want, say, 
Kerberos, which is supported by a PAM module, then turning on "password" 
will DO THE WRONG THING.

In this configuration OpenSSH is not "tunneling standard password".  It's 
not "tunneling" anything; keyboard-interactive is not a wrapper around 
other userauth methods; it is its own method.  The "password" method is 
"the user sends a password up front, which the server either accepts or 
not".  That is not what keyboard-interactive does, and it is not how PAM 
works -- the model used by both of those is "ask the user a question; get 
an answer; repeat until you have everything you need".  In some cases there 
is only one question which happens to be "what is your password?", but that 
is entirely up to the underlying PAM module which is making the 
determination as to who the user is and whether they should be allowed in.

We can argue all we want about whether the model PAM uses is good or bad, 
but we are not in this working group going to change how PAM works, or the 
fact that SSH servers have to support it, or the fact that people will want 
to configure their machines with complex mechanisms that require a 
challenge, or more than one round trip, or something that is only available 
to them as a PAM module.

We can argue all we want about whether it is a good or bad thing that Linux 
distributions come with OpenSSH configured to support keyboard-interactive 
but not password, but we are not in this working group going to change the 
fact that those vendors use PAM and support mechanisms that fall into one 
of the aforementioned categories.  And, we cannot tell people how they must 
configure their machines.

RFC2119 requirements language is not about assuring that all instances of 
everything will interoperate.  It's about making sure that, given any 
arbitrary combination of implementations, it is _possible_ to configure 
them in an interoperable way.  In practice, it is often not possible to get 
both complete interoperability and the features you want.  Linux 
distributors want to support user login via Kerberos or challenge/response 
tokens or icky try-sending-a-password-to-LDAP-and-see-if-it-works, not to 
mention whatever other things someone comes up with in the future, and to 
accomplish that in a clean way they give up interoperability with ssh 
clients that don't support "password".

As an operator of a distributed computing facility where support for our 
infrastructure including authentication is an absolute requirement, and as 
someone who is getting tired of porting an ancient 4.2BSD login to every 
new platform, I support that decision.  It will make life a lot easier for 
me and a lot of other people, at the expense of making life slightly harder 
for the handful of people who have these machines and want to allow remote 
password-based (not public-key) ssh connections from clients which don't 
support keyboard-interactive, by requiring them to <gasp> change a config 
file.

-- Jeffrey T. Hutzelman (N3NHS) <[email-masked]>
   Sr. Research Systems Programmer
   School of Computer Science - Research Computing Facility
   Carnegie Mellon University - Pittsburgh, PA

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 30 Sep 2004 12:04:45 -0400

On Thursday, September 30, 2004 17:09:52 +1200 Peter Gutmann 
<[email-masked]> wrote:

>>> but you  can send it the password dressed up as keyboard-interactive
>>> auth provided you  don't tell it that it's a password).
>>
>> You mean, that you don't randomly make up a submethod name that the
>> server has never heard of?  Well, yes.
>
> 1. It immediately rejects attempts to auth.using any method it hasn't
> heard    of.

I'll agree that's somewhat unfortunate, and I imagine it wouldn't hurt to 
change.  Note that I have nothing to do with OpenSSH, so I can't do 
anything about it in any case.

OTOH, "implementation-defined" does translate to "don't use this unless you 
know in advance what it will do".  The expected behaviour is that clients 
won't set this field unless the user tells it to, and that users won't do 
that unless they know what is the correct thing to request.  It's not 
expected that a non-empty value will work with a server whose capabilities 
you don't know in advance -- doing that would require creating a submethod 
registry, which moves us beyond the realm of "implementation-defined".

That said, from a UI standpoint it seems like it would be useful to have a 
mechanism by which the server can choose to report what submethods it 
supports and provide a description for each, so that client can present the 
user with a set of choices.

-- Jeff

---

From: Jeffrey Hutzelman <[email-masked]>
Date: Thu, 30 Sep 2004 12:12:30 -0400

On Thursday, September 30, 2004 17:24:17 +1000 Damien Miller 
<[email-masked]> wrote:

> continue with a random method that the server picks.

Indeed.  Imagine the scenario where a user has a Kerberos password and some 
kind of challenge/response token.  Now, suppose the user comes in to work 
and leaves the token home.  He wants to use Kerberos, so he sends a 
submethod list containing "kerb5".  The server doesn't support "kerb5", 
because the method is called "krb5".

Now, should the server
(a) fail the exchange, reporting that it can't support any of the methods
    the user asked for
(b) continue by arbitrarily selecting the method that requires the
    challenge/response token which the user left at home

?