Expose GIT_CERT_SSH_SHA256 #690

Merged
cypres merged 1 commits from expose-cert-ssh-sha256 into master 2020-11-26 19:32:24 -06:00
cypres commented 2020-11-26 17:17:48 -06:00 (Migrated from github.com)

Seems libgit2 had SHA256 fingerprint support since libgit2 v0.99.0; https://github.com/libgit2/libgit2/pull/5307

This means that if you run libssh2 1.9.0 or newer, most likely the CertificateCheckCallback doesn't actually work for ssh/hostkey based remotes, because the fingerprint is not available.

~In addition I don't think we should be doing memcpy of the structures not set by libgit2.~

I have a test for it as well, but you can't anonymously connect over ssh, so I need a deploy key (which is read only) from https://github.com/libgit2/TestGitRepository. I had intended for the test to be automatically skipped if the ssh feature was not available, and check the hostkey's against the fingerprints from https://api.github.com/meta
If you could provide said deploy key, I'd be happy to add the test to this PR?

Let me know if anything needs to be changed. Thank you.

Seems libgit2 had SHA256 fingerprint support since libgit2 v0.99.0; https://github.com/libgit2/libgit2/pull/5307 This means that if you run libssh2 1.9.0 or newer, most likely the CertificateCheckCallback doesn't actually work for ssh/hostkey based remotes, because the fingerprint is not available. ~In addition I don't think we should be doing memcpy of the structures not set by libgit2.~ I have a test for it as well, but you can't anonymously connect over ssh, so I need a deploy key (which is read only) from https://github.com/libgit2/TestGitRepository. I had intended for the test to be automatically skipped if the ssh feature was not available, and check the hostkey's against the fingerprints from https://api.github.com/meta If you could provide said deploy key, I'd be happy to add the test to this PR? Let me know if anything needs to be changed. Thank you.
lhchavez (Migrated from github.com) reviewed 2020-11-26 18:04:41 -06:00
lhchavez (Migrated from github.com) left a comment

Neat, thanks for adding this!

This means that if you run libssh2 1.9.0 or newer, most likely the CertificateCheckCallback doesn't actually work for ssh/hostkey based remotes, because the fingerprint is not available.

FYI I'm going to upstream some changes to avoid depending on libssh2 altogether soon-ish, since it's a bit hard to work with just the hashes in CertificateCheckCallback and would be better to have the full ssh.PublicKey available. That depends on https://github.com/libgit2/libgit2/pull/5704 being merged, so stay tuned!

I have a test for it as well, but you can't anonymously connect over ssh, so I need a deploy key (which is read only) from https://github.com/libgit2/TestGitRepository. I had intended for the test to be automatically skipped if the ssh feature was not available, and check the hostkey's against the fingerprints from https://api.github.com/meta
If you could provide said deploy key, I'd be happy to add the test to this PR?

we can also do the same thing that libgit2 does: b8cdc9c9c5/ci/test.sh (L109-L150)

(TL;DR spin up an ssh server locally and create the keys on the fly to avoid having to depend on network connectivity, which could be flaky)

Neat, thanks for adding this! > This means that if you run libssh2 1.9.0 or newer, most likely the CertificateCheckCallback doesn't actually work for ssh/hostkey based remotes, because the fingerprint is not available. FYI I'm going to upstream some changes to avoid depending on libssh2 altogether soon-ish, since it's a bit hard to work with just the hashes in `CertificateCheckCallback` and would be better to have the full `ssh.PublicKey` available. That depends on https://github.com/libgit2/libgit2/pull/5704 being merged, so stay tuned! > I have a test for it as well, but you can't anonymously connect over ssh, so I need a deploy key (which is read only) from https://github.com/libgit2/TestGitRepository. I had intended for the test to be automatically skipped if the ssh feature was not available, and check the hostkey's against the fingerprints from https://api.github.com/meta If you could provide said deploy key, I'd be happy to add the test to this PR? we can also do the same thing that libgit2 does: https://github.com/libgit2/libgit2/blob/b8cdc9c9c59c61fc699550af5245c57744c1bcbd/ci/test.sh#L109-L150 (TL;DR spin up an ssh server locally and create the keys on the fly to avoid having to depend on network connectivity, which could be flaky)
lhchavez (Migrated from github.com) commented 2020-11-26 17:56:22 -06:00

since these fields are explicitly zero-initialized in libgit2, it's completely fine if they are memcpy()-ed unconditionally (as the code was doing before), since they are also zero-initialized here and won't be filled with garbage.

if that is still undersirable, we have to fix a documentation bug in L180: that's supposed to be a bit field, not a normal enum! the source of confusion can be traced all the way back to a comment in libgit2, which should be fixed in https://github.com/libgit2/libgit2/pull/5704/files#diff-ae6473f1bca49ada56eedc638fa7c80395ac6a1f5a3e5ee8ae9574d587567715R100

since these fields are explicitly zero-initialized in [libgit2](https://github.com/libgit2/libgit2/blob/b8cdc9c9c59c61fc699550af5245c57744c1bcbd/src/transports/ssh.c#L564), it's completely fine if they are `memcpy()`-ed unconditionally (as the code was doing before), since they are also zero-initialized here and won't be filled with garbage. if that is still undersirable, we have to fix a documentation bug in L180: that's supposed to be a bit field, not a normal enum! the source of confusion can be traced all the way back to a comment in libgit2, which should be fixed in https://github.com/libgit2/libgit2/pull/5704/files#diff-ae6473f1bca49ada56eedc638fa7c80395ac6a1f5a3e5ee8ae9574d587567715R100
cypres (Migrated from github.com) reviewed 2020-11-26 18:19:03 -06:00
cypres (Migrated from github.com) commented 2020-11-26 18:19:02 -06:00

In my testing they don't come back zero-initialized for some reason. I'm not sure why that is?
Running libgit2 1.1.0-1 on arch linux

In my testing they don't come back zero-initialized for some reason. I'm not sure why that is? Running libgit2 1.1.0-1 on arch linux
lhchavez (Migrated from github.com) reviewed 2020-11-26 18:23:17 -06:00
lhchavez (Migrated from github.com) commented 2020-11-26 18:23:17 -06:00

ah then that means that libgit2 was successfully able to ask libssh2 for that particular certificate hash and it should be the correct hash.

unless, of course, there's a bug lurking somewhere :S

ah then that means that libgit2 was successfully able to ask libssh2 for that particular certificate hash and it _should_ be the correct hash. unless, of course, there's a bug lurking somewhere :S
cypres commented 2020-11-26 18:23:56 -06:00 (Migrated from github.com)

This means that if you run libssh2 1.9.0 or newer, most likely the CertificateCheckCallback doesn't actually work for ssh/hostkey based remotes, because the fingerprint is not available.

FYI I'm going to upstream some changes to avoid depending on libssh2 altogether soon-ish, since it's a bit hard to work with just the hashes in CertificateCheckCallback and would be better to have the full ssh.PublicKey available. That depends on libgit2/libgit2#5704 being merged, so stay tuned!

Oh that's awesome! I think it would still be good to expose the hash when we can, especially because it can be back-ported?

Having the full public key would mean we can likely use golang.org/x/crypto/ssh/knownhosts to validate against known hosts files.

(TL;DR spin up an ssh server locally and create the keys on the fly to avoid having to depend on network connectivity, which could be flaky)

True, though don't the other tests in remote_test.go already depend on network connectivity for their HTTPS based tests?

> > This means that if you run libssh2 1.9.0 or newer, most likely the CertificateCheckCallback doesn't actually work for ssh/hostkey based remotes, because the fingerprint is not available. > > FYI I'm going to upstream some changes to avoid depending on libssh2 altogether soon-ish, since it's a bit hard to work with just the hashes in `CertificateCheckCallback` and would be better to have the full `ssh.PublicKey` available. That depends on [libgit2/libgit2#5704](https://github.com/libgit2/libgit2/pull/5704) being merged, so stay tuned! Oh that's awesome! I think it would still be good to expose the hash when we can, especially because it can be back-ported? Having the full public key would mean we can likely use golang.org/x/crypto/ssh/knownhosts to validate against known hosts files. > > (TL;DR spin up an ssh server locally and create the keys on the fly to avoid having to depend on network connectivity, which could be flaky) True, though don't the other tests in remote_test.go already depend on network connectivity for their HTTPS based tests?
lhchavez commented 2020-11-26 18:27:18 -06:00 (Migrated from github.com)

This means that if you run libssh2 1.9.0 or newer, most likely the CertificateCheckCallback doesn't actually work for ssh/hostkey based remotes, because the fingerprint is not available.

FYI I'm going to upstream some changes to avoid depending on libssh2 altogether soon-ish, since it's a bit hard to work with just the hashes in CertificateCheckCallback and would be better to have the full ssh.PublicKey available. That depends on libgit2/libgit2#5704 being merged, so stay tuned!

Oh that's awesome! I think it would still be good to expose the hash when we can, especially because it can be back-ported?

totally! :D

Having the full public key would mean we can likely use golang.org/x/crypto/ssh/knownhosts to validate against known hosts files.

yeah, that's The Dream™️!

(TL;DR spin up an ssh server locally and create the keys on the fly to avoid having to depend on network connectivity, which could be flaky)

True, though don't the other tests in remote_test.go already depend on network connectivity for their HTTPS based tests?

oh yeah, we should fix that at some point. it's easy enough to spin up an http server within the tests.

> > > This means that if you run libssh2 1.9.0 or newer, most likely the CertificateCheckCallback doesn't actually work for ssh/hostkey based remotes, because the fingerprint is not available. > > > > > > FYI I'm going to upstream some changes to avoid depending on libssh2 altogether soon-ish, since it's a bit hard to work with just the hashes in `CertificateCheckCallback` and would be better to have the full `ssh.PublicKey` available. That depends on [libgit2/libgit2#5704](https://github.com/libgit2/libgit2/pull/5704) being merged, so stay tuned! > > Oh that's awesome! I think it would still be good to expose the hash when we can, especially because it can be back-ported? totally! :D > Having the full public key would mean we can likely use golang.org/x/crypto/ssh/knownhosts to validate against known hosts files. yeah, that's The Dream™️! > > (TL;DR spin up an ssh server locally and create the keys on the fly to avoid having to depend on network connectivity, which could be flaky) > > True, though don't the other tests in remote_test.go already depend on network connectivity for their HTTPS based tests? oh yeah, we should fix that at some point. it's easy enough to spin up an http server within the tests.
cypres (Migrated from github.com) reviewed 2020-11-26 18:36:20 -06:00
cypres (Migrated from github.com) commented 2020-11-26 18:36:20 -06:00

Ah, I get it now. It will actually return all the hashes, which is why it's a bit field. The returned hashes didn't match my expectations from the output of ssh-keyscan -t rsa whoami.filippo.io | ssh-keygen -l -E md5 -f - but it's likely me that is holding it wrong. I'll do some more testing, and remove this part of the PR.

Ah, I get it now. It will actually return all the hashes, which is why it's a bit field. The returned hashes didn't match my expectations from the output of `ssh-keyscan -t rsa whoami.filippo.io | ssh-keygen -l -E md5 -f -` but it's likely me that is holding it wrong. I'll do some more testing, and remove this part of the PR.
cypres (Migrated from github.com) reviewed 2020-11-26 18:39:34 -06:00
cypres (Migrated from github.com) commented 2020-11-26 18:39:34 -06:00

Nevermind, I was authenticating with Ed25519 when I thought I was using RSA 🤦, which is why the hash'es didn't match.

Nevermind, I was authenticating with Ed25519 when I thought I was using RSA :facepalm:, which is why the hash'es didn't match.
cypres commented 2020-11-26 18:48:06 -06:00 (Migrated from github.com)

oh yeah, we should fix that at some point. it's easy enough to spin up an http server within the tests.

I can take a stab at that in the next couple of days if you want. Is that blocking this PR from proceeding?

> > oh yeah, we should fix that at some point. it's easy enough to spin up an http server within the tests. I can take a stab at that in the next couple of days if you want. Is that blocking this PR from proceeding?
lhchavez commented 2020-11-26 19:31:35 -06:00 (Migrated from github.com)

oh yeah, we should fix that at some point. it's easy enough to spin up an http server within the tests.

I can take a stab at that in the next couple of days if you want.

yay, thanks :D

Is that blocking this PR from proceeding?

not at all! this is still making forward progress, so let's merge this now

> > oh yeah, we should fix that at some point. it's easy enough to spin up an http server within the tests. > > I can take a stab at that in the next couple of days if you want. yay, thanks :D > Is that blocking this PR from proceeding? not at all! this is still making forward progress, so let's merge this now
lhchavez (Migrated from github.com) approved these changes 2020-11-26 19:31:52 -06:00
Sign in to join this conversation.
No reviewers
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: jcarr/git2go#690
No description provided.