Esys_ActivateCredential error #254

lkatalin · 2021-09-29T22:20:26Z

 WARN  keylime_agent > INSECURE: Keylime is using a software TPM emulator rather than a real hardware TPM.
 WARN  keylime_agent > INSECURE: The security of Keylime is NOT linked to a hardware root of trust.
 WARN  keylime_agent > INSECURE: Only use Keylime in this mode for testing or debugging purposes.
 INFO  keylime_agent > Starting server with API version v1.0...
 INFO  keylime_agent > Agent UUID: eb7cd2df-9a40-5019-830e-d5805fc9276f
 INFO  keylime_agent::registrar_agent > Requesting agent registration from http://100.64.14.12:8890/v1.0/agents/eb7cd2df-9a40-5019-830e-d5805fc9276f for eb7cd2df-9a40-5019-830e-d5805fc9276f
 INFO  keylime_agent                  > SUCCESS: Agent eb7cd2df-9a40-5019-830e-d5805fc9276f registered



WARNING:esys:src/tss2-esys/api/Esys_ActivateCredential.c:321:Esys_ActivateCredential_Finish() Received TPM Error
ERROR:esys:src/tss2-esys/api/Esys_ActivateCredential.c:105:Esys_ActivateCredential() Esys Finish ErrorCode (0x00000101)
Error: Tpm { err: Tss2Error(FormatZero(FormatZeroResponseCode { .0: 257, error_number: 1, format_selector: false, version: true, tcg_vendor_indicator: false, severity: false })), kind: Some(Failure), message: "commands not being accepted because of a TPM failure. NOTE: This may be returned by TPM2_GetTestResult() as the testResultparameter" }

This seems similar to #168 , but we are not passing in any --allow-signing flag to the swtpm.
This error appeared without us changing any code, but we did update the TSS-ESAPI version, so I am trying to test whether reverting it will fix the error...

The error is occurring in my test setup on the F34 VM with swtpm as well as in IBM's setup.

The Python agent works normally.

The text was updated successfully, but these errors were encountered:

lkatalin · 2021-09-30T23:59:27Z

@puiterwijk @ueno @lukehinds Any ideas on this one? I cannot get past this error since Tuesday and IBM has not seen it before, either (they got the same error in their environment once they updated the Rust agent code). We also haven't changed much code since the commit where the agent was running without this error, which made me think maybe it is something that was updated in the TSS-ESAPI.

Things I've tried so far:
0. tpm2_clear and other commands to clear ownership of the TPM. Result: error persists.

Reverting to an earlier version of the Rust agent to see where the error is introduced. Results: when I can get something to compile, the error somehow persists in versions from July, where I know I didn't see this error at that time. But I'm also having a lot of trouble getting anything to compile that is not fully updated, due to the error fixed in Fix clippy errors, update to newest TSS-ESAPI #253 , despite rolling back rust / cargo / clippy versions to 1.53.0 as shown in this known-good CI from here.
Making a minimum reproducible error using just activate_credential from the TSS-ESAPI. Results: Basically ended up bringing in most of the Rust agent code anyway to get activate_credential running, so this didn't help.

THS-on · 2021-10-01T11:09:47Z

I've just tested my changes with the latest master on a real Intel TPM and there it works.

 INFO  keylime_agent > Starting server with API version v1.0...
 INFO  keylime_agent > Agent UUID: d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent::registrar_agent > Requesting agent registration from http://127.0.0.1:8890/v1.0/agents/d432fbb3-d2f1-4a97-9ef7-75bd81c00000 for d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent                  > SUCCESS: Agent d432fbb3-d2f1-4a97-9ef7-75bd81c00000 registered
 INFO  keylime_agent::registrar_agent > Requesting agent activation from http://127.0.0.1:8890/v1.0/agents/d432fbb3-d2f1-4a97-9ef7-75bd81c00000 for d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent                  > SUCCESS: Agent d432fbb3-d2f1-4a97-9ef7-75bd81c00000 activated
 INFO  keylime_agent                  > Listening on http://0.0.0.0:9002
 INFO  keylime_agent::quotes_handler  > GET invoked from "172.22.0.4:50740" with uri /v1.0/quotes/identity?nonce=7evD5z9aSYcVPp3cHsQO
 INFO  keylime_agent::quotes_handler  > Calling Identity Quote with nonce: 7evD5z9aSYcVPp3cHsQO

I can try to reproduce this issue on my test setup next week.

ueno · 2021-10-01T11:46:52Z

I've updated my copr package to the latest git master (+ #246 and #249), but couldn't reproduce it as well. Does it happen without any tenant interaction? I'm using this swtpm setup.

lkatalin · 2021-10-01T13:44:39Z

@THS-on @ueno Thank you both for testing. That's actually encouraging that it does not always happen, though puzzling. Especially as it happened in two separate test environments.

@ueno Yes, it happens without any tenant interaction, just after registration, during activation. Something goes wrong when calling Esys_ActivateCredential(). I'll try with your swtpm setup and see if that improves things and report back.

lkatalin · 2021-10-01T19:03:30Z

@ueno I'm trying to understand how your swtpm setup differs from what I regularly use, which is basically this.

The biggest difference appears to be using swtpm socket instead of swtpm chardev, but I'm not sure what the tradeoffs are between these - or if they are equally appropriate? Then we set TPM2TOOLS_TCTI and TCTI to "device:/dev/tpmrm0", which I assume is a more appropriate TCTI for chardev using the kernel resource manager. Does the socket setup also use the kernel resource manager or does it need tabrmd?

When running this setup, I am getting a different error, namely the one you mention in #257 (even on first run).

Edit: I found in my notes that the swtpm socket does indeed have to have tpm2-abrmd, so I will try with this.

Edit2: However, running with the tabrmd seems to require starting tpm2-abrmd and then setting the TCTI var to tabrmd:bus_type=session and not swtpm:port=2321 at least in the terminal running the Rust agent. To be honest it has never been clear to me how the TCTI and TPM2TOOLS_TCTI vars are interpreted by each process here or how they need to match up. Why is the swtpm TCTI simply swtpm + port number but the agent's TCTI is tabrmd? Is it because the agent potentially needs to share the swtpm resource?

lkatalin · 2021-10-01T20:57:41Z

Also: is anyone able to successfully cargo build either:

Rust agent code from commit 890e8c9e226ffb9fe3b34571ed11901661437fc0 or earlier
TSS ESAPI code from commit 382ce8246761f529e8467a1166b6f8c142c30e40 or earlier

... and if so, what version of rustc / cargo / clippy are you using?

ueno · 2021-10-02T08:00:36Z

Since the build failure was caused by the update of zeroize_derive (indirect dependency of tss-esapi), if you can pin the version somehow it should build:

git checkout 890e8c9e226ffb9fe3b34571ed11901661437fc0
cargo update
# Edit Cargo.lock to replace zeroize_derive version to 1.1.0 and remove the checksum line
cargo build

There might be a cleaner way to pin indirect dependencies other than by modifying Cargo.lock.

THS-on · 2021-10-04T09:53:28Z

Using the steps provided by @ueno it works for me:

sudo KEYLIME_AGENT_CONTACT_PORT=9002 KEYLIME_AGENT_CONTACT_IP=127.0.0.1 KEYLIME_CONFIG=/home/user/git/rust-keylime/keylime.conf  RUST_LOG=keylime_agent=trace  ./target/debug/keylime_agent 
 INFO  keylime_agent > Starting server with API version v1.0...
 INFO  keylime_agent > Agent UUID: d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent::registrar_agent > Requesting agent registration from http://127.0.0.1:8890/v1.0/agents/d432fbb3-d2f1-4a97-9ef7-75bd81c00000 for d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent                  > SUCCESS: Agent d432fbb3-d2f1-4a97-9ef7-75bd81c00000 registered
 INFO  keylime_agent::registrar_agent > Requesting agent activation from http://127.0.0.1:8890/v1.0/agents/d432fbb3-d2f1-4a97-9ef7-75bd81c00000 for d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent                  > SUCCESS: Agent d432fbb3-d2f1-4a97-9ef7-75bd81c00000 activated
 INFO  keylime_agent                  > Listening on http://127.0.0.1:9002

TPM: Intel
Cargo and rustc : 1.55.0
clippy: 0.1.55
tpm2-tss: 3.1.0
Distribution: Arch Linux

lkatalin · 2021-10-04T14:17:54Z

Since the build failure was caused by the update of zeroize_derive (indirect dependency of tss-esapi), if you can pin the version somehow it should build:
git checkout 890e8c9e226ffb9fe3b34571ed11901661437fc0
cargo update
# Edit Cargo.lock to replace zeroize_derive version to 1.1.0 and remove the checksum line
cargo build
There might be a cleaner way to pin indirect dependencies other than by modifying Cargo.lock.

@ueno Good idea, thanks, or maybe I can just modify the deps in my fork of the TSS-ESAPI if I continue this debug method.

lkatalin · 2021-10-04T14:19:11Z

Using the steps provided by @ueno it works for me:

sudo KEYLIME_AGENT_CONTACT_PORT=9002 KEYLIME_AGENT_CONTACT_IP=127.0.0.1 KEYLIME_CONFIG=/home/user/git/rust-keylime/keylime.conf  RUST_LOG=keylime_agent=trace  ./target/debug/keylime_agent 
 INFO  keylime_agent > Starting server with API version v1.0...
 INFO  keylime_agent > Agent UUID: d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent::registrar_agent > Requesting agent registration from http://127.0.0.1:8890/v1.0/agents/d432fbb3-d2f1-4a97-9ef7-75bd81c00000 for d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent                  > SUCCESS: Agent d432fbb3-d2f1-4a97-9ef7-75bd81c00000 registered
 INFO  keylime_agent::registrar_agent > Requesting agent activation from http://127.0.0.1:8890/v1.0/agents/d432fbb3-d2f1-4a97-9ef7-75bd81c00000 for d432fbb3-d2f1-4a97-9ef7-75bd81c00000
 INFO  keylime_agent                  > SUCCESS: Agent d432fbb3-d2f1-4a97-9ef7-75bd81c00000 activated
 INFO  keylime_agent                  > Listening on http://127.0.0.1:9002

TPM: Intel Cargo and rustc : 1.55.0 clippy: 0.1.55 tpm2-tss: 3.1.0 Distribution: Arch Linux

@THS-on I notice this is on Arch Linux, are you also able to get it running correctly on F34 with a swtpm?

lkatalin · 2021-10-04T14:20:07Z

Right now I'm building a new F34 test VM to see if this helps since the bug does not reappear for either of you.

THS-on · 2021-10-04T14:29:01Z

@lkatalin should I test with swtpm through QEMU/libvrit integration or running it locally by setting the TCTI environment variable?

lkatalin · 2021-10-04T14:35:13Z

@lkatalin should I test with swtpm through QEMU/libvrit integration or running it locally by setting the TCTI environment variable?

@THS-on I'd try it locally first as I believe that's what my setup is doing and what @ueno 's setup is doing. But I am still figuring out all the swtpm configurations so if you have a better idea then I welcome that too.

lkatalin · 2021-10-04T23:46:35Z

After setting up an entirely new VM test environment, trying various swtpm setups (some of which fail with tabrmd, which is its own separate issue), and trying various versions of the code, the only thing that worked is checking out @ueno 's branch hmac on commit e7f3fed74fa1016934fbb4f4d4e61f4d44d08b0c and this ran without the error (same swtpm setup I normally use with chardev, nothing else different that I know of).

After this, I went to checkout upstream main commit e3e0925f0cd2a1d378fee8634aef4882eb25ac52 and this now also ran without the TPM error (even though I recall this having the error on this branch before). The tss-esapi version in both of these is 6.1.1.

I'm leaving this open because I do not understand it and it may also recur, but I'm no longer actively blocked on it at this moment.

lkatalin · 2021-11-17T16:40:29Z

Re-activating this as it has become a blocker again. It is more thoroughly documented in the above issue. @ueno @THS-on if you get a chance could you try to reproduce the error on Fedora 34 with swtpm with this setup?

lkatalin · 2021-11-17T19:48:18Z

Update: I believe this error was resolved in the rust-tss-esapi here. I will go ahead and close it.

ueno mentioned this issue Oct 4, 2021

"out of memory for session contexts" after restarting the agent #257

Closed

lkatalin mentioned this issue Nov 4, 2021

ActivateCredential error on non-6.1.1 versions (Esys Finish ErrorCode (0x00000101)) parallaxsecond/rust-tss-esapi#285

Closed

lkatalin closed this as completed Nov 17, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Esys_ActivateCredential error #254

Esys_ActivateCredential error #254

lkatalin commented Sep 29, 2021 •

edited

Loading

lkatalin commented Sep 30, 2021

THS-on commented Oct 1, 2021

ueno commented Oct 1, 2021

lkatalin commented Oct 1, 2021

lkatalin commented Oct 1, 2021 •

edited

Loading

lkatalin commented Oct 1, 2021

ueno commented Oct 2, 2021

THS-on commented Oct 4, 2021

lkatalin commented Oct 4, 2021

lkatalin commented Oct 4, 2021

lkatalin commented Oct 4, 2021

THS-on commented Oct 4, 2021

lkatalin commented Oct 4, 2021

lkatalin commented Oct 4, 2021

lkatalin commented Nov 17, 2021

lkatalin commented Nov 17, 2021

Esys_ActivateCredential error #254

Esys_ActivateCredential error #254

Comments

lkatalin commented Sep 29, 2021 • edited Loading

lkatalin commented Sep 30, 2021

THS-on commented Oct 1, 2021

ueno commented Oct 1, 2021

lkatalin commented Oct 1, 2021

lkatalin commented Oct 1, 2021 • edited Loading

lkatalin commented Oct 1, 2021

ueno commented Oct 2, 2021

THS-on commented Oct 4, 2021

lkatalin commented Oct 4, 2021

lkatalin commented Oct 4, 2021

lkatalin commented Oct 4, 2021

THS-on commented Oct 4, 2021

lkatalin commented Oct 4, 2021

lkatalin commented Oct 4, 2021

lkatalin commented Nov 17, 2021

lkatalin commented Nov 17, 2021

lkatalin commented Sep 29, 2021 •

edited

Loading

lkatalin commented Oct 1, 2021 •

edited

Loading