Embed TUF targets to minimize downloads

[haydentherapper]

Description

For the Golang library, we embed a copy of the target files, and check if the target file matches the target metadata here (a necessary check since the target file may have been updated and would need to be downloaded). Would it be possible to do the same for sigstore-python?

[di]

I think we'd need support from python-tuf before we can include the files here. Seems like this is the relevant issue: theupdateframework/python-tuf#2225 (cc @jku)

[jku]

No, what hayden suggests is totally doable right now -- would just need a few lines of code to pre-populate the target cache from the embedded copy on startup if the target cache does not exist already.

We discussed this during the original implementation and IIRC you or william did not want to embed the target files -- at least not without an automated systems that would make sure the embedded target files are up-to-date.

[woodruffw]

We discussed this during the original implementation and IIRC you or william did not want to embed the target files -- at least not without an automated systems that would make sure the embedded target files are up-to-date.

I think that was probably me 🙂 -- my memory is a little fuzzy, but I think I remember being concerned that embedding here would mean older installs breaking over time. However, that shouldn't actually be true if my understanding of TUF is right, since we'll just refresh from the repo when the embedded targets are stale.

The other concern I remember was the cache priming feeling a little hacky -- IIRC it'd boil down to copying the embedded targets into the TUF repository's directory. But I don't remember why I was opposed to that; I think that's a reasonable tradeoff for minimizing TUF roundtrips 🙂

[jku]

The other concern I remember was the cache priming feeling a little hacky -- IIRC it'd boil down to copying the embedded targets into the TUF repository's directory

into the target cache directory, yes that's how I'd do it. python-tuf will verify anything it finds in the local cache so this does not bypass any security measures if that's what you're thinking.

[tnytown]

Just to clarify, the proposal right now does not include caching of the timestamp and root+1, correct?

Keeping in mind the recent bundled trust root changes, would we want to bundle the other targets, the trust root, or both?

[haydentherapper]

The proposal is to bundle only the target files, not any other metadata (besides the initial TUF root, already embedded). With the new trusted_root file, you could embed only that.

And correct, it shouldn’t break older installs because if the metadata digest doesn’t match what’s embedded, the client should download the freshest copy from the online repo.

[tetsuo-cpp]

@tnytown Are you already looking at this? If not, I can take a stab tomorrow.

[tnytown]

@tnytown Are you already looking at this? If not, I can take a stab tomorrow.

Yep, I have this queued up locally! Let me brush up the tests and check it in