EPSS enricher #1439
EPSS enricher #1439
Conversation
daynewlee
force-pushed
the
yli3/epssUpdater
branch
6 times, most recently
from
e876194
to
32941b1
Compare
|
|
||
| const ( | ||
| // Type is the type of data returned from the Enricher's Enrich method. | ||
| Type = `message/vnd.clair.map.vulnerability; enricher=clair.epss schema=https://csrc.nist.gov/schema/nvd/feed/1.1/cvss-v3.x.json` |
There was a problem hiding this comment.
converting CSV file to json, not sure what the Type should look like.
There was a problem hiding this comment.
Yeah, we need to point to an actual schema here
daynewlee
force-pushed
the
yli3/epssUpdater
branch
3 times, most recently
from
f5058ef
to
7fbe5d3
Compare
daynewlee
force-pushed
the
yli3/epssUpdater
branch
2 times, most recently
from
34f4e08
to
5f69717
Compare
daynewlee
requested review from
crozzy,
RTann and
jvdm
and removed request for
a team
daynewlee
force-pushed
the
yli3/epssUpdater
branch
from
183e018
to
1f7633c
Compare
|
As of Nov 20th, we decide to include model version and score date in the enrichment data |
daynewlee
force-pushed
the
yli3/epssUpdater
branch
from
5712f01
to
843f8f6
Compare
| ctx = zlog.ContextWithValues(ctx, "component", "enricher/epss/Enricher/Configure") | ||
| var cfg Config | ||
| e.c = c | ||
| if f == nil { |
There was a problem hiding this comment.
Will leave this to @crozzy but I see there is no precedence for nil checking, and in the current usage, any nil function is replaced with a noop one
| return Type, []json.RawMessage{b}, nil | ||
| } | ||
|
|
||
| func newItemFeed(record []string, headers []string, modelVersion string, scoreDate string) (driver.EnrichmentRecord, error) { |
There was a problem hiding this comment.
will review this once the other comments are resolved
daynewlee
force-pushed
the
yli3/epssUpdater
branch
3 times, most recently
from
9525df0
to
6bdb39d
Compare
| // | ||
| // It allows for "CVE" to be case insensitive and for dashes and underscores | ||
| // between the different segments. | ||
| var cveRegexp = regexp.MustCompile(`(?i:cve)[-_][0-9]{4}[-_][0-9]{4,}`) |
There was a problem hiding this comment.
This is a copy, we should find a place (probably within the enrichment package) where this will live.
| "github.com/quay/claircore" | ||
| "github.com/quay/claircore/libvuln/driver" | ||
| "github.com/quay/claircore/pkg/tmp" | ||
| "github.com/quay/zlog" |
There was a problem hiding this comment.
This still needs to be updated to format @RTann described
|
|
||
| const ( | ||
| // Type is the type of data returned from the Enricher's Enrich method. | ||
| Type = `message/vnd.clair.map.vulnerability; enricher=clair.epss schema=https://csrc.nist.gov/schema/nvd/feed/1.1/cvss-v3.x.json` |
There was a problem hiding this comment.
Yeah, we need to point to an actual schema here
enricher/epss/epss.go
Outdated
| func (e *Enricher) FetchEnrichment(ctx context.Context, _ driver.Fingerprint) (io.ReadCloser, driver.Fingerprint, error) { | ||
| ctx = zlog.ContextWithValues(ctx, "component", "enricher/epss/Enricher/FetchEnrichment") | ||
| newUUID := uuid.New() | ||
| hint := driver.Fingerprint(newUUID.String()) |
There was a problem hiding this comment.
The updaters are going to run multiple times per day so wouldn't the data be the same in subsequent requests a lot of the time? If yes, we should use the etag/fingerprint to determine if anything has changed, and if not, don't bother processing again.
enricher/epss/epss.go
Outdated
| item := make(map[string]interface{}) // Use interface{} to allow mixed types | ||
| for i, value := range record { | ||
| // epss details are numeric values | ||
| if f, err := strconv.ParseFloat(value, 64); err == nil { |
There was a problem hiding this comment.
Are there certain known values that will not parse into floats that we'd still want?
There was a problem hiding this comment.
I think originally value here was string type
Signed-off-by: daynewlee <[email protected]>
Signed-off-by: daynewlee <[email protected]>
Signed-off-by: daynewlee <[email protected]>
daynewlee
force-pushed
the
yli3/epssUpdater
branch
2 times, most recently
from
4709a48
to
411d044
Compare
Signed-off-by: daynewlee <[email protected]>
daynewlee
force-pushed
the
yli3/epssUpdater
branch
from
5daafdc
to
0383991
Compare
Epss enricher should be able to update, parse EPSS data and enrich vulnerability report with the data