Linux module method

[ilch1]

Add support for generating a symbols file when debug symbols for a Linux kernel are not available.

The types can be obtained from a kernel module that is compiled against the kernel headers (same approach as in volatility2 profiles). Symbol addresses and names come from System.map. This PR provides the following additional features:

• linux banner information can be supplied as a string via a command line flag
• types for symbols defined in System.map can be obtained from a reference symbols file, which is supplied via a command line flag

[jgru]

Dear @ilch1,

I tested this PR, but I did not get it to work with a kernel in version 5.10.0.9 on Debian Bullseye.
I followed the instructions here:
cadb50d#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R68

First, I compiled the module.ko, then I collected System.map as well as banner.txt and generated reference.json .
Afterwards, I ran

sudo ./dwarf2json linux \
  --elf-types ./linux_build_module/module.ko \
  --system-map System.map.txt  \
  --linux-banner "$(cat banner.txt)"  \
  --reference-symbols reference.json

on an exact same system just without the debug-symbol.
However, I received the following error message (even if I tried to specify the vmlinux-file explicitly ):
Failed linux processing: error processing DWARF: decoding dwarf section str at offset 0x0: underflow

Could you clarify, whether this is a bug or an error on my side?

Thank you already in advance for a short reply.

Best regards,
jgru

banner.txt
System.map.txt

[ilch1]

Hi @jgru,

Thanks for trying out the branch and providing feedback. The error is indicative on an issue with parsing the DWARF in module.ko. We are actively working on creating a DWARF testing framework to catch such issues. In the mean time, do you mine uploading module.ko so we can confirm that the error is indeed from parsing that file?

Regards,
ilch1

[jgru]

Hi @ilch1,

thanks for getting back to me. Please see the attached module.ko, hth.
module.ko.tar.gz

Looking foward for this feature becoming usable. Thank you very much for working on this!

Regards,
jgru

[ikelos]

I haven't looked through the PR, but I'd like to make sure there's a way to differentiate ISF files made using a full kernel and those made using the module.ko mechanism, ideally versioned, if possible?

I'm just worried there'll be a symbol that we don't currently put in the module, but that we'll come to rely on and (even though people should support kernels before/after the symbol exists) I suspect some plugins won't fail gracefully.

Providing them a means to do so would be really advantageous (particularly versioned, so if we add a symbol to the module.ko, they can support it based on the version number)...

[ilch1]

Hi @jgru,

I'm not able to reproduce the reported error and can generate an ISF with the files you provided. I'm likely using a different reference.json than you are, but that is probably irrelevant.

Running a couple of other commands would help with debugging this discrepancy.

1. What is the output of go version?

2. The current hypothesis is that the error occurs when processing module.ko. If that is true, I believe running the following should return the same error you were seeing earlier:

   ./dwarf2json linux \
   --elf-types ./linux_build_module/module.ko
   

Could you try ^^^ and report the results?

Thanks,
ilch1

[jgru]

Hi Ilya,

thanks for getting back to me.
I'm running Go version 1.14

go version
go version go1.14 linux/amd64

Running the command as you proposed resulted in the exact same error:

./dwarf2json linux --elf-types ./linux_build_module/module.ko
Failed linux processing: error processing DWARF: decoding dwarf section str at offset 0x0: underflow

Thanks,
jgru

[jgru]

Hi Ilya,

I upgraded Go to version 1.17.5, but nothing changed when running the a/m command, however.
Thanks for your help.

Regards,
jgru

[ilch1]

Hi @ikelos,

You are raising valid points.

I agree that it would be useful to differentiate ISF files created with debug kernels and those created with module.ko method. The ISF files stores metadata for each source used for its creation. This includes names of files and would show that module.ko was used to create it. Perhaps, it would be useful to add some other information, but at the very least, it should be trivial to tell if an ISF was created from module.ko.

I want to clarify that module.ko is only used to get the types for a kernel. Symbols come from System.map and the reference ISF file. I agree that versioning module.ko could be useful if a new type is needed.

[ilch1]

Hi @jgru,

Thanks for this additional information.

It looks like I had a small code change in my local repo that hasn't been committed. It explains the difference in behavior that we were seeing. Can you try to apply the following patch locally and see if it works for you:

diff --git a/main.go b/main.go
index 7f8a24c..7769c35 100644
--- a/main.go
+++ b/main.go
@@ -1007,7 +1007,7 @@ func generateLinux(files FilesToProcess, linuxBanner string) (*vtypeJson, error)
                                endian = "big"
                        }

-                       data, err := DWARF(elfFile)
+                       data, err := elfFile.DWARF()
                        if err != nil {
                                return nil, fmt.Errorf("could not get DWARF from %s: %v", f.FilePath, err)
                        }

It undoes a fix for a different DWARF processing problem, so it won't go in this way.

Thanks,
ilch1

[ikelos]

Yep, it might be worth including an additional field in the file that a) identifies it as based on module.ko and b) contains the version of module.ko that was used? I don't know if there's a way of encoding that as something that would end up in the final .ko file, but might be worth investigating to see if there's a way of achieving it. Maybe a symbol or some data in the DWARF that the parser can interpret as a version number and not include it in the final set of types?

[jgru]

It looks like I had a small code change in my local repo that hasn't been committed. It explains the difference in behavior that we were seeing. Can you try to apply the following patch locally and see if it works for you:

diff --git a/main.go b/main.go
index 7f8a24c..7769c35 100644
--- a/main.go
+++ b/main.go
@@ -1007,7 +1007,7 @@ func generateLinux(files FilesToProcess, linuxBanner string) (*vtypeJson, error)
                                endian = "big"
                        }

-                       data, err := DWARF(elfFile)
+                       data, err := elfFile.DWARF()
                        if err != nil {
                                return nil, fmt.Errorf("could not get DWARF from %s: %v", f.FilePath, err)
                        }

It undoes a fix for a different DWARF processing problem, so it won't go in this way.

Hi @ilch1,

I applied the patch and was able to generate a JSON-file using module.ko.
So your assumption, that elfFile.DWARF() was the culprit, seems to be correct.
Thanks for working on this.

Regards,
jgru

[ilch1]

@ikelos,

It's worth discussing. The current interface to dwarf2json is very generic. It receives a list of dwarf files and the client specifies for each whether to extract types, symbols, or both types+symbols. There's currently no special logic for processing module.ko -- to dwarf2json it is just a DWARF file from which to extract types.

It is definitely possible to change the interface to dwarf2json to make it aware of the module-ko-method for generating ISF files. I'm just wondering if it makes more sense to create a 2nd binary for that specific purpose instead. I'd need to discuss it with @npetroni and @mkonshie.

[ikelos]

I agree, that's why I figured it had to be something specific that existed in the DWARF output of module.ko. There currently is specific logic for handling the banner to be able to embed that, so I figured something similar that could read a specific symbol from the DWARF data and put it into the metadata instead of into the types could work? I'm not sure creating a second file would help since someone could end up using only one and we'd not get the data we needed. Yeah, would be good to discuss ideas, just wanted to make sure it was considered before we moved forward or said it was done and usable... 5:)

[ilch1]

I'm not sure creating a second file would help since someone could end up using only one and we'd not get the data we needed.

I think we would remove options (such as --reference-symbols) from dwarf2json if a 2nd executable was to exist for the sole purpose of creating ISF files using module.ko method. That should hopefully prevent someone from making that mistake. The decision of how split that logic could be separated from what additional versioning information is embedded in module.ko.