Feature request: VLAN tagging support

[Schattenschimmer]

It would be very good to see the packets oder frames with the vlan id if it's tagged.
It works with win10pcap, but not with npcap.

Is it possible to realize this feature?

Many thanks!

[guyharris]

It may be that VLAN-tagged packets might be indicated to NDIS with a NDIS_NET_BUFFER_LIST_8021Q_INFO structure, such that you have to extract data from that structure, if present, and insert the VLAN tag back into the packet after the source address in received packets.

If you're transmitting packets, you may also have to remove the VLAN tag and add that structure for devices that have the NDIS_MAC_OPTION_8021Q_VLAN flag set in the MAC options.

[Str1ker17]

Yes! I just installed Win10pcap over npcap and REALLY miss this feature in npcap!
@guyharris probably we can do it ourselves in collaboration. Does it look complex?

[guyharris]

Note that this complicates packet filtering; that happens before the packet is copied to the buffer (so that we don't waste CPU time or buffer space on packets uninteresting to the person or program doing the capturing), so the VLAN tag is out-of-band data and a simple load from the mbuf chainmemory descriptor list won't fetch it.

The good news is that Linux has the exact same issue, as it can also remove VLAN tags so that they're out-of-band data that 1) needs to be handled specially by generated BPF code and 2) must be reinserted into the packet data before being handed to libpcap's caller.

For BPF, Linux's in-kernel BPF code allows some "special" packet offsets, with the uppermost bit set (so that, if interpreted as signed, they're negative) that refer to packet metadata. See the "Possible BPF extensions are shown in the following table" table in Linux Socket Filtering aka Berkeley Packet Filter (BPF). (Its BPF code translates BPF programs to eBPF programs, and the latter might be interpreted or translated to machine code, so finding the code that implements this in the Linux kernel is a bit of work, and, besides, that code is under GPLv2; see, instead, the case BPF_LD|BPF_B|BPF_ABS code in libpcap's bpf_filter.c.)

For re-inserting the VLAN tag, the Linux kernel doesn't do that - it just makes the metadata available; libpcap re-inserts the VLAN tag. For NPF, given that the driver is copying from the MDL to the buffer anyway, it might make sense to re-insert the VLAN tag in that process.

[fyodor]

I discussed this with @dmiller-nmap today and he agrees that there may be a good way to implement this based on how libpcap handles it on Linux. It won't be super easy though. The trick with this and other metadata is how to 1. communicate it to the user without messing with the line format packet and 2. let BPF filters work correctly, especially if we do mess with the frame (like insert VLAN tag). Linux has the same problem of VLAN info being metadata in the kernel. The kernel BPF filter supports "extensions" which are represented by a negative offset, so you can do something like "vlan 2" and it's compiled into something that checks the metadata, not the actual VLAN tag in the frame. User-mode libpcap inserts the vlan tag into the packet data there (802.1q). So this may be easier to implement than some of our previous thoughts about replacing Packet.dll or introducing pcap-ng mode.

[fyodor]

It's also worth noting that some Windows interface drivers strip VLAN tags or drop tagged frames. This is configurable by some drivers. Some useful information on various adapters can be found on the Windows VLAN capture section of the Wireshark wiki.

[dmiller-nmap]

Current brain-dump notes:

Generate BPF with negative offset "extension bit"

• see libpcap's gencode.c
• see Linux include/uapi/linux/filter.h
• see https://andreaskaris.github.io/blog/networking/bpf-and-tcpdump/
• Need to #define some constants to match Linux kernel to activate that code:
  • SKF_AD_OFF: the offset where our extensions live (Linux uses -0x1000)
  • SKF_AD_VLAN_TAG_PRESENT: offset for boolean extension test (BPF_B) for whether extension has VLAN tag info, so we can distinguish VLAN 0 from "no vlan" (Linux: 48)
  • SKF_AD_VLAN_TAG: VLAN tag value (BPF_H) (Linux: 44)

Then make win_bpf_filter handle it correctly. Only valid cases should be:

• BPF_STMT(BPF_LD|BPF_B|BPF_ABS, SKF_AD_OFF + SKF_AD_VLAN_TAG_PRESENT)
• BPF_STMT(BPF_LD|BPF_H|BPF_ABS, SKF_AD_OFF + SKF_AD_VLAN_TAG)

HOWEVER! gencode.c currently (1.10.5) loads VLAN tag as a byte, not halfword. Also, maybe some code will do BPF_LDX?

We will need to let bpf_filter() access the metadata somewhere. Linux does this by putting a sk_buff in BPF_CTX_REG, but we don't have the same BPF engine idea. Could just pass pNBL as an additional argument. Some code:

NDIS_NET_BUFFER_LIST_8021Q_INFO pQinfo = {0};
pQinfo.Value = NET_BUFFER_LIST_INFO(pNBL, Ieee8021QNetBufferListInfo);
u_int16 vlan_id = pQinfo.TagHeader.VlanId;

Then we need to copy VLAN info from metadata to the actual frame. Update NPF_CopyFromNetBufferToNBCopy (pseudocode):

if (vlan metadata) pBuffer = pNBCopy->Buffer + 4;
else pBuffer = pNBCopy->Buffer;
pSrcBuf = NdisGetDataBuffer(pNetBuffer, ulDesired, pBuffer, 1, 0);
...
if (vlan metadata) {
shift 12 bytes from pBuffer to pNBCopy->Buffer;
Insert 802.1Q tag at pNBCopy->Buffer[12];
}

Need to ensure lengths are adjusted properly. Check NPF_CAP_OBJ_SIZE, ulDesired, ulCaplen, etc. Which places use the "real" length and which use the Q-length?

[dmiller-nmap]

Note that Win10pcap does not do this correctly. They misunderstand the operator precedence between << and &, so the class-of-service field is always 0 (SoftEtherVPN/Win10Pcap#12).

[dmiller-nmap]

Recent commits relevant to this issue:

• 2174ea4 - BPF extension to load VLAN metadata for comparison
• cde84d5 - VLAN header added to packet capture data when available
• 637624b - New IOCTL code for querying driver capabilities, BIOCGETINFO
• 36801e0 - Driver capability ID for querying available BPF extensions (Commit message omits this fact, unfortunately)
• 95ddb32 - New Packet.dll function for issuing BIOCGETINFO queries

The remaining tasks are in libpcap, so we will open pull requests for those when they are ready.