Support hardware performance counters - WSL2 #4678

AnFunctionArray · 2019-11-14T06:53:30Z

How to enable them - currently they are disabled by default:

[ 0.184805] Performance Events: unsupported p6 CPU model 142 no PMU driver, software events only.

I tried this guide. However when I type Get-VM the list is empty.

The text was updated successfully, but these errors were encountered:

therealkenc · 2019-11-14T18:38:15Z

ghost · 2019-11-14T18:38:20Z

Hi! We've identified this issue as a duplicate of another one that already exists in this repository. This specific instance is being closed in favor of tracking the concern over on the referenced thread.

Thanks for your report!

therealkenc · 2019-11-14T18:40:34Z

Wait your ask is different. References above still worthy, but de-duped.

Dr-Emann · 2019-12-04T21:18:27Z

I think this is a blocker for running rr in wsl2.

zexuan · 2019-12-17T06:03:56Z

Any plan to support hardware performance counters?

AnFunctionArray · 2019-12-17T15:56:24Z

@zexuan You can enable those thinkering with vmwp binary. It is self explanatory once you open it with let's say IDA (you need to patch only the initial calls in the function setting the flags). However you can't rr windows binaries so it's kind of pointless to me.

fberlakovich · 2020-06-03T07:28:40Z

@bsld Could you maybe expand how you did this? Hardware performance counters would be tremendously helpful for my work as I need reverse debugging and gdb record lacks a few features Mozilla rr has.

Is this feature on the WSL2 roadmap at some point?

kruxmeier · 2020-08-26T22:05:04Z

The ProcessorSettings are read from some VmRepository/configuration/settings/processors.
If someone finds that repo you could alter the settings enable_perfmon_pmu, enable_perfmon_lbr, enable_perfmon_pebs, enable_perfmon_ipt.

Or you patch vmwp.exe.

Trass3r · 2020-11-12T19:27:54Z

I tried this guide. However when I type Get-VM the list is empty.

FWIW hcsdiag list lists the container at least.
But Get-VM -Id does not accept that uuid.
Oh and: MicrosoftDocs/windowsserverdocs#4984

Trass3r · 2020-11-12T19:44:06Z

Hardware performance counters would be tremendously helpful for my work as I need reverse debugging and gdb record lacks a few features Mozilla rr has.

rr is also way faster.

AutoFDO is also prevented by the lack of hw perf counters.
And general system profiling with perf and maybe LTTng.

Radrik5 · 2021-02-13T21:24:31Z

Or you patch vmwp.exe.

The patch doesn't work in Windows 10 Build 19042. Patched WSL cannot start and prints "Incorrect function."

Trass3r · 2021-02-14T01:42:37Z

Well since they change the file with every update patching it is not really an option anyway.

GitMensch · 2021-04-22T06:43:30Z

... and patching that way seems to be host-processor specific, too ...

Nufflee · 2021-08-23T04:51:23Z

Any updates on this?

benhillis · 2021-10-19T15:23:06Z

This change is makings its way to the Store version of WSL soon.

GitMensch · 2021-10-19T18:58:48Z

😕 Which change? Do you patch vmvp.exe in the distributed WSL? Which "store version" (only distributions are part of the store, not WSL itself, which is a Windows feature and therefore updated with Windows update)? ❓

Trass3r · 2021-10-19T19:02:17Z

No they finally fixed it but it's gonna be Win11-only it seems.
https://devblogs.microsoft.com/commandline/a-preview-of-wsl-in-the-microsoft-store-is-now-available/

GitMensch · 2021-10-19T19:04:57Z

No they finally fixed it

Yay!

but it's gonna be Win11-only it seems.

Oh no, the store is blocked (and there's no Win11 for a multitude of persons) - @benhillis Can you please ensure that this is getting to the WSL feature, too (or at least, if this is really necessary push that over to W10, too).

benhillis · 2021-10-19T21:01:14Z

@GitMensch - this is a perfect example of why we want to move to shipping outside of Windows. It is a very expensive proposition to change in-market versions of Windows and something like this would be unlikely to meet the bar.

benhillis · 2021-10-19T21:03:19Z

We are evaluating bringing the store version of WSL back to older versions of Windows 10, but cannot yet comment on timelines.

GitMensch · 2021-10-19T22:08:07Z

So with the current store version - can someone please the output of:

perf stat -e br_inst_retired.conditional true
dmesg | grep PMU

eiennohito · 2021-10-20T01:06:31Z

Installed WSL preview from the Windows Store just now (I use W11 Insiders Beta channel):

% dmesg | grep PMU
[    0.184369] Performance Events: PMU not available due to virtualization, using software events only.
% uname -a
Linux ARU-PCDESK 5.10.60.1-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 GNU/Linux

Is it possible to post instructions/minimum versions to diagnose this issue?

benhillis · 2021-11-18T16:28:01Z

@oscarbg - would you please create a new issue for the corrupt bios issue?

Trass3r · 2021-11-18T16:36:03Z

Does anyone know if this also affects /sys/kernel/tracing/ functionality?

StephanDollberg · 2022-03-12T13:51:45Z

Anybody got IPT working with this?

Trass3r · 2022-03-12T18:25:01Z

fwiw /sys/devices/intel_pt does not exist.

GitMensch · 2022-04-04T06:58:15Z

We know that this won't land in Windows 10 - but is there any option to install the new WSL2 release in a Windows 11 "normal" version (no insider)?
Opening the msixbundle fails with an error message that this package cannot be installed with app-installer...

mati865 · 2022-04-04T15:43:57Z

@GitMensch you have to install it using shell with admin rights. I'm away from home and don't have the command on my phone.

GitMensch · 2022-04-04T16:01:20Z

@mati865 Thanks for the pointer - add-appxpackage -Path ".\Microsoft.WSL_0.56.2.0_x64_ARM64.msixbundle" did the trick, the store now says that was installed.

After wsl --shutdown && wsl --update and starting WSL Preview I've tried

uname -r
sudo apt update
sudo apt uprade
sudo apt install linux-perf
sudo sh -c 'echo -1 >/proc/sys/kernel/perf_event_paranoid'
dmesg | grep PMU
perf list | grep br_inst_retired

I don't get any br_inst_retired perf events (which is different than @hbirler) but

5.10.102.1-microsoft-standard-WSL2
[ 0.335568] Performance Events: Fam17h+ core perfctr, AMD PMU driver.

Did I miss something?

GitMensch · 2022-04-04T20:09:01Z

I've missed something - people that check for br_inst_retired on WSL2 and not founding it likely don't use an Intel processor (which provides this event). For AMD it is likely better to test for perf stat -e ex_ret_cond true instead ... (which is available)

lemire · 2022-05-11T20:22:33Z

My tiger-lake processor does not appear to be supported. Is that expected?

I have an up-to-date WSL, and up-to-date Ubuntu and an up-to-date Windows.

yingdam · 2022-05-25T09:23:30Z

My tiger-lake processor does not appear to be supported. Is that expected?

I have an up-to-date WSL, and up-to-date Ubuntu and an up-to-date Windows.

My processor is also tiger-lake, and got the same output with no PMU driver.

Trass3r · 2022-05-25T09:41:47Z

@benhillis probably needs to be updated for newer processors?

GitMensch · 2022-05-25T10:42:31Z

Not sure, but I guess that's kernel related. Would be nice to get a 5.15 kernel, soon (release-ring has 5.10.102 as noted in the images above, not sure if insiders have a newer version).

Trass3r · 2022-05-25T12:23:17Z

Ah yeah worth a try: https://github.com/nathanchance/WSL2-Linux-Kernel

lemire · 2022-05-26T14:15:31Z

Upgrading the kernel to @nathanchance's version does not help me.

jrmuizel · 2022-05-31T16:08:58Z

unsupported p6 comes from https://github.com/torvalds/linux/blob/5bfc75d92efd494db37f5c4c173d3639d4772966/arch/x86/events/intel/p6.c#L272 and this code path is likely being triggered because !cpu_has(&boot_cpu_data, X86_FEATURE_ARCH_PERFMON) from https://github.com/torvalds/linux/blob/a6b450573b912316ad36262bfc70e7c3870c56d1/arch/x86/events/intel/core.c#L5618

X86_FEATURE_ARCH_PERFMON is set when:

	if (c->cpuid_level > 9) {
		unsigned eax = cpuid_eax(10);
		/* Check for version and the number of counters */
		if ((eax & 0xff) && (((eax>>8) & 0xff) > 1))
			set_cpu_cap(c, X86_FEATURE_ARCH_PERFMON);
	}

So the cpuid values in the guest might explain what's going on.

lemire · 2022-05-31T16:41:09Z

Possibly related #8155

Evian-Zhang · 2022-06-09T09:03:26Z

I am using Intel i9-12900K CPU with latest Windows 11, latest WSL2 (using wsl --update) and latest Ubuntu 22.04, but I still can't using Intel PT:

dmesg | grep PMU
[    0.040335] Performance Events: unsupported p6 CPU model 151 no PMU driver, software events only.

Linux kernel version:

uname -a
Linux evian-workstation 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Trass3r · 2022-09-28T21:32:22Z

We are evaluating bringing the store version of WSL back to older versions of Windows 10, but cannot yet comment on timelines.

@benhillis is there any hope the decoupled WSL will still land on Win10?

Trass3r · 2022-11-17T12:40:39Z

It just landed in KB5020030!

GitMensch · 2022-11-17T12:49:12Z

There's no info in https://support.microsoft.com/en-us/topic/november-15-2022-kb5020030-os-builds-19042-2311-19043-2311-19044-2311-and-19045-2311-preview-237a9048-f853-4e29-a3a2-62efdbea95e2 related to WSL...

Trass3r · 2022-11-17T12:56:59Z

I know. But after it you get the 1.0 store version via wsl --update.

clemenswasser · 2022-11-17T15:35:40Z

@Trass3r do you know if there is a KB5020030 equivalent for Win 11 22H2? (Even with WSL v1.0, perf counters are still not working for me, Alderlake with latest Win 11 updates)

Trass3r · 2022-11-17T18:27:56Z

There's no need on Win11, that KB is more of backport of the changes that allow the separation of WSL from the base system.
You're probably looking for #8155.

apjanke · 2024-03-17T14:50:16Z

Hi, folks. Thanks for working on this!

Could someone summarize where the state of hw perf counters on Windows 10 ended up? I read through this ticket and the related linked pages, but I'm not sure I fully followed it.

My read here was that on an up-to-date Windows 10 22H2, doing wsl --install would install the Store ("decoupled") version of WSL, and hw perf counters would (might?) work. (I took this "it just landed...!" comment to mean that Windows 10 no supported Store installation of WSL.) But on my Windows 10 box, the hw perf counters still show up as unsupported, and wsl --install seems to install both the windows-feature and Store versions of WSL. Is that expected?

My experience

I've got a home-made PC "fflewdur" with an AMD 3950x CPU, dual-booting Windows 10, 11, and Linux. To test this, I'd do a fresh install of Windows 10 or 11 from a bootable USB drive of the latest installer (downloaded a couple days ago) to a freshly-formatted volume, do a full Windows Update, full Microsoft Store update, sign in to OneDrive, and then run wsl --install, reboot, run that WSL Ubuntu, do apt update; apt upgrade -y; apt install linux-tools-generic, and then run perf. On Windows 10, I tried the process with wsl --install run both in an elevated Administrator PS prompt and a regular non-elevated PS prompt; same results for each.

On a fresh Win 10 22h2 like that, doing wsl --install produced the following:

Installing: Virtual Machine Platform
Virtual Machine Platform has been installed.
Installing: Windows Subsystem for Linux
Windows Subsystem for Linux has been installed.
Installing: Windows Subsystem for Linux
Windows Subsystem for Linux has been installed.
Installing: Ubuntu
Ubuntu has been installed.
The requested operation is successful. Changes will not be effective until the system is rebooted.

Note there were two "Windows Subsystem for Linux" items there. When watching the installation, I could see each one got its own progress bar.

My machine state after a reboot:

PS C:\Windows\system32> wsl --version
WSL version: 2.1.5.0
Kernel version: 5.15.146.1-2
WSLg version: 1.0.60
MSRDC version: 1.2.5105
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.19045.4170
PS C:\Windows\system32>
PS C:\Windows\system32> systeminfo | select-string OS

Host Name:                 FFLEWDUR
OS Name:                   Microsoft Windows 10 Pro
OS Version:                10.0.19045 N/A Build 19045
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
BIOS Version:              American Megatrends Inc. F5, 9/4/2019

[...]
System Manufacturer:       Gigabyte Technology Co., Ltd.
System Model:              X570 I AORUS PRO WIFI
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: AMD64 Family 23 Model 113 Stepping 0 AuthenticAMD ~3493 Mhz

In that new WSL Ubuntu:

janke@fflewdur:/usr/lib/linux-tools$ uname -a
Linux fflewdur 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
janke@fflewdur:/usr/lib/linux-tools$ dmesg | grep -i 'perf\|PMU'
[    0.072252] Performance Events: PMU not available due to virtualization, using software events only.
[    0.225946]     Performance counters on
janke@fflewdur:/usr/lib/linux-tools$
janke@fflewdur:/usr/lib/linux-tools$ ./5.15.0-100-generic/perf stat ls
5.15.0-100-generic

 Performance counter stats for 'ls':

              0.42 msec task-clock:u              #    0.675 CPUs utilized
                 0      context-switches:u        #    0.000 /sec
                 0      cpu-migrations:u          #    0.000 /sec
                97      page-faults:u             #  230.404 K/sec
   <not supported>      cycles:u
   <not supported>      instructions:u
   <not supported>      branches:u
   <not supported>      branch-misses:u
[...]

I can see a WSL item showing as installed in both the Windows features and Microsoft Store (and Settings > Apps & features) interfaces.

Over on the Windows 11 side of things on this same machine, the same process resulted in a WSL where the hw perf counters are working, and WSL only shows up in Settings > Apps & features, not in the "Turn Windows features on and off".

janke@fflam:/usr/lib/linux-tools$ uname -a
Linux fflam 5.15.146.1-microsoft-standard-WSL2 #1 SMP Thu Jan 11 04:09:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
janke@fflam:/usr/lib/linux-tools$ dmesg | grep -i 'perf\|PMU'
[    0.070620] Performance Events: Fam17h+ core perfctr, AMD PMU driver.
[    0.468557]     Performance counters on
janke@fflam:/usr/lib/linux-tools$ ./5.15.0-100-generic/perf stat ls
5.15.0-100-generic

 Performance counter stats for 'ls':

              0.52 msec task-clock:u              #    0.675 CPUs utilized
                 0      context-switches:u        #    0.000 /sec
                 0      cpu-migrations:u          #    0.000 /sec
                97      page-faults:u             #  186.790 K/sec
            748586      cycles:u                  #    1.442 GHz
             53448      stalled-cycles-frontend:u #    7.14% frontend cycles idle
            209332      stalled-cycles-backend:u  #   27.96% backend cycles idle
            531560      instructions:u            #    0.71  insn per cycle
                                                  #    0.39  stalled cycles per insn
            105118      branches:u                #  202.422 M/sec
              6101      branch-misses:u           #    5.80% of all branches

Trass3r · 2024-03-17T14:57:47Z

https://superuser.com/a/1754527

wsl --install --web-download
wsl --update --pre-release

vimkim · 2024-09-05T06:21:35Z

Any tigerlake updates?

pwsh wsl update

> wsl --update --pre-release --web-download
Checking for updates.
The most recent version of Windows Subsystem for Linux is already installed.

wsl ubuntu kernel version

$ uname -a
Linux WIN-ER7KSLJRTMP 5.15.153.1-microsoft-standard-WSL2 #1 SMP Fri Mar 29 23:14:13 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

dmesg result

$ dmesg | grep -i 'perf'
[    0.078762] Performance Events: unsupported p6 CPU model 170 no PMU driver, software events only.
[    0.327316]     Performance counters on

wsl ubuntu os version

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.4 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.4 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

I wish I could run rr debugger on my laptop.

AnFunctionArray changed the title ~~Support hardware performance counters~~ Support hardware performance counters - WSL2 Nov 14, 2019

therealkenc closed this as completed Nov 14, 2019

ghost added the duplicate label Nov 14, 2019

therealkenc removed the duplicate label Nov 14, 2019

therealkenc reopened this Nov 14, 2019

Trass3r mentioned this issue Aug 24, 2020

can i add perf in this ubuntu #329

Closed

Trass3r mentioned this issue Feb 4, 2021

LTTng kernel tracing support #6137

Open

therealkenc added the feature label Feb 15, 2021

benhillis closed this as completed Nov 18, 2021

nico-abram mentioned this issue May 17, 2022

Add cargo-flamegraph windows support flamegraph-rs/flamegraph#202

Merged

Trass3r mentioned this issue Jun 9, 2022

Consider backporting PMU support for Alder Lake #8155

Open

vjpr mentioned this issue Jul 16, 2022

Running on MacBook Pro, Windows 10 Bootcamp, WSL 2 rr-debugger/rr#3333

Open

Support hardware performance counters - WSL2 #4678

Support hardware performance counters - WSL2 #4678

Comments

AnFunctionArray commented Nov 14, 2019 • edited Loading

therealkenc commented Nov 14, 2019

ghost commented Nov 14, 2019

therealkenc commented Nov 14, 2019

Dr-Emann commented Dec 4, 2019

zexuan commented Dec 17, 2019

AnFunctionArray commented Dec 17, 2019

fberlakovich commented Jun 3, 2020

kruxmeier commented Aug 26, 2020 • edited Loading

Trass3r commented Nov 12, 2020

Trass3r commented Nov 12, 2020 • edited Loading

Radrik5 commented Feb 13, 2021

Trass3r commented Feb 14, 2021

GitMensch commented Apr 22, 2021

Nufflee commented Aug 23, 2021

benhillis commented Oct 19, 2021

GitMensch commented Oct 19, 2021

Trass3r commented Oct 19, 2021

GitMensch commented Oct 19, 2021

benhillis commented Oct 19, 2021

benhillis commented Oct 19, 2021

GitMensch commented Oct 19, 2021

eiennohito commented Oct 20, 2021 • edited Loading

benhillis commented Nov 18, 2021

Trass3r commented Nov 18, 2021

StephanDollberg commented Mar 12, 2022

Trass3r commented Mar 12, 2022

GitMensch commented Apr 4, 2022

mati865 commented Apr 4, 2022

GitMensch commented Apr 4, 2022 • edited Loading

GitMensch commented Apr 4, 2022

lemire commented May 11, 2022

yingdam commented May 25, 2022

Trass3r commented May 25, 2022

GitMensch commented May 25, 2022

Trass3r commented May 25, 2022

lemire commented May 26, 2022

jrmuizel commented May 31, 2022

lemire commented May 31, 2022

Evian-Zhang commented Jun 9, 2022

Trass3r commented Sep 28, 2022

Trass3r commented Nov 17, 2022

GitMensch commented Nov 17, 2022

Trass3r commented Nov 17, 2022

clemenswasser commented Nov 17, 2022

Trass3r commented Nov 17, 2022

apjanke commented Mar 17, 2024 • edited Loading

My experience

Trass3r commented Mar 17, 2024 • edited Loading

vimkim commented Sep 5, 2024 • edited Loading

pwsh wsl update

wsl ubuntu kernel version

dmesg result

wsl ubuntu os version

AnFunctionArray commented Nov 14, 2019 •

edited

Loading

kruxmeier commented Aug 26, 2020 •

edited

Loading

Trass3r commented Nov 12, 2020 •

edited

Loading

eiennohito commented Oct 20, 2021 •

edited

Loading

GitMensch commented Apr 4, 2022 •

edited

Loading

apjanke commented Mar 17, 2024 •

edited

Loading

Trass3r commented Mar 17, 2024 •

edited

Loading

vimkim commented Sep 5, 2024 •

edited

Loading