Skip to content

Commit

Permalink
Implement PGO in pipelines for AMD64 architecture; supply training te…
Browse files Browse the repository at this point in the history
…st scenarios (#10071)

Implement PGO in pipelines for AMD64 architecture; supply training test scenarios

## References
- #3075 - Relevant to speed interests there and other linked issues.

## PR Checklist
* [x] Closes #6963
* [x] I work here.
* [x] New UIA Tests added and passed. Manual build runs also tested.

## Detailed Description of the Pull Request / Additional comments
- Creates a new pipeline run for creating instrumented binaries for Profile Guided Optimization (PGO).
- Creates a new suite of UIA tests on the full Windows Terminal app to run PGO training scenarios on instrumented binaries (and incidentally can be used to write other UIA tests later for the full Terminal app.)
- Creates a new NuGet artifact to store trained PGO databases (PGD files) at `Microsoft.Internal.Windows.Terminal.PGODatabase`
- Creates a new NuGet artifact to supply large-scale test content for automated tests at `Microsoft.Internal.Windows.Terminal.TestContent`
- Adjusts the release pipeline to run binaries in PGO optimized mode where content from PGO databases is leveraged at link time to optimize the final release build

The following binaries are trained:
- OpenConsole.exe
- WindowsTerminal.exe
- TerminalApp.dll
- TerminalConnection.dll
- Microsoft.Terminal.Control.dll
- Microsoft.Terminal.Remoting.dll
- Microsoft.Terminal.Settings.Editor.dll
- Microsoft.Terminal.Settings.Model.dll

In the future, adding `<PgoTarget>true</PgoTarget>` to a new `vcxproj` file will automatically enroll the DLL/EXE for PGO instrumentation and optimization going forward.

Two training test scenarios are implemented:
- Smoke test the Terminal by just opening it and typing a bit of text then exiting. (Should help focus on the standard launch path.)
- Optimize bulk text output by launching terminal, outputting `big.txt`, then exiting.

Additional scenarios can be contributed to the `WindowsTerminal_UIATests` project with the `[TestProperty("IsPGO", "true")]` annotation to add them to the suite of scenarios for PGO.

**NOTE:** There are currently no weights applied to the various test scenarios. We will revisit that in the future when/if necessary.

## Validation Steps Performed
- [x] - Training run completed at https://dev.azure.com/ms/terminal/_build?definitionId=492&_a=summary
- [x] - Optimization run completed locally (by forcing `PGOBuildMode` to `Optimize` on my local machine, manually retrieving the databases with NuGet, and building).
- [x] - Validated locally that x86 and ARM64 do not get trained and automatically skip optimization as databases are not present for them.
- [x] - Smoke tested optimized binary versus latest releases. `big.txt` output through CMD is ~11-12seconds prior to PGO and just over 8 seconds with PGO.
  • Loading branch information
miniksa authored May 13, 2021
1 parent a3a2a41 commit 7dadde5
Show file tree
Hide file tree
Showing 48 changed files with 2,496 additions and 95 deletions.
28 changes: 24 additions & 4 deletions .github/actions/spelling/expect/expect.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ ACIOSS
ACover
actctx
ACTCTXW
actionmap
activatable
ACTIVEBORDER
ACTIVECAPTION
adaa
ADDALIAS
ADDB
ADDREF
Expand Down Expand Up @@ -199,6 +199,7 @@ byref
bytearray
bytebuffer
cac
cacafire
callee
cang
capslock
Expand Down Expand Up @@ -298,6 +299,8 @@ codepage
codepath
codepoint
codeproject
coffgroup
coffgrp
coinit
COLLECTIONURI
colorizing
Expand Down Expand Up @@ -487,6 +490,7 @@ dai
DATABLOCK
DATAVIEW
DATAW
datetime
DBatch
dbcs
DBCSCHAR
Expand Down Expand Up @@ -676,6 +680,7 @@ dxgidwm
dxinterop
dxttbmp
eachother
eae
eaf
EASTEUROPE
eb
Expand All @@ -698,6 +703,7 @@ elems
elif
elseif
emacs
emplate
EMPTYBOX
enabledelayedexpansion
endian
Expand Down Expand Up @@ -761,6 +767,7 @@ fcb
fcharset
fclose
fcntl
fd
fdc
FDD
fdopen
Expand All @@ -781,6 +788,7 @@ FILESYSPATH
filesystem
FILETIME
FILETYPE
fileurl
FILEW
FILLATTR
FILLCONSOLEOUTPUT
Expand Down Expand Up @@ -846,6 +854,7 @@ fuzzwrapper
fwdecl
fwe
fwlink
Fx
GAUSSIAN
gb
gci
Expand All @@ -854,6 +863,7 @@ gcy
gdi
gdip
gdirenderer
GENPROFILE
geopol
GETALIAS
GETALIASES
Expand Down Expand Up @@ -973,6 +983,7 @@ hfont
hglobal
hh
hhh
HHmm
hhook
hhx
HIBYTE
Expand Down Expand Up @@ -1003,6 +1014,7 @@ horiz
HORZ
hostable
hostlib
Hostx
HPA
HPAINTBUFFER
hpcon
Expand Down Expand Up @@ -1129,6 +1141,7 @@ INPUTPROCESSORPROFILE
inputrc
Inputreadhandledata
INSERTMODE
installationpath
intellisense
INTERACTIVITYBASE
INTERCEPTCOPYPASTE
Expand Down Expand Up @@ -1430,6 +1443,7 @@ mkdir
MMBB
mmcc
MMCPL
MMdd
mmsystem
MNC
MNOPQ
Expand Down Expand Up @@ -1618,6 +1632,7 @@ numlock
numpad
NUMSCROLL
nupkg
nuspec
NVIDIA
NVR
Nx
Expand Down Expand Up @@ -1760,7 +1775,11 @@ PFNCONSOLECREATEIOTHREAD
PFONT
PFONTENUMDATA
PFS
pgd
pgdn
pgorepro
pgort
PGU
pguid
pgup
PHANDLE
Expand Down Expand Up @@ -1994,7 +2013,6 @@ Replymessage
repositorypath
rescap
Resequence
reserialize
RESETCONTENT
resheader
resizable
Expand Down Expand Up @@ -2271,6 +2289,7 @@ STX
stylecop
SUA
subcompartment
subfolder
subkey
SUBLANG
sublicensable
Expand Down Expand Up @@ -2521,7 +2540,7 @@ UNORM
unparseable
unpause
Unregister
Unregistering
unregistering
untests
untextured
untimes
Expand Down Expand Up @@ -2831,7 +2850,6 @@ xdy
XEncoding
xes
Xes
XES
xff
XFile
XFORM
Expand Down Expand Up @@ -2879,6 +2897,7 @@ YVIRTUALSCREEN
Yw
YWalk
yx
yy
YZ
Zc
ZCmd
Expand All @@ -2890,3 +2909,4 @@ zsh
zu
zxcvbnm
zy
zz
16 changes: 16 additions & 0 deletions NOTICE.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,10 +252,26 @@ DEALINGS IN THE SOFTWARE.
```

# Microsoft Open Source

This product also incorporates source code from other Microsoft open source projects, all licensed under the MIT license.

## `GSL`

**Source**: [https://github.com/microsoft/GSL](https://github.com/microsoft/GSL)

## `Microsoft-UI-XAML`

**Source**: [https://github.com/microsoft/Microsoft-UI-XAML](https://github.com/microsoft/Microsoft-UI-XAML)

## `VirtualDesktopUtils`

**Source**: [https://github.com/microsoft/PowerToys](https://github.com/microsoft/PowerToys)

## `wil`

**Source**: [https://github.com/microsoft/wil](https://github.com/microsoft/wil)

### License

```
Expand Down
37 changes: 37 additions & 0 deletions OpenConsole.sln
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,8 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "UnitTests_Control", "src\ca
{CA5CAD1A-44BD-4AC7-AC72-6CA5B3AB89ED} = {CA5CAD1A-44BD-4AC7-AC72-6CA5B3AB89ED}
EndProjectSection
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "WindowsTerminal.UIA.Tests", "src\cascadia\WindowsTerminal_UIATests\WindowsTerminal.UIA.Tests.csproj", "{F19DACD5-0C6E-40DC-B6E4-767A3200542C}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
AuditMode|Any CPU = AuditMode|Any CPU
Expand Down Expand Up @@ -3256,6 +3258,40 @@ Global
{C323DAEE-B307-4C7B-ACE5-7293CBEFCB5B}.Release|x64.Build.0 = Release|x64
{C323DAEE-B307-4C7B-ACE5-7293CBEFCB5B}.Release|x86.ActiveCfg = Release|Win32
{C323DAEE-B307-4C7B-ACE5-7293CBEFCB5B}.Release|x86.Build.0 = Release|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.AuditMode|Any CPU.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.AuditMode|ARM.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.AuditMode|ARM64.ActiveCfg = Debug|ARM64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.AuditMode|DotNet_x64Test.ActiveCfg = Debug|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.AuditMode|DotNet_x86Test.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.AuditMode|x64.ActiveCfg = Debug|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.AuditMode|x86.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|Any CPU.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|ARM.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|ARM64.ActiveCfg = Debug|ARM64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|ARM64.Build.0 = Debug|ARM64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|DotNet_x64Test.ActiveCfg = Debug|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|DotNet_x86Test.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|x64.ActiveCfg = Debug|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|x64.Build.0 = Debug|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|x86.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Debug|x86.Build.0 = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Fuzzing|Any CPU.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Fuzzing|ARM.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Fuzzing|ARM64.ActiveCfg = Debug|ARM64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Fuzzing|DotNet_x64Test.ActiveCfg = Debug|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Fuzzing|DotNet_x86Test.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Fuzzing|x64.ActiveCfg = Debug|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Fuzzing|x86.ActiveCfg = Debug|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|Any CPU.ActiveCfg = Release|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|ARM.ActiveCfg = Release|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|ARM64.ActiveCfg = Release|ARM64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|ARM64.Build.0 = Release|ARM64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|DotNet_x64Test.ActiveCfg = Release|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|DotNet_x86Test.ActiveCfg = Release|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|x64.ActiveCfg = Release|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|x64.Build.0 = Release|x64
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|x86.ActiveCfg = Release|Win32
{F19DACD5-0C6E-40DC-B6E4-767A3200542C}.Release|x86.Build.0 = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
Expand Down Expand Up @@ -3353,6 +3389,7 @@ Global
{9921CA0A-320C-4460-8623-3A3196E7F4CB} = {59840756-302F-44DF-AA47-441A9D673202}
{05D9052F-D78F-478F-968A-2DE38A6DB996} = {E8F24881-5E37-4362-B191-A3BA0ED7F4EB}
{C323DAEE-B307-4C7B-ACE5-7293CBEFCB5B} = {BDB237B6-1D1D-400F-84CC-40A58FA59C8E}
{F19DACD5-0C6E-40DC-B6E4-767A3200542C} = {BDB237B6-1D1D-400F-84CC-40A58FA59C8E}
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {3140B1B7-C8EE-43D1-A772-D82A7061A271}
Expand Down
Loading

0 comments on commit 7dadde5

Please sign in to comment.