Lots of time of the OpenAPI conversion spent in removeShapes

[kubukoz]

Hi! I had a brief look at profiling our code to see what takes the most time in the build process. One of the things we do in that process is building OpenAPI for a bunch of services. That's actually the longest part of the process, at the moment.

One thing stood out to me - there are two removeShapes calls in the call tree, which together take ~54% of the OpenAPI conversion - 324 seconds for the whole thing (we parallelize this by service, but that only gets us so far).

I wonder if there are some optimization opportunities in there. Here's a breakdown ofremoveShapes:

I wonder if there's something obvious we could do about CleanStructureAndUnionMembers.

[kubukoz]

Now that I think of it... as your model grows and you have more services, computing scoped models by removing shapes is a hopeless cause - it'll just get consistently slower as more shapes are added.

Shouldn't the scopeModelToService be changed to use a builder that only adds the shapes the service still refers to? That seems much easier than trying to optimize here.