Pytorch 2.3.1

[sdake]

Builds a wheel in Dockerfiles/target containing pytorch 2.3.1. This is needed for vllm because Neural Magic's dynamic quantization uses compute features only available in compute>8.0. Pytorch has a hardcode rejection unless compute>8.9.

[sdake]

vllm-project/vllm#6864

[rstarmer]

Do you want to remove the commented out run statements? If they're not needed, they probably shouldn't be included. Looks good otherwise!

[MostAwesomeDude]

I had to make the given edit in order to build the container.

For what it's worth: the container is still building. By my estimates, that's some 36 CPU-hours. In the future, please consider enrolling containers with CI so that we can use GitHub's CPU hours instead. Here's where it's at, in the final build step:

[8379/8672] Building CXX object test_api/CMakeFiles/test_api.dir/grad_mode.cpp.o

[MostAwesomeDude]

Suggested change

	COPY /workspace/patches/pytorch-compute-86-override.patch /workspace/patches
	COPY pytorch-compute-86-override.patch /workspace/patches

[sdake]

Thanks for catching that. I had a diverged branch and an update that had fixed this wasn't pushed.

[rstarmer]

LGTM

[sdake]

@MostAwesomeDude understood related to build time. I am sort of at a loss as to how to handle building properly. I do not want this repository to be responsible for CI-ing the builds as the builds of our various components take an hour or more. even something simple like cloud-hypervisor takes about 15 minutes. builds. I handled this (In Istio, for example) by creating a separate repo that built and pushed new go binaries. (See dockerfile).

I like @howardjohn 's Istio Release Builder as inspiration, although something small and tactical would be a better start IMO.

have resolved the issues here.