Making max-tokens configurable in the benchmark client.

[gangmuk]

🚀 Feature Description and Motivation

Making max-tokens configurable in the benchmark client
max-tokens is the argument that limits the output token lengths. It should be exposed to the user explicitly in case they want to set.

response = await client.chat.completions.create(
            model=model,
            messages=prompt,
            temperature=0,
            max_tokens=max_tokens, // this config is not here before.
            stream=True,
            stream_options={"include_usage": True},
        )

Use Case

configuring output token length

Proposed Solution

No response