update README and reject GPU hardware detection feature

Signed-off-by: louie-tsai <louie.tsai@intel.com>

Author: louie-tsai

tools/recipes/README.md

@@ -2,6 +2,24 @@
 
 Utilities for consuming deployment configurations from [vLLM Recipes](https://recipes.vllm.ai/) and converting them into files that can be used directly with vLLM.
 
+
+## Optimized Deployment Flow
+
+The recipe provides the validated deployment baseline. Hardware and workload
+information are optional inputs that can refine deployment-sensitive values
+before the generated configuration is passed to vLLM.
+
+```mermaid
+flowchart LR
+    R["vLLM Recipe"] --> C["Recipe Converter"]
+    H["Hardware Info (optional)"] --> C
+    W["Workload Info (optional)"] --> C
+    C --> F["config.yml + env.sh"]
+    F --> D["vLLM Docker Image"]
+    D --> E["OpenAI Endpoint"]
+```
+
+
 ## `recipe_json_to_vllm_config.py`
 
 Converts a hardware-specific vLLM Recipes JSON rendering into:
@@ -154,3 +172,108 @@ Strategy discovery and config conversion are separate concerns.
 The converter can resolve any generated strategy JSON exposed by the Recipes API. It currently emits one `config.yml`, so the selected rendering must contain a single `vllm serve` `argv`.
 
 Single-process renderings such as `single_node_tp` can be converted directly. Multi-node, disaggregated prefill/decode, or other multi-process renderings expose fields such as `head_argv`, `worker_argv`, `prefill`, or `decode`; the converter intentionally exits instead of generating an incomplete single-process config.
+
+## Optional Runtime Tuning
+
+The Recipes JSON remains the baseline. vLLM Recipes already provide validated
+model, hardware, strategy, environment variables, and serving arguments.
+Runtime tuning is optional and is intended for parameters whose best value can
+depend on the actual deployment resources or expected request workload.
+
+### Deployment-Time Parameters
+
+The parameters below may already exist in a recipe. They are candidates for
+deployment-time refinement when the recipe value is missing, generic, or based
+on a validation environment that differs from the user's target deployment.
+
+| Runtime parameter | Why it may need deployment-time refinement | Main decision input | Current draft policy |
+| --- | --- | --- | --- |
+| `tensor-parallel-size` | The effective CPU/NUMA topology available to a container or pod can differ from the system used to validate the recipe. | Hardware topology | Use the largest power-of-two TP value that does not exceed the effective NUMA-node count. |
+| `gpu-memory-utilization` | Available memory can differ by machine size, container limits, and other memory use. The vLLM option name is also used by the CPU backend. | Hardware memory + recipe baseline | Compute a conservative safe fraction, reserve 10%, cap at `0.90`, and never increase a smaller recipe value. If the recipe omits it, start from `0.80`. |
+| `max-num-seqs` | The useful scheduler concurrency depends on the number of requests expected to be active at the same time. | Workload concurrency | Set `max-num-seqs` to `--concurrency`. |
+| `max-num-batched-tokens` | The batching budget depends strongly on request input length and concurrency. | Workload token shape + concurrency | Compute `input_tokens * min(concurrency, 8)`, round up to a power of two, and clamp to `2048..32768`. |
+| `data-parallel-size` | The required replica count depends on the requested throughput and the capacity of one replica. | Capacity target | Keep the recipe value today. `--target-qps` is captured for a future capacity-based policy. |
+
+These policies are intentionally isolated in `runtime_tuning.py` so the
+decision rules can evolve as more benchmark data becomes available.
+
+### How Runtime Tuning Determines the Values
+
+The converter can combine the recipe baseline with optional deployment
+information. Different information sources are used for different parameters:
+
+| Information source | How it is obtained | Examples | Parameters it can help determine |
+| --- | --- | --- | --- |
+| vLLM Recipe | Recipes JSON API or direct recipe JSON | model ID, recipe hardware, strategy, existing `argv`, environment variables | Provides the baseline for all parameters and selects the applicable hardware tuning policy. |
+| Hardware (optional) | `--detect-hardware` | effective NUMA nodes, allowed CPUs, physical cores, per-NUMA total/available memory | `tensor-parallel-size`, `gpu-memory-utilization` |
+| Workload (optional) | User CLI inputs | `--input-tokens`, `--output-tokens`, `--concurrency` | `max-num-seqs`, `max-num-batched-tokens`; output length is also available for future KV-cache-aware policies. |
+| SLO / capacity (optional) | User CLI inputs | `--ttft-sla-ms`, `--tpot-sla-ms`, `--target-qps` | Future SLA-aware batching and `data-parallel-size` capacity decisions. |
+
+All additional inputs are optional. If none are supplied, the converter keeps
+the normal Recipes conversion behavior.
+
+The effective precedence is:
+
+```text
+vLLM defaults
+    -> vLLM Recipes baseline
+        -> hardware refinement (optional)
+            -> workload / SLO refinement (optional)
+                -> config.yml + env.sh
+```
+
+For example, hardware detection can refine TP and memory sizing without any
+workload input:
+
+```bash
+python3 tools/recipes/recipe_json_to_vllm_config.py \
+  --model meta-llama/Llama-3.1-8B-Instruct \
+  --hardware xeon6 \
+  --detect-hardware
+```
+
+Workload information can be added when it is known:
+
+```bash
+python3 tools/recipes/recipe_json_to_vllm_config.py \
+  --model meta-llama/Llama-3.1-8B-Instruct \
+  --hardware xeon6 \
+  --detect-hardware \
+  --input-tokens 2048 \
+  --output-tokens 128 \
+  --concurrency 32 \
+  --ttft-sla-ms 3000 \
+  --tpot-sla-ms 100
+```
+
+`--output-tokens`, `--ttft-sla-ms`, and `--tpot-sla-ms` are accepted as policy
+inputs, but the current draft does not force them into an arbitrary formula.
+Output length affects KV-cache residency and request lifetime, while TTFT and
+TPOT constrain how aggressive scheduler batching can be. These inputs can be
+used once benchmark-derived or otherwise validated decision rules are available.
+
+### Runtime-Tuning Hardware Scope
+
+Runtime tuning is selected from the resolved recipe JSON's `hardware` field,
+rather than by inspecting which physical devices happen to be present on the
+host. This matters because a GPU server also exposes its host CPU topology.
+
+The current runtime-tuning policy registry contains `xeon6`. A tuning request
+for unregistered recipe hardware, such as `b200`, fails before host hardware
+detection:
+
+```text
+ERROR: Runtime tuning is not supported for recipe hardware 'b200'.
+Currently supported: xeon6.
+```
+
+Plain recipe conversion remains available for all recipe hardware. The gate only
+applies when optional runtime-tuning inputs are requested.
+
+The implementation remains modular:
+
+- `hardware_detection.py` collects effective CPU/NUMA/memory information.
+- `runtime_tuning.py` owns hardware-policy selection and independent parameter
+  tuning functions.
+- `recipe_json_to_vllm_config.py` resolves the recipe, collects optional inputs,
+  applies the selected policy, and generates `config.yml` and `env.sh`.

tools/recipes/recipe_json_to_vllm_config.py

@@ -58,7 +58,11 @@
     raise SystemExit("PyYAML is required. Install it with: pip install pyyaml") from exc
 
 from hardware_detection import detect_hardware
-from runtime_tuning import WorkloadHints, finetune_runtime_config
+from runtime_tuning import (
+    WorkloadHints,
+    finetune_runtime_config,
+    get_runtime_tuning_policies,
+)
 
 
 DEFAULT_API_BASE = "https://recipes.vllm.ai"
@@ -123,14 +127,18 @@ def parse_args() -> argparse.Namespace:
 
     tuning = p.add_argument_group(
         "optional runtime tuning",
-        "Refine the recipe baseline only when additional information is supplied.",
+        (
+            "Refine the recipe baseline only when additional information is supplied. "
+            "Runtime tuning is enabled only for recipe hardware with a registered "
+            "policy (currently: xeon6)."
+        ),
     )
     tuning.add_argument(
         "--detect-hardware",
         action="store_true",
         help=(
-            "Detect effective CPU/NUMA/memory resources and allow hardware "
-            "policies to override recipe runtime arguments."
+            "Detect effective CPU/NUMA/memory resources and allow the selected "
+            "recipe-hardware policy to override recipe runtime arguments."
         ),
     )
     tuning.add_argument(
@@ -689,7 +697,6 @@ def main() -> int:
         argv = recipe_argv(recipe)
         config = argv_to_config(argv)
 
-        hardware = detect_hardware() if args.detect_hardware else None
         workload = WorkloadHints(
             input_tokens=args.input_tokens,
             output_tokens=args.output_tokens,
@@ -698,22 +705,41 @@ def main() -> int:
             tpot_sla_ms=args.tpot_sla_ms,
             target_qps=args.target_qps,
         )
-        tuning = finetune_runtime_config(
-            config,
-            hardware=hardware,
-            workload=workload,
+        tuning_requested = args.detect_hardware or any(
+            value is not None
+            for value in (
+                workload.input_tokens,
+                workload.output_tokens,
+                workload.concurrency,
+                workload.ttft_sla_ms,
+                workload.tpot_sla_ms,
+                workload.target_qps,
+            )
         )
-        config.update(tuning.overrides)
+
+        tuning = None
+        if tuning_requested:
+            recipe_hardware = recipe.get("hardware")
+            policies = get_runtime_tuning_policies(recipe_hardware)
+            hardware = detect_hardware() if args.detect_hardware else None
+            tuning = finetune_runtime_config(
+                config,
+                hardware=hardware,
+                workload=workload,
+                policies=policies,
+            )
+            config.update(tuning.overrides)
 
         write_config(args.config_out, source, recipe, config)
         write_env(args.env_out, source, recipe)
 
-        if tuning.overrides:
-            print("Applied runtime tuning overrides:")
-            for key, value in tuning.overrides.items():
-                print(f"  {key}: {value}")
-        for note in tuning.notes:
-            print(f"  tuning: {note}")
+        if tuning is not None:
+            if tuning.overrides:
+                print("Applied runtime tuning overrides:")
+                for key, value in tuning.overrides.items():
+                    print(f"  {key}: {value}")
+            for note in tuning.notes:
+                print(f"  tuning: {note}")
     except Exception as exc:
         print(f"ERROR: {exc}", file=sys.stderr)
         return 1

tools/recipes/runtime_tuning.py

@@ -205,6 +205,33 @@ def _resolve_data_parallel_size(
     _resolve_data_parallel_size,
 )
 
+# Select tuning behavior from the hardware requested by the Recipes rendering,
+# not from devices physically present in the host. A GPU server still exposes
+# its host CPU topology, so host inspection alone cannot determine deployment
+# intent.
+HARDWARE_TUNING_POLICIES: dict[str, tuple[Policy, ...]] = {
+    "xeon6": DEFAULT_POLICIES,
+}
+
+
+def get_runtime_tuning_policies(
+    recipe_hardware: object,
+) -> tuple[Policy, ...]:
+    if not isinstance(recipe_hardware, str) or not recipe_hardware:
+        raise ValueError(
+            "Runtime tuning requires the resolved Recipes JSON to declare "
+            "a non-empty `hardware` field."
+        )
+
+    policies = HARDWARE_TUNING_POLICIES.get(recipe_hardware)
+    if policies is None:
+        supported = ", ".join(sorted(HARDWARE_TUNING_POLICIES))
+        raise ValueError(
+            "Runtime tuning is not supported for recipe hardware "
+            f"{recipe_hardware!r}. Currently supported: {supported}."
+        )
+    return policies
+
 
 def finetune_runtime_config(
     config: dict[str, Any],