starter.wiki搜索⌘KEN

决策地图

AI：从模型能力、评估和运行控制开始

覆盖 LLM API、SDK、路由、Agent 与嵌入；关注产品任务的可测质量，而非模型排行榜。

Recommendation

先直接使用一个主模型 API；多模型路由、Agent 和额外观测只在任务与风险要求证明必要时加入。

先用真实任务建立评估集，再决定模型、SDK 与路由层；Agent 不是所有工作流的默认答案。

打开起始决策

范围

用这条边界避免在错误层级解决相邻问题。

包含

  • Model and provider API selection for a measured workload
  • Provider-native or provider-normalizing AI application SDK integration
  • The no-agent gate and bounded model-directed agent runtime decision
  • Embedding API selection with an explicit handoff to Data-owned Vector Search
  • Image generation, speech recognition, and speech synthesis
  • Multi-provider gateway and model-routing controls
  • LLM and agent-specific traces, evaluations, token and model cost, and behavior controls

不包含

  • Primary ownership of vector storage, indexing, filtering, and retrieval architecture
  • Deterministic background and scheduled workflow execution
  • Generic application logging and Application Performance Monitoring
  • Media storage and delivery
  • A phantom workflow-orchestration Task or route
125789

重要工具

这些工具代表此决策领域内不同路线；请先打开相关任务页面，再将任何工具视作默认。

决策顺序

先完成基础归属判断，再增加专门能力。

  1. What workload and output modality must be evaluated?

    Define text or reasoning, image, speech recognition, or speech synthesis requirements before selecting a model or provider.

    LLM APIs Image Generation Speech-to-Text Text-to-Speech

  2. How should application code integrate provider calls and tools?

    Choose provider-native access or an SDK abstraction after the model workload is defined, not as a substitute for model evaluation.

    LLM APIs AI SDKs

  3. Does the workflow actually require model-directed autonomy?

    Stop the AI Agents branch when deterministic execution is sufficient; activate runtime, tool, guardrail, state, and human-intervention questions only after the no-agent gate passes.

    AI Agents

  4. Does the application need semantic representations for retrieval or comparison?

    Choose Embedding APIs here, then follow the explicit cross-category handoff to Data-owned Vector Search for storage, indexing, filtering, and retrieval.

    Embedding APIs

  5. Do provider diversity, reliability, budget, policy, or behavior-evaluation needs justify production control layers?

    Evaluate gateway and routing controls separately from AI-specific traces, token and cost semantics, agent behavior, and output evaluation.

    AI Gateways & Model Routing AI Observability

124915

决策分组

每个任务都对应一个独立的用户决策，而非产品功能标签。

13781112

常见混淆

这些边界避免相邻任务被压缩成一个泛泛的候选清单。

LLM APIs and AI SDKs are treated as the same provider decision.
LLM APIs choose model and provider capability; AI SDKs choose the application integration and abstraction layer.
Any multi-step model call is assumed to require an AI Agent.
AI Agents activate only when model-directed autonomy is justified; deterministic work remains with existing Background Jobs or Scheduled Jobs rather than a phantom orchestration page.
Embedding APIs and Vector Search are treated as one AI-owned Task.
Embedding APIs create numerical representations; Data-owned Vector Search stores, indexes, filters, and retrieves them.
Gateway analytics are assumed to cover all AI Observability needs.
Gateways control the request path; AI Observability also covers generations, tools, handoffs, agents, token and cost semantics, and output evaluation across gateway and non-gateway calls.
AI Observability is treated as a replacement for general Logging and Application Performance Monitoring.
Generic telemetry remains in Observability; AI Observability adds model, token, generation, tool, agent, and evaluation semantics.
2456910

暂缓引入

不要在产品真正需要之前引入专门系统。

The workflow can be implemented with deterministic application logic, jobs, or schedules.

Model-directed autonomy introduces runtime, tool, state, guardrail, evaluation, and human-intervention obligations without a justified benefit.

The application uses one provider and does not require centralized fallback, routing, rate, budget, cache, or policy controls.

A gateway adds another request-path control plane that is unnecessary until shared operational requirements exist.

The product has no image or audio workload.

Modality-specific APIs and rights, latency, streaming, storage, and safety constraints are inactive.

49101115

Starter Stack

查看这些决策如何出现在完整应用起始方案中。

官方资源

支撑此分类边界的标准与一手文档。

Official documentationOpenAI model catalog
Official specification repositoryOpenTelemetry GenAI Semantic Conventions

来源

此决策地图使用的主张级参考资料。

  1. 1
    Model catalog

    OpenAI · Accessed Official

  2. 2
    Providers and Models

    Vercel AI SDK · Accessed Official

  3. 3
    Tools

    Vercel AI SDK · Accessed Official

  4. 4
    A practical guide to building agents

    OpenAI · Accessed Official

  5. 5
    Vector embeddings

    OpenAI · Accessed Official

  6. 6
    Vector search in Elasticsearch

    Elastic · Accessed Official

  7. 7
    Image generation

    OpenAI · Accessed Official

  8. 8
    Audio and speech

    OpenAI · Accessed Official

  9. 9
    AI Gateway overview

    Cloudflare · Accessed Official

  10. 10
    Dynamic routing

    Cloudflare · Accessed Official

  11. 11
    Agents SDK tracing

    OpenAI · Accessed Official

  12. 12
    OpenTelemetry GenAI Semantic Conventions

    OpenTelemetry · Accessed Official

  13. 13
    Generative AI metrics

    OpenTelemetry · Accessed Official

  14. 14
    Signals

    OpenTelemetry · Accessed Official

  15. 15
    Evaluation best practices

    OpenAI · Accessed Official