跳到内容

模块

DSPy 模块是使用语言模型（LM）的程序的构建块。

  • 每个内置模块都抽象了一种提示技术（例如思维链或 ReAct）。关键在于，它们被泛化处理任何签名。

  • DSPy 模块具有可学习参数（即构成提示和语言模型权重的细小部分），并且可以被调用来处理输入并返回输出。

  • 多个模块可以组合成更大的模块（程序）。DSPy 模块直接借鉴了 PyTorch 中的神经网络模块，但应用于语言模型程序。

如何使用内置模块，例如 dspy.Predict 或 dspy.ChainOfThought？

让我们从最基本的模块 dspy.Predict 开始。在内部，所有其他 DSPy 模块都是使用 dspy.Predict 构建的。我们假设您已经至少对 DSPy 签名有所了解，签名是用于定义我们在 DSPy 中使用的任何模块行为的声明性规范。

要使用一个模块，我们首先通过为其指定签名来声明它。然后，我们使用输入参数调用该模块，并提取输出字段！

sentence = "it's a charming and often affecting journey."  # example from the SST-2 dataset.

# 1) Declare with a signature.
classify = dspy.Predict('sentence -> sentiment: bool')

# 2) Call with input argument(s). 
response = classify(sentence=sentence)

# 3) Access the output.
print(response.sentiment)
输出 
True

当我们声明一个模块时，可以向其传递配置键。

下面，我们将传递 n=5 来请求五个补全。我们还可以传递 temperature 或 max_len 等参数。

让我们使用 dspy.ChainOfThought。在许多情况下，只需将 dspy.Predict 替换为 dspy.ChainOfThought 就能提高质量。

question = "What's something great about the ColBERT retrieval model?"

# 1) Declare with a signature, and pass some config.
classify = dspy.ChainOfThought('question -> answer', n=5)

# 2) Call with input argument.
response = classify(question=question)

# 3) Access the outputs.
response.completions.answer
可能的输出 
['One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.',
 'Its ability to efficiently retrieve relevant information from large document collections.',
 'One great thing about the ColBERT retrieval model is its superior performance compared to other models and its efficient use of pre-trained language models.',
 'One great thing about the ColBERT retrieval model is its superior efficiency and accuracy compared to other models.',
 'One great thing about the ColBERT retrieval model is its ability to incorporate user feedback and support complex queries.']

让我们在这里讨论输出对象。dspy.ChainOfThought 模块通常会在您的签名输出字段之前注入一个 reasoning（推理过程）。

让我们检查（第一个）推理过程和答案！

print(f"Reasoning: {response.reasoning}")
print(f"Answer: {response.answer}")
可能的输出 
Reasoning: We can consider the fact that ColBERT has shown to outperform other state-of-the-art retrieval models in terms of efficiency and effectiveness. It uses contextualized embeddings and performs document retrieval in a way that is both accurate and scalable.
Answer: One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.

无论我们请求一个还是多个补全，都可以访问此信息。

我们还可以将不同的补全作为 Prediction 列表或多个列表（每个字段一个列表）进行访问。

response.completions[3].reasoning == response.completions.reasoning[3]
输出 
True

还有哪些 DSPy 模块？如何使用它们？

其他模块非常相似。它们主要改变了实现您签名的内部行为！

  1. dspy.Predict：基本预测器。不修改签名。处理关键的学习形式（即存储指令、示例以及对语言模型的更新）。

  2. dspy.ChainOfThought：教导语言模型在确定签名响应之前进行逐步思考。

  3. dspy.ProgramOfThought：教导语言模型输出代码，代码的执行结果将决定响应。

  4. dspy.ReAct：一个可以使用工具来实现给定签名的 Agent。

  5. dspy.MultiChainComparison：可以比较来自 ChainOfThought 的多个输出，以产生最终预测。

我们还有一些函数风格的模块

  1. dspy.majority：可以进行基本投票，从一组预测中返回最常见的响应。

DSPy 模块在简单任务上的一些示例。

配置您的 lm 后，尝试下面的示例。调整字段以探索您的语言模型开箱即用地能完成哪些任务。

1
2
math = dspy.ChainOfThought("question -> answer: float")
math(question="Two dice are tossed. What is the probability that the sum equals two?")

可能的输出 

Prediction(
    reasoning='When two dice are tossed, each die has 6 faces, resulting in a total of 6 x 6 = 36 possible outcomes. The sum of the numbers on the two dice equals two only when both dice show a 1. This is just one specific outcome: (1, 1). Therefore, there is only 1 favorable outcome. The probability of the sum being two is the number of favorable outcomes divided by the total number of possible outcomes, which is 1/36.',
    answer=0.0277776
)


1
2
3
4
5
6
7
8
9
def search(query: str) -> list[str]:
    """Retrieves abstracts from Wikipedia."""
    results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)
    return [x['text'] for x in results]

rag = dspy.ChainOfThought('context, question -> response')

question = "What's the name of the castle that David Gregory inherited?"
rag(context=search(question), question=question)

可能的输出 

Prediction(
    reasoning='The context provides information about David Gregory, a Scottish physician and inventor. It specifically mentions that he inherited Kinnairdy Castle in 1664. This detail directly answers the question about the name of the castle that David Gregory inherited.',
    response='Kinnairdy Castle'
)


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
from typing import Literal

class Classify(dspy.Signature):
    """Classify sentiment of a given sentence."""

    sentence: str = dspy.InputField()
    sentiment: Literal['positive', 'negative', 'neutral'] = dspy.OutputField()
    confidence: float = dspy.OutputField()

classify = dspy.Predict(Classify)
classify(sentence="This book was super fun to read, though not the last chapter.")

可能的输出

Prediction(
    sentiment='positive',
    confidence=0.75
)

1
2
3
4
5
6
7
8
text = "Apple Inc. announced its latest iPhone 14 today. The CEO, Tim Cook, highlighted its new features in a press release."

module = dspy.Predict("text -> title, headings: list[str], entities_and_metadata: list[dict[str, str]]")
response = module(text=text)

print(response.title)
print(response.headings)
print(response.entities_and_metadata)

可能的输出 

Apple Unveils iPhone 14
['Introduction', 'Key Features', "CEO's Statement"]
[{'entity': 'Apple Inc.', 'type': 'Organization'}, {'entity': 'iPhone 14', 'type': 'Product'}, {'entity': 'Tim Cook', 'type': 'Person'}]


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
def evaluate_math(expression: str) -> float:
    return dspy.PythonInterpreter({}).execute(expression)

def search_wikipedia(query: str) -> str:
    results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)
    return [x['text'] for x in results]

react = dspy.ReAct("question -> answer: float", tools=[evaluate_math, search_wikipedia])

pred = react(question="What is 9362158 divided by the year of birth of David Gregory of Kinnairdy castle?")
print(pred.answer)

可能的输出

5761.328

如何将多个模块组合成一个更大的程序？

DSPy 就是使用模块的 Python 代码，您可以使用任何控制流程，并在 compile 时内部进行一些巧妙处理来追踪您的语言模型调用。这意味着您可以自由地调用这些模块。

请参阅 多跳搜索 等教程，其模块示例如下所示。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
class Hop(dspy.Module):
    def __init__(self, num_docs=10, num_hops=4):
        self.num_docs, self.num_hops = num_docs, num_hops
        self.generate_query = dspy.ChainOfThought('claim, notes -> query')
        self.append_notes = dspy.ChainOfThought('claim, notes, context -> new_notes: list[str], titles: list[str]')

    def forward(self, claim: str) -> list[str]:
        notes = []
        titles = []

        for _ in range(self.num_hops):
            query = self.generate_query(claim=claim, notes=notes).query
            context = search(query, k=self.num_docs)
            prediction = self.append_notes(claim=claim, notes=notes, context=context)
            notes.extend(prediction.new_notes)
            titles.extend(prediction.titles)

        return dspy.Prediction(notes=notes, titles=list(set(titles)))

如何追踪语言模型使用情况？

版本要求

语言模型使用情况追踪在 DSPy 2.6.16 及更高版本中可用。

DSPy 提供跨所有模块调用的语言模型使用情况内置追踪功能。要启用追踪

dspy.settings.configure(track_usage=True)

启用后，您可以从任何 dspy.Prediction 对象访问使用统计信息

usage = prediction_instance.get_lm_usage()

使用情况数据作为字典返回，该字典将每个语言模型的名称映射到其使用统计信息。这是一个完整示例

import dspy

# Configure DSPy with tracking enabled
dspy.settings.configure(
    lm=dspy.LM("openai/gpt-4o-mini", cache=False),
    track_usage=True
)

# Define a simple program that makes multiple LM calls
class MyProgram(dspy.Module):
    def __init__(self):
        self.predict1 = dspy.ChainOfThought("question -> answer")
        self.predict2 = dspy.ChainOfThought("question, answer -> score")

    def __call__(self, question: str) -> str:
        answer = self.predict1(question=question)
        score = self.predict2(question=question, answer=answer)
        return score

# Run the program and check usage
program = MyProgram()
output = program(question="What is the capital of France?")
print(output.get_lm_usage())

这将输出类似以下的使用统计信息

{
    'openai/gpt-4o-mini': {
        'completion_tokens': 61,
        'prompt_tokens': 260,
        'total_tokens': 321,
        'completion_tokens_details': {
            'accepted_prediction_tokens': 0,
            'audio_tokens': 0,
            'reasoning_tokens': 0,
            'rejected_prediction_tokens': 0,
            'text_tokens': None
        },
        'prompt_tokens_details': {
            'audio_tokens': 0,
            'cached_tokens': 0,
            'text_tokens': None,
            'image_tokens': None
        }
    }
}

使用 DSPy 的缓存功能（内存缓存或通过 litellm 进行磁盘缓存）时，缓存的响应将不计入使用统计信息。例如

# Enable caching
dspy.settings.configure(
    lm=dspy.LM("openai/gpt-4o-mini", cache=True),
    track_usage=True
)

program = MyProgram()

# First call - will show usage statistics
output = program(question="What is the capital of Zambia?")
print(output.get_lm_usage())  # Shows token usage

# Second call - same question, will use cache
output = program(question="What is the capital of Zambia?")
print(output.get_lm_usage())  # Shows empty dict: {}