代理（Agent）系統(tǒng)能夠幫助開發(fā)人員創(chuàng)建智能的自主系統(tǒng)，因此變得越來越流行。大語言模型（LLM）能夠遵循各種指令，是管理 Agent 的理想選擇，在許多場景中幫助我們盡可能減少人工干預(yù)、處理更多復(fù)雜任務(wù)。例如，Agent 系統(tǒng)解答客戶咨詢的問題，甚至根據(jù)客戶偏好進行交叉銷售。

本文將探討如何使用 Llama-agents 和 Milvus 構(gòu)建 Agent 系統(tǒng)。通過將 LLM 的強大功能與 Milvus 的向量相似性搜索能力相結(jié)合，我們可以創(chuàng)建智能且高效、可擴展的復(fù)雜 Agent 系統(tǒng)。

本文還將探討如何使用不同的 LLM 來實現(xiàn)各種操作。對于較簡單的任務(wù)，我們將使用規(guī)模較小且更價格更低的 Mistral Nemo 模型。對于更復(fù)雜的任務(wù)，如協(xié)調(diào)不同 Agent，我們將使用 Mistral Large 模型。

01.

Llama-agents、Ollama、Mistral Nemo 和 Milvus Lite 簡介

• Llama-agents：LlamaIndex 的擴展，通常與 LLM 配套使用，構(gòu)建強大的 stateful、多 Actor 應(yīng)用。

• Ollama 和 Mistral Nemo: Ollama 是一個 AI 工具，允許用戶在本地計算機上運行大語言模型（如 Mistral Nemo），無需持續(xù)連接互聯(lián)網(wǎng)或依賴外部服務(wù)器。

• Milvus Lite: Milvus 的輕量版，您可以在筆記本電腦、Jupyter Notebook 或 Google Colab 上本地運行 Milvus Lite。它能夠幫助您高效存儲和檢索非結(jié)構(gòu)化數(shù)據(jù)。

Llama-agents 原理

LlamaIndex 推出的 Llama-agents 是一個異步框架，可用于構(gòu)建和迭代生產(chǎn)環(huán)境中的多 Agent 系統(tǒng)，包括多代理通信、分布式工具執(zhí)行、人機協(xié)作等功能！

在 Llama-agents 中，每個 Agent 被視為一個服務(wù)，不斷處理傳入的任務(wù)。每個 Agent 從消息隊列中提取和發(fā)布消息。

02.

安裝依賴

第一步先安裝所需依賴。

!?pip?install?llama-agents?pymilvus?python-dotenv
!?pip?install?llama-index-vector-stores-milvus?llama-index-readers-file?llama-index-embeddings-huggingface?llama-index-llms-ollama?llama-index-llms-mistralai

#?This?is?needed?when?running?the?code?in?a?Notebook
import?nest_asyncio
nest_asyncio.apply()from?dotenv?import?load_dotenv
import?osload_dotenv()

03.

將數(shù)據(jù)加載到 Milvus

從 Llama-index 上下載示例數(shù)據(jù)。其中包含有關(guān) Uber 和 Lyft 的 PDF 文件。

!mkdir?-p?'data/10k/'
!wget?'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10k/uber_2021.pdf'?-O?'data/10k/uber_2021.pdf'
!wget?'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/docs/examples/data/10k/lyft_2021.pdf'?-O?'data/10k/lyft_2021.pdf'

現(xiàn)在，我們可以提取數(shù)據(jù)內(nèi)容，然后使用 Embedding 模型將數(shù)據(jù)轉(zhuǎn)換為 Embedding 向量，最終存儲在 Milvus 向量數(shù)據(jù)庫中。本文使用的模型為 bge-small-en-v1.5 文本 Embedding 模型。該模型較小且資源消耗量更低。

接著，在 Milvus 中創(chuàng)建 Collection 用于存儲和檢索數(shù)據(jù)。本文使用 Milvus 輕量版—— Milvus Lite。Milvus 是一款高性能的向量向量數(shù)據(jù)庫，提供向量相似性搜索能力，適用于搭建 AI 應(yīng)用。僅需通過簡單的 pip install pymilvus 命令即可快速安裝 Milvus Lite。

PDF 文件被轉(zhuǎn)換為向量，我們將向量數(shù)據(jù)庫存儲到 Milvus 中。

from?llama_index.vector_stores.milvus?import?MilvusVectorStore
from?llama_index.core?import?Settings
from?llama_index.embeddings.huggingface?import?HuggingFaceEmbeddingfrom?llama_index.core?import?SimpleDirectoryReader,?VectorStoreIndex,?StorageContext,?load_index_from_storage
from?llama_index.core.tools?import?QueryEngineTool,?ToolMetadata#?Define?the?default?Embedding?model?used?in?this?Notebook.
#?bge-small-en-v1.5?is?a?small?Embedding?model,?it's?perfect?to?use?locally
Settings.embed_model?=?HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5"
)input_files=["./data/10k/lyft_2021.pdf",?"./data/10k/uber_2021.pdf"]#?Create?a?single?Milvus?vector?store
vector_store?=?MilvusVectorStore(uri="./milvus_demo_metadata.db",collection_name="companies_docs"?dim=384,overwrite=False,
)#?Create?a?storage?context?with?the?Milvus?vector?store
storage_context?=?StorageContext.from_defaults(vector_store=vector_store)#?Load?data
docs?=?SimpleDirectoryReader(input_files=input_files).load_data()#?Build?index
index?=?VectorStoreIndex.from_documents(docs,?storage_context=storage_context)#?Define?the?query?engine
company_engine?=?index.as_query_engine(similarity_top_k=3)

04.

定義工具

我們需要定義兩個與我們數(shù)據(jù)相關(guān)的工具。第一個工具提供關(guān)于 Lyft 的信息。第二個工具提供關(guān)于 Uber 的信息。在后續(xù)的內(nèi)容中，我們將進一步探討如何定義一個更廣泛的工具。

#?Define?the?different?tools?that?can?be?used?by?our?Agent.
query_engine_tools?=?[QueryEngineTool(query_engine=company_engine,metadata=ToolMetadata(name="lyft_10k",description=("Provides?information?about?Lyft?financials?for?year?2021.?""Use?a?detailed?plain?text?question?as?input?to?the?tool."),),),QueryEngineTool(query_engine=company_engine,metadata=ToolMetadata(name="uber_10k",description=("Provides?information?about?Uber?financials?for?year?2021.?""Use?a?detailed?plain?text?question?as?input?to?the?tool."),),),
]

05.

使用 Mistral Nemo 設(shè)置 Agent

我們將使用 Mistral Nemo 和 Ollama 限制資源用量、降低應(yīng)用成本。Mistral Nemo + Ollama 的組合能夠幫助我們在本地運行模型。Mistral Nemo 是一個小型 LLM，提供高達 128k Token 的大上下文窗口，這在處理大型文檔時非常有用。此外，該 LLM 還經(jīng)過微調(diào)，可以遵循精確的推理指令、處理多輪對話和生成代碼。

from?llama_index.llms.ollama?import?Ollama
from?llama_index.core.agent?import?AgentRunner,?ReActAgentWorker,?ReActAgent#?Set?up?the?agent
llm?=?Ollama(model="mistral-nemo",?temperature=0.4)
agent?=?ReActAgent.from_tools(query_engine_tools,?llm=llm,?verbose=True)#?Example?usage
response?=?agent.chat("Compare?the?revenue?of?Lyft?and?Uber?in?2021.")
print(response)

生成響應(yīng)如下所示：

>?Running?step?7ed275f6-b0de-4fd7-b2f2-fd551e58bfe2.?Step?input:?Compare?the?revenue?of?Lyft?and?Uber?in?2021.
Thought:?The?current?language?of?the?user?is:?English.?I?need?to?use?tools?to?help?me?answer?the?question.
Action:?lyft_10k
Action?Input:?{'input':?"What?was?Lyft's?total?revenue?in?2021?"}
huggingface/tokenizers:?The?current?process?just?got?forked,?after?parallelism?has?already?been?used.?Disabling?parallelism?to?avoid?deadlocks...
To?disable?this?warning,?you?can?either:-?Avoid?using?`tokenizers`?before?the?fork?if?possible-?Explicitly?set?the?environment?variable?TOKENIZERS_PARALLELISM=(true?|?false)
Observation:?The?total?revenue?for?Lyft?in?2021?was?generated?primarily?from?its?ridesharing?marketplace?connecting?drivers?and?riders,?with?revenue?recognized?from?fees?paid?by?drivers?for?using?the?Lyft?Platform?offerings?in?accordance?with?ASC?606.
>?Running?step?33064fd3-3c3a-42c4-ab5a-e7ebf8a9325b.?Step?input:?None
Thought:?I?need?to?compare?the?revenue?of?Lyft?and?Uber?in?2021.
Action:?uber_10k
Action?Input:?{'input':?"What?was?Uber's?total?revenue?in?2021?"}
Observation:?$17,455
>?Running?step?7eacfef4-d9da-4cbf-ac07-18f2ff6a3951.?Step?input:?None
Thought:?I?have?obtained?Uber's?total?revenue?for?2021.?Now,?I?need?to?compare?it?with?Lyft's.
Action:?lyft_10k
Action?Input:?{'input':?"What?was?Lyft's?total?revenue?in?2021?"}
Observation:?The?total?revenue?for?Lyft?in?2021?was?generated?primarily?from?their?ridesharing?marketplace?connecting?drivers?and?riders.?The?revenue?was?recognized?from?fees?paid?by?drivers?for?using?the?Lyft?Platform?offerings?in?accordance?with?ASC?606.
>?Running?step?88673e15-b74c-4755-8b9c-2b7ef3acea48.?Step?input:?None
Thought:?I?have?obtained?both?Uber's?and?Lyft's?total?revenues?for?2021.?Now,?I?need?to?compare?them.
Action:?Compare
Action?Input:?{'Uber':?'$17,455',?'Lyft':?'$3.6?billion'}
Observation:?Error:?No?such?tool?named?`Compare`.
>?Running?step?bed5941f-74ba-41fb-8905-88525e67b785.?Step?input:?None
Thought:?I?need?to?compare?the?revenues?manually?since?there?isn't?a?'Compare'?tool.
Answer:?In?2021,?Uber's?total?revenue?was?$17.5?billion,?while?Lyft's?total?revenue?was?$3.6?billion.?This?means?that?Uber?generated?approximately?four?times?more?revenue?than?Lyft?in?the?same?year.
Response?without?metadata?filtering:
In?2021,?Uber's?total?revenue?was?$17.5?billion,?while?Lyft's?total?revenue?was?$3.6?billion.?This?means?that?Uber?generated?approximately?four?times?more?revenue?than?Lyft?in?the?same?year.

06.

使用 Milvus 進行元數(shù)據(jù)過濾

雖然為每個公司的文檔定義一個工具代理非常方便，但如果需要處理的文檔很多，這種方法并不具備良好的擴展性。更好的解決方案是使用 Milvus 提供的元數(shù)據(jù)過濾功能。我們可以將來自不同公司的數(shù)據(jù)存儲在同一個 Collection 中，但只檢索特定公司的相關(guān)數(shù)據(jù)，從而節(jié)省時間和資源。

以下代碼展示了如何使用元數(shù)據(jù)過濾功能：

from?llama_index.core.vector_stores?import?ExactMatchFilter,?MetadataFilters#?Example?usage?with?metadata?filtering
filters?=?MetadataFilters(filters=[ExactMatchFilter(key="file_name",?value="lyft_2021.pdf")]
)filtered_query_engine?=?index.as_query_engine(filters=filters)#?Define?query?engine?tools?with?the?filtered?query?engine
query_engine_tools?=?[QueryEngineTool(query_engine=filtered_query_engine,metadata=ToolMetadata(name="company_docs",description=("Provides?information?about?various?companies'?financials?for?year?2021.?""Use?a?detailed?plain?text?question?as?input?to?the?tool."),),),
]#?Set?up?the?agent?with?the?updated?query?engine?tools
agent?=?ReActAgent.from_tools(query_engine_tools,?llm=llm,?verbose=True)

我們的檢索器將過濾數(shù)據(jù)，僅考慮屬于?lyft_2021.pdf文檔的部分?jǐn)?shù)據(jù)。因此，我們應(yīng)該是搜索不到 Uber 相關(guān)的信息和內(nèi)容的。

try:response?=?agent.chat("What?is?the?revenue?of?uber?in?2021?")print("Response?with?metadata?filtering:")print(response)
except?ValueError?as?err:print("we?couldn't?find?the?data,?reached?max?iterations")

讓我們測試一下。當(dāng)我們針對 Uber 2021 年的公司收入進行提問時，Agent 沒有檢索出結(jié)果。

Thought:?The?user?wants?to?know?Uber's?revenue?for?2021.
Action:?company_docs
Action?Input:?{'input':?'Uber?Revenue?2021'}
Observation:?I'm?sorry,?but?based?on?the?provided?context?information,?there?is?no?mention?of?Uber's?revenue?for?the?year?2021.?The?information?primarily?focuses?on?Lyft's?revenue?per?active?rider?and?critical?accounting?policies?and?estimates?related?to?their?financial?statements.
>?Running?step?c0014d6a-e6e9-46b6-af61-5a77ca857712.?Step?input:?None

但當(dāng)我們針對 Lyft 2021 年的公司收入進行提問時，Agent 能夠檢索出正確的答案。

try:response?=?agent.chat("What?is?the?revenue?of?Lyft?in?2021?")print("Response?with?metadata?filtering:")print(response)
except?ValueError?as?err:print("we?couldn't?find?the?data,?reached?max?iterations")

返回結(jié)果如下：

>?Running?step?7f1eebe3-2ebd-47ff-b560-09d09cdd99bd.?Step?input:?What?is?the?revenue?of?Lyft?in?2021?
Thought:?The?current?language?of?the?user?is:?English.?I?need?to?use?a?tool?to?help?me?answer?the?question.
Action:?company_docs
Action?Input:?{'input':?'Lyft?revenue?2021'}
Observation:?Lyft's?revenue?in?2021?was?primarily?generated?from?its?ridesharing?marketplace?connecting?drivers?and?riders.?The?revenue?was?recognized?from?fees?paid?by?drivers?for?using?the?Lyft?Platform?offerings?in?accordance?with?ASC?606.?Additionally,?revenue?per?Active?Rider?reached?an?all-time?high?in?the?fourth?quarter?of?2021?due?to?increased?ride?frequency,?a?shift?towards?higher?revenue?rides,?and?revenues?from?licensing?and?data?access?agreements.
>?Running?step?072a3253-7eee-44e3-a787-397c9cbe80d8.?Step?input:?None
Thought:?The?current?language?of?the?user?is?English.?I?need?to?use?a?tool?to?help?me?answer?the?question.
Action:?company_docs
Action?Input:?{'input':?'Lyft?revenue?2021'}
Observation:?Revenue?for?Lyft?in?2021?was?driven?by?an?increase?in?ride?frequency,?a?shift?towards?higher?revenue?rides?such?as?airport?rides,?and?revenues?from?licensing?and?data?access?agreements.?This?resulted?in?Revenue?per?Active?Rider?reaching?an?all-time?high?in?the?fourth?quarter?of?2021?compared?to?the?previous?quarter.
>?Running?step?6494fe6d-27ad-484f-9204-0c4683bfa1c2.?Step?input:?None
Thought:?The?user?is?asking?for?Lyft's?revenue?in?2021.
Action:?company_docs
Action?Input:?{'input':?'Lyft?revenue?2021'}
Observation:?Lyft's?revenue?in?2021?was?primarily?generated?from?its?ridesharing?marketplace?connecting?drivers?and?riders.?Revenue?per?Active?Rider?reached?a?record?high?in?the?fourth?quarter?of?2021?due?to?increased?ride?frequency?and?a?shift?towards?higher?revenue?rides,?such?as?airport?rides.?Additionally,?revenue?was?boosted?by?licensing?and?data?access?agreements?starting?in?the?second?quarter?of?2021.
>?Running?step?0076b6dd-e7d0-45ac-a39a-4afa5f1aaf47.?Step?input:?None
Answer:?Observation:?Lyft's?total?revenue?in?2021?was?$3.4?billion.
Response?with?metadata?filtering:
Observation:?Lyft's?total?revenue?in?2021?was?$3.4?billion.

07.

使用 LLM 自動創(chuàng)建元數(shù)據(jù)過濾器

現(xiàn)在，讓我們基于用戶問題使用 LLM 自動創(chuàng)建元數(shù)據(jù)過濾器，從而提升 Agent 效率。

from?llama_index.core.prompts.base?import?PromptTemplate#?Function?to?create?a?filtered?query?engine
def?create_query_engine(question):#?Extract?metadata?filters?from?question?using?a?language?modelprompt_template?=?PromptTemplate("Given?the?following?question,?extract?relevant?metadata?filters.\n""Consider?company?names,?years,?and?any?other?relevant?attributes.\n""Don't?write?any?other?text,?just?the?MetadataFilters?object""Format?it?by?creating?a?MetadataFilters?like?shown?in?the?following\n""MetadataFilters(filters=[ExactMatchFilter(key='file_name',?value='lyft_2021.pdf')])\n""If?no?specific?filters?are?mentioned,?returns?an?empty?MetadataFilters()\n""Question:?{question}\n""Metadata?Filters:\n")prompt?=?prompt_template.format(question=question)llm?=?Ollama(model="mistral-nemo")response?=?llm.complete(prompt)metadata_filters_str?=?response.text.strip()if?metadata_filters_str:metadata_filters?=?eval(metadata_filters_str)return?index.as_query_engine(filters=metadata_filters)return?index.as_query_engine()

我們可以將上述 Function 整合到 Agent 中。

#?Example?usage?with?metadata?filtering
question?=?"What?is?Uber?revenue??This?should?be?in?the?file_name:?uber_2021.pdf"
filtered_query_engine?=?create_query_engine(question)#?Define?query?engine?tools?with?the?filtered?query?engine
query_engine_tools?=?[QueryEngineTool(query_engine=filtered_query_engine,metadata=ToolMetadata(name="company_docs_filtering",description=("Provides?information?about?various?companies'?financials?for?year?2021.?""Use?a?detailed?plain?text?question?as?input?to?the?tool."),),),
]#?Set?up?the?agent?with?the?updated?query?engine?tools
agent?=?ReActAgent.from_tools(query_engine_tools,?llm=llm,?verbose=True)response?=?agent.chat(question)
print("Response?with?metadata?filtering:")
print(response)

現(xiàn)在，Agent 使用鍵值file_name 和 uber_2021.pdf 來創(chuàng)建 Metadatafilters。Prompt 越復(fù)雜，Agent 能夠創(chuàng)建更多高級過濾器。

MetadataFilters(filters=[ExactMatchFilter(key='file_name',?value='uber_2021.pdf')])
<class?'str'>
eval:?filters=[MetadataFilter(key='file_name',?value='uber_2021.pdf',?operator=<FilterOperator.EQ:?'=='>)]?condition=<FilterCondition.AND:?'and'>
>?Running?step?a2cfc7a2-95b1-4141-bc52-36d9817ee86d.?Step?input:?What?is?Uber?revenue??This?should?be?in?the?file_name:?uber_2021.pdf
Thought:?The?current?language?of?the?user?is?English.?I?need?to?use?a?tool?to?help?me?answer?the?question.
Action:?company_docs
Action?Input:?{'input':?'Uber?revenue?2021'}
Observation:?$17,455?million

08.

使用 Mistral Large 作為編排系統(tǒng)

Mistral Large 是一款比 Mistral Nemo 更強大的模型，但它也會消耗更多資源。如果僅將其用作編排器，我們可以節(jié)省部分資源，同時享受智能 Agent 帶來的便利。

為什么使用 Mistral Large？

Mistral Large是Mistral AI推出的旗艦型號，具有頂級推理能力，支持復(fù)雜的多語言推理任務(wù)，包括文本理解、轉(zhuǎn)換和代碼生成，非常適合需要大規(guī)模推理能力或高度專業(yè)化的復(fù)雜任務(wù)。其先進的函數(shù)調(diào)用能力正是我們協(xié)調(diào)不同 Agent 時所需的功能。

我們無需針對每個任務(wù)都使用一個重量級的模型，這會對我們的系統(tǒng)造成負(fù)擔(dān)。我們可以使用 Mistral Large 指導(dǎo)其他 Agent 進行特定的任務(wù)。這種方法不僅優(yōu)化了性能，還降低了運營成本，提升系統(tǒng)可擴展性和效率。

Mistral Large 將充當(dāng)中央編排器的角色，協(xié)調(diào)由 Llama-agents 管理的多個 Agent 活動：

• Task Delegation（分派任務(wù)）：當(dāng)收到復(fù)雜查詢時，Mistral Large 確定最合適的 Agent 和工具來處理查詢的各個部分。

• Agent Coordination（代理協(xié)調(diào)）：Llama-agents 管理這些任務(wù)的執(zhí)行情況，確保每個 Agent 接收到必要的輸入，且其輸出被正確處理和整合。

• Result Synthesis（綜合結(jié)果）：Mistral Large 然后將來自各個 Agent 的輸出編譯成一個連貫且全面的響應(yīng)，確保最終輸出大于其各部分的總和。

Llama Agents

將 Mistral Large 作為編排器，并使用 Agent 生成回答。

from?llama_agents?import?(AgentService,ToolService,LocalLauncher,MetaServiceTool,ControlPlaneServer,SimpleMessageQueue,AgentOrchestrator,
)from?llama_index.core.agent?import?FunctionCallingAgentWorker
from?llama_index.llms.mistralai?import?MistralAI#?create?our?multi-agent?framework?components
message_queue?=?SimpleMessageQueue()
control_plane?=?ControlPlaneServer(message_queue=message_queue,orchestrator=AgentOrchestrator(llm=MistralAI('mistral-large-latest')),
)#?define?Tool?Service
tool_service?=?ToolService(message_queue=message_queue,tools=query_engine_tools,running=True,step_interval=0.5,
)#?define?meta-tools?here
meta_tools?=?[await?MetaServiceTool.from_tool_service(t.metadata.name,message_queue=message_queue,tool_service=tool_service,)for?t?in?query_engine_tools
]#?define?Agent?and?agent?service
worker1?=?FunctionCallingAgentWorker.from_tools(meta_tools,llm=MistralAI('mistral-large-latest')
)agent1?=?worker1.as_agent()
agent_server_1?=?AgentService(agent=agent1,message_queue=message_queue,description="Used?to?answer?questions?over?differnet?companies?for?their?Financial?results",service_name="Companies_analyst_agent",
)

import?logging#?change?logging?level?to?enable?or?disable?more?verbose?logging
logging.getLogger("llama_agents").setLevel(logging.INFO)

##?Define?Launcher
launcher?=?LocalLauncher([agent_server_1,?tool_service],control_plane,message_queue,
)

query_str?=?"What?are?the?risk?factors?for?Uber?"
print(launcher.launch_single(query_str))>?Some?key?risk?factors?for?Uber?include?fluctuations?in?the?number?of?drivers?and?merchants?due?to?dissatisfaction?with?the?brand,?pricing?models,?and?safety?incidents.?Investing?in?autonomous?vehicles?may?also?lead?to?driver?dissatisfaction,?as?it?could?reduce?the?need?for?human?drivers.?Additionally,?driver?dissatisfaction?has?previously?led?to?protests,?causing?business?interruptions.

09.

總結(jié)

本文介紹了如何使用 Llama-agents 框架創(chuàng)建和使用代理，該框架由 Mistral Nemo 和 Mistral Large 兩個不同的大語言模型驅(qū)動。我們展示了如何利用不同 LLM 的優(yōu)勢，有效協(xié)調(diào)資源，搭建一個智能、高效的系統(tǒng)。

如果您喜歡本文內(nèi)容，請在 GitHub 上為我們點亮🌟https://github.com/milvus-io/milvus。歡迎在 Milvus 社區(qū)中分享您的見解！

作者介紹

Stephen Batifol

Developer Advocate at Zilliz

推薦閱讀