Eino: React Agent Manual
Introduction
Eino React Agent is an Agent framework that implements React logic, which users can use to quickly and flexibly build and invoke React Agents.
π‘ For the code implementation, see: Implementation Code Directory
Example code path: https://github.com/cloudwego/eino-examples/blob/main/flow/agent/react/react.go
Node Topology & Data Flow Diagram
The React Agent uses compose.Graph as the orchestration scheme at its core. Typically, there are 2 nodes: ChatModel and Tools. All historical messages during the intermediate running process are stored in the state. Before passing all historical messages to the ChatModel, the messages are copied and processed by the MessageModifier, and the processed results are then passed to the ChatModel. This process continues until there are no more tool calls in the messages returned by the ChatModel, and then it returns the final message.
When at least one Tool in the Tools list is configured with ReturnDirectly, the ReAct Agent structure becomes more complex: a Branch is added after the ToolsNode to determine whether a Tool configured with ReturnDirectly is called. If so, it directly ends (END), otherwise, it proceeds as usual to the ChatModel.
Initialization
A ReactAgent initialization function is provided, with mandatory parameters being Model and ToolsConfig, and optional parameters being MessageModifier, MaxStep, ToolReturnDirectly, and StreamToolCallChecker.
go get github.com/cloudwego/eino-ext/components/model/openai@latest
go get github.com/cloudwego/eino@latest
import (
"github.com/cloudwego/eino-ext/components/model/openai"
"github.com/cloudwego/eino/components/model"
"github.com/cloudwego/eino/components/tool"
"github.com/cloudwego/eino/compose"
"github.com/cloudwego/eino/flow/agent/react"
"github.com/cloudwego/eino/schema"
)
func main() {
// Initialize the required chatModel first
toolableChatModel, err := openai.NewChatModel(...)
// Initialize the required tools
tools := compose.ToolsNodeConfig{
InvokableTools: []tool.InvokableTool{mytool},
StreamableTools: []tool.StreamableTool{myStreamTool},
}
// Create an agent
agent, err := react.NewAgent(ctx, react.AgentConfig{
Model: toolableChatModel,
ToolsConfig: tools,
...
})
}
Model
The model receives a ChatModel, and within the agent, it will call the BindTools interface, defined as:
type ChatModel interface {
Generate(ctx context.Context, input []*schema.Message, opts ...Option) (*schema.Message, error)
Stream(ctx context.Context, input []*schema.Message, opts ...Option) (
*schema.StreamReader[*schema.Message], error)
BindTools(tools []*schema.ToolInfo) error
}
Currently, eino provides implementations such as openai and ark. As long as the underlying model supports tool call, it is sufficient.
go get github.com/cloudwego/eino-ext/components/model/openai@latest
go get github.com/cloudwego/eino-ext/components/model/ark@latest
import (
"github.com/cloudwego/eino-ext/components/model/openai"
"github.com/cloudwego/eino-ext/components/model/ark"
)
func openaiExample() {
chatModel, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
BaseURL: os.Getenv("OPENAI_BASE_URL"),
Key: os.Getenv("OPENAI_ACCESS_KEY"),
ByAzure: true,
Model: "{{model name which support tool call}}",
})
agent, err := react.NewAgent(ctx, react.AgentConfig{
Model: chatModel,
ToolsConfig: ...,
})
}
func arkExample() {
arkModel, err := ark.NewChatModel(context.Background(), ark.ChatModelConfig{
APIKey: os.Getenv("ARK_API_KEY"),
Model: os.Getenv("ARK_MODEL"),
})
agent, err := react.NewAgent(ctx, react.AgentConfig{
Model: arkModel,
ToolsConfig: ...,
})
}
ToolsConfig
The toolsConfig type is compose.ToolsNodeConfig. In eino, to build a Tool node, you need to provide information about the Tool and call the Tool’s function. The interface definition for the tool is as follows:
type InvokableRun func(ctx context.Context, arguments string, opts ...Option) (content string, err error)
type StreamableRun func(ctx context.Context, arguments string, opts ...Option) (content *schema.StreamReader[string], err error)
type BaseTool interface {
Info() *schema.ToolInfo
}
// InvokableTool the tool for ChatModel intent recognition and ToolsNode execution.
type InvokableTool interface {
BaseTool
Run() InvokableRun
}
// StreamableTool the stream tool for ChatModel intent recognition and ToolsNode execution.
type StreamableTool interface {
BaseTool
Run() StreamableRun
}
Users can implement the required tool according to the tool’s interface definition. The framework also provides a more straightforward method for constructing tools:
userInfoTool := utils.NewTool(
&schema.ToolInfo{
Name: "user_info",
Desc: "Query user's company, position, and salary information based on the user's name and email",
ParamsOneOf: schema.NewParamsOneOfByParams(map[string]*schema.ParameterInfo{
"name": {
Type: "string",
Desc: "User's name",
},
"email": {
Type: "string",
Desc: "User's email",
},
}),
},
func(ctx context.Context, input *userInfoRequest) (output *userInfoResponse, err error) {
return &userInfoResponse{
Name: input.Name,
Email: input.Email,
Company: "Cool Company LLC.",
Position: "CEO",
Salary: "9999",
}, nil
})
toolConfig := &compose.ToolsNodeConfig{
InvokableTools: []tool.InvokableTool{invokeTool},
}
MessageModifier
MessageModifier is executed each time before all historical messages are passed to the ChatModel. It is defined as:
// modify the input messages before the model is called.
type MessageModifier func(ctx context.Context, input []*schema.Message) []*schema.Message
The framework provides a convenient PersonaModifier to add a system message representing the agent’s personality at the top of the message list. It is used as follows:
import (
"github.com/cloudwego/eino/flow/agent/react"
"github.com/cloudwego/eino/schema"
)
func main() {
persona := `You are an expert golang developer.`
agent, err := react.NewAgent(ctx, &react.AgentConfig{
Model: toolableChatModel,
ToolsConfig: tools,
// MessageModifier
MessageModifier: react.NewPersonaModifier(persona),
})
agent.Generate(ctx, []*schema.Message{schema.UserMessage("Write a hello world code")})
// The actual input to the ChatModel would be
// []*schema.Message{
// {Role: schema.System, Content: "You are an expert golang developer."},
// {Role: schema.Human, Content: "Write a hello world code"}
//}
}
MaxStep
Specifies the maximum running steps for an Agent. Each transition from one node to another counts as one step. The default value is the number of nodes + 2.
Since one loop in the Agent comprises the ChatModel and Tools, it equals 2 steps. Therefore, the default value of 12 allows up to 6 loops. However, since the final step must be a ChatModel response (because the ChatModel determines no further tool runs are needed to return the final result), up to 5 tool runs are possible.
Similarly, if you want the Agent to run up to 10 loops (10 ChatModel + 9 Tools), set MaxStep to 20. If you want the Agent to run up to 20 loops, set MaxStep to 40.
func main() {
agent, err := react.NewAgent(ctx, &react.AgentConfig{
Model: toolableChatModel,
ToolsConfig: tools,
MaxStep: 20,
}
}
ToolReturnDirectly
If you wish for the Agent to directly return the Tool’s Response ToolMessage after the ChatModel selects and executes a specific Tool, you can configure this Tool in ToolReturnDirectly.
a, err = NewAgent(ctx, &AgentConfig{
Model: cm,
ToolsConfig: compose.ToolsNodeConfig{
Tools: []tool.BaseTool{fakeTool, fakeStreamTool},
},
MaxStep: 40,
ToolReturnDirectly: map[string]struct{}{fakeToolName: {}}, // one of the two tools is return directly
})
StreamToolCallChecker
Different models may output tool calls differently in streaming mode: some models (such as OpenAI) directly output tool calls; some models (such as Claude) first output text and then output tool calls. Therefore, different methods are needed to determine this. This field is used to specify the function that determines whether the model’s streaming output contains tool calls.
Optional to fill in, if not filled, the determination will be based on whether the first package contains a tool call.
agent, err := react.NewAgent(ctx, &react.AgentConfig{
Model: toolableChatModel,
ToolsConfig: tools,
StreamToolCallChecker: func(___ context.Context, _sr_ *schema.StreamReader[*schema.Message]) (bool, error) {
defer sr.Close()
msg, err := sr.Recv()
if err != nil {
return false, err
}
if len(msg.ToolCalls) == 0 {
return false, nil
}
return true, nil
}
}
Some models output a piece of text first when they output tool calls in streaming mode (such as Claude), which may cause the default StreamToolCallChecker to mistakenly determine that there is no tool call and directly return. When using such models, you must implement the correct StreamToolCallChecker yourself.
Invocation
Generate
agent, _ := react.NewAgent(...)
var outMessage *schema.Message
outMessage, err = agent.Generate(ctx, []*schema.Message{
schema.UserMessage("Write a hello world program in golang"),
})
Stream
agent, _ := react.NewAgent(...)
var msgReader *schema.StreamReader[*schema.Message]
msgReader, err = agent.Stream(ctx, []*schema.Message{
schema.UserMessage("Write a hello world program in golang"),
})
for {
// msg type is *schema.Message
msg, err := msgReader.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
// finish
break
}
// error
log.Printf("failed to recv: %v\n", err)
return
}
fmt.Print(msg.Content)
}
WithCallbacks
Callback is a function that executes at specific times when the Agent is running. Since the Agent graph only includes ChatModel and ToolsNode, the Agent’s Callback is essentially the Callback for the ChatModel and Tool. The react package provides a helper function to help users quickly build Callback Handlers for these two component types.
// BuildAgentCallback builds a callback handler for the agent.
// e.g.
//
// callback := BuildAgentCallback(modelHandler, toolHandler)
// agent, err := react.NewAgent(ctx, &AgentConfig{})
// agent.Generate(ctx, input, agent.WithComposeOptions(compose.WithCallbacks(callback)))
func BuildAgentCallback(modelHandler *template.ModelCallbackHandler, toolHandler *template.ToolCallbackHandler) callbacks.Handler {
return template.NewHandlerHelper().ChatModel(modelHandler).Tool(toolHandler).Handler()
}
Agent In Graph/Chain
Agent can be embedded as a Lambda into other Graphs:
agent, _ := NewAgent(ctx, &AgentConfig{
Model: cm,
ToolsConfig: compose.ToolsNodeConfig{
Tools: []tool.BaseTool{fakeTool, &fakeStreamToolGreetForTest{}},
},
MaxStep: 40,
})
chain := compose.NewChain[[]*schema.Message, string]()
agentLambda, _ := compose.AnyLambda(agent.Generate, agent.Stream, nil, nil)
chain.
AppendLambda(agentLambda).
AppendLambda(compose.InvokableLambda(func(ctx context.Context, input *schema.Message) (string, error) {
t.Log("got agent response: ", input.Content)
return input.Content, nil
}))
r, _ := chain.Compile(ctx)
res, _ := r.Invoke(ctx, []*schema.Message{{Role: schema.User, Content: "hello"}},
compose.WithCallbacks(callbackForTest))