Pi Agent: How One Sentence Becomes a Look at Your Desktop Files

Scenario: You tell Pi, "Show me what files are on my desktop."
Goal: Understand how an agent works with the model, tools, operating system, session storage, and terminal interface.
This article uses ASCII diagrams instead of Mermaid so they can be viewed directly in Sublime Text's Markdown preview.

1. Start with the Right Mental Model

Pi does not "understand" your request by itself, and it does not decide on its own to run commands such as ls or find.

Pi is an agent harness / runtime. The model handles understanding and decisions; Pi is responsible for:

Organizing user input, system rules, history, and tool descriptions into a model request;
Receiving the tool-call requests the model proposes;
Letting tools interact with the outside world, such as the local shell and file system;
Returning tool results to the model safely and in a structured form;
Saving the session and showing the whole process in the terminal.

In one sentence:

The model thinks; Pi lets the model see and act reliably.

2. The Big Picture

┌─────────┐
│   You   │
└────┬────┘
     │ Input: "Show me what files are on my desktop"
     ▼
┌──────────────────────────┐
│ Pi TUI (terminal interface)│
│ Takes input, shows progress │
└────┬─────────────────────┘
     ▼
┌──────────────────────────┐
│ AgentSession / Agent Core │
│ Session, context, tools, queue │
└────┬─────────────────────┘
     │ Sends: rules + history + tool schemas + user question
     ▼
┌──────────────────────────┐
│          LLM API           │
│ Understands task, picks tool │
└────┬─────────────────────┘
     │ tool call: bash("find ~/Desktop ...")
     ▼
┌──────────────────────────┐
│ Pi Tool Runtime           │
│ Args, cancel, collect output │
└────┬─────────────────────┘
     ▼
┌──────────────────────────┐
│ Shell / macOS file system  │
│ zsh/bash → find/ls → Desktop│
└────┬─────────────────────┘
     │ stdout / stderr / exit code
     ▼
┌──────────────────────────┐
│ ToolResult + Session JSONL│
│ Record result, return it    │
└────┬─────────────────────┘
     │ One more model request
     ▼
┌──────────────────────────┐
│ Model writes final answer    │
└────┬─────────────────────┘
     ▼
┌─────────┐
│   You   │
└─────────┘

There are usually at least two model requests:

Decision request: After reading the question, the model decides which tool to use;
Summary request: After reading the real directory contents returned by the tool, the model turns them into a human answer.

3. The Participants: What Does Each Component Do?

Component	Main responsibility	Not responsible for
Pi TUI	Input, displaying text/tool calls/results, handling shortcuts	Understanding natural language or choosing commands
`AgentSession`	Managing sessions, tools, message queues, persistence, and context compaction	Deciding which tool to call
Agent Core	Driving the "model → tool → model" loop	Accessing the desktop directly
Model Runtime	Sending network requests to the selected provider and model	Running local programs
LLM	Understanding intent, choosing tools, interpreting tool results	Having access to the local file system
Tool Registry	Exposing available tools and their parameter structures to the model	Making decisions for the model
`bash` / `ls` / `find` tools	Performing controlled local operations and returning results	Understanding the user intent behind "desktop files"
Shell and macOS	Running commands and reading the file system	Knowing about Pi, the model, or the session
Session Manager	Saving messages, tool results, model switches, and more as JSONL	Reasoning
Extension / Skill	Adding tools, rules, UI, or event handling	Being part of Pi's minimal core by default

4. Step by Step: What Actually Happens?

Step 0: What Has Pi Prepared at Startup?

Before you type anything, Pi has already initialized its runtime environment.

0.1 Determine the Current Working Directory

Pi has a current working directory (cwd). In this example, it might be:

/Users/bw

The "desktop" usually corresponds to:

/Users/bw/Desktop

The model can use an absolute path in tool arguments, or a shortcut the shell understands:

~/Desktop

The shell expands ~ to the current user's home directory, such as /Users/bw.

0.2 Load Rules and Context Files

Pi reads context files such as AGENTS.md or CLAUDE.md according to their scope, for example:

~/.pi/agent/AGENTS.md              Global rules
AGENTS.md / CLAUDE.md in parents   Project or directory rules
AGENTS.md in current directory     Current project rules

These files affect the model's behavior, such as:

Which tools are available;
Whether to read files with read or bash;
Which commands or directories are restricted;
Whether a Skill should be read first;
The output format and language.

0.3 Register Tools and Describe Them to the Model

Pi provides the currently enabled tools to the model. Available tools may differ depending on the installation, extensions, and startup options.

Common built-in or environment-provided tools include:

read    Read files or images
write   Write new files
edit    Make precise edits to existing files
bash    Run shell commands
find    Find files
ls      List directories
grep    Search text

The model does not see the executable program itself; it sees a "tool manual." Conceptually, the bash definition looks like this:

{
  "name": "bash",
  "description": "Run a bash command, such as ls, grep, or find.",
  "parameters": {
    "command": "string",
    "timeout": "number, optional"
  }
}

This structure is usually called a tool schema or function definition.

A model is not naturally able to operate a computer. Based on the tool description, it outputs a structured request that says, in effect, "please call this tool for me."

Step 1: You Enter a Request in the Terminal

You type:

Show me what files are on my desktop

When you press Enter, the TUI passes the text to AgentSession.prompt().

Think of AgentSession as Pi's "session manager." It is responsible for:

- Saving and restoring sessions
- Managing the current model, provider, and thinking level
- Managing enabled tools
- Receiving agent events
- Handling the message queue
- Handling context compaction and retries
- Writing key records to JSONL

If you press Enter again while the agent is working, Pi queues the message as a steering message by default. If you use Alt+Enter, it is queued as a follow-up and handled only after the current work is completely finished.

Step 2: The User Message Enters the Session

Pi adds the sentence to the current session state as a user message and, by default, appends it to the session JSONL file.

Conceptually, it looks like this:

{
  "type": "message",
  "id": "a1b2c3d4",
  "parentId": "ID of the previous message",
  "timestamp": "2026-08-14T10:00:00.000Z",
  "message": {
    "role": "user",
    "content": "Show me what files are on my desktop"
  }
}

By default, session files are saved in:

~/.pi/agent/sessions/

Pi stores sessions as JSONL (JSON Lines): each line is a JSON object. This makes sessions easy to append, restore, export, and debug.

Step 3: Pi Builds the Model Request

Pi does not send your sentence to the model on its own. It builds the full context, which looks roughly like this:

┌──────────────────────────────────────┐
│ System Prompt                         │
│ "You are a coding agent..."            │
│ Tool-use rules and behavioral limits   │
├──────────────────────────────────────┤
│ Context Files / Skills                │
│ AGENTS.md and Skill-specific guidance  │
├──────────────────────────────────────┤
│ Tool Definitions                      │
│ Names, descriptions, and args for read / bash / edit / write │
├──────────────────────────────────────┤
│ Conversation History                  │
│ History and tool results on this session branch │
├──────────────────────────────────────┤
│ Current User Message                  │
│ "Show me what files are on my desktop" │
└──────────────────────────────────────┘

"History" does not mean endlessly concatenating every old message. When a long session approaches the model's context-window limit, Pi can use compaction: it summarizes older history, keeps recent messages, and reduces token usage.

Step 4: Pi Sends the Request to the Model Through a Provider

Based on the current settings, Pi's model runtime chooses:

Provider: for example, DeepSeek / Anthropic / OpenAI / Gemini
Model: a specific model ID
Authentication: an API key or subscription login credentials
Transport: SSE, WebSocket, or automatic selection

It then sends an HTTP request to the model service and starts receiving a streaming response.

Streaming means the model's text, thinking blocks (if supported by the provider), and tool-call blocks may arrive piece by piece. Pi passes these incremental events to the terminal interface, so you usually see content appear gradually instead of waiting for everything to finish.

Step 5: The Model Decides, "I Need to Look at the Real Desktop"

After seeing your question, the model usually reasons:

The user is asking about real files on this computer.
I cannot answer from training knowledge.
I have tools such as bash / ls / find.
I should list the contents of ~/Desktop first.

So the model may generate a tool call such as:

{
  "type": "toolCall",
  "id": "call_abc123",
  "name": "bash",
  "arguments": {
    "command": "find ~/Desktop -maxdepth 1 -type f -print"
  }
}

It may also choose:

ls -la ~/Desktop

Or, if separate ls / find tools are available, it may call one of them directly.

Important: Who Decides Which Command to Use?

Decision	Primarily responsible
Whether to inspect the real environment	LLM
Whether to use `bash`, `find`, `ls`, or another tool	LLM
Command arguments, paths, and filters	LLM
Whether a tool exists and is enabled	Pi runtime / configuration
How the tool call is actually executed	Pi tool implementation
How the shell accesses the file system	Operating system

In other words, Pi provides the "hands and eyes," and the model decides when and how to use them.

Step 6: Pi Receives the Tool Call and Dispatches the Tool

The model cannot run local commands directly; it can only return a structured "tool-call intent."

When Pi's agent runtime receives it, it:

1. Finds the tool definition for name = "bash";
2. Reads arguments.command;
3. Creates a cancellable AbortSignal for this operation;
4. Emits a tool-call event for the TUI and extensions to observe;
5. Calls the bash tool's execute();
6. Waits for the tool to succeed, fail, time out, or be cancelled.

So what you see in Pi's terminal:

$ find ~/Desktop -maxdepth 1 -type f -print

is the TUI rendering a tool-call event. The model does not actually have its own terminal window.

Step 7: How Does the `bash` Tool Interact with macOS?

Pi's local bash tool starts a shell using Node.js child-process APIs. Conceptually:

spawn(shell, shellArguments + [command], {
  cwd: currentWorkingDirectory,
  env: environment,
  stdio: [/* stdin, stdout, stderr */]
})

The actual chain:

Pi bash tool
  │ Node.js child_process.spawn()
  ▼
macOS shell (zsh / bash)
  │ Interprets command and ~ path
  ▼
System commands such as find or ls
  │ System calls
  ▼
macOS file system
  │ Returns directory entries
  ▼
stdout / stderr + exit code
  │
  ▼
Pi bash tool

For example:

find ~/Desktop -maxdepth 1 -type f -print

The shell expands ~ to:

/Users/bw

So the directory accessed is:

/Users/bw/Desktop

Suppose it contains:

/Users/bw/Desktop/Project Notes.md
/Users/bw/Desktop/Screenshot.png
/Users/bw/Desktop/Test Data.xlsx

This text is returned to Pi through the command's standard output (stdout).

Step 8: Live Output, Truncation, Cancellation, and Errors During Tool Execution

8.1 Live Output

The bash tool listens to the child process's:

stdout: normal output
stderr: error output

If a command runs for a long time and keeps producing output, Pi can send update events to the TUI as data arrives, so the interface shows partial results.

8.2 Output Truncation

If the output is huge, Pi does not stuff all of it into the model context. The tool keeps a limited number of lines and bytes. When output is truncated, it tells the model:

Output truncated; the full output was saved to a temporary file.

This is an important point in agent engineering: tool output is also context, and it consumes tokens.

8.3 User Cancellation

When you press Escape, the typical chain is:

You press Escape
  → TUI calls session.abort()
  → Agent / tool receives AbortSignal
  → bash tool terminates the shell and its process tree
  → The current tool call ends as aborted
  → Pi updates the UI and session state

8.4 Timeouts

If the tool call has a timeout:

The time limit is exceeded
  → bash tool terminates the process tree
  → Returns a "command timed out" error
  → The error is passed to the model as a tool result

8.5 Missing Files or Insufficient Permissions

For example, if the directory does not exist, the shell may return:

find: /Users/bw/Desktop: No such file or directory

Pi does not swallow the error. It wraps it as a tool result with isError: true. When the model sees it, it can:

Explain the error;
Try another path;
Ask the user;
Stop.

Step 9: stdout Becomes a Structured Tool Result

When the tool finishes, Pi converts its output into a structured message linked to the original call ID:

{
  "role": "toolResult",
  "toolCallId": "call_abc123",
  "toolName": "bash",
  "content": [
    {
      "type": "text",
      "text": "/Users/bw/Desktop/Project Notes.md\n/Users/bw/Desktop/Screenshot.png\n/Users/bw/Desktop/Test Data.xlsx"
    }
  ],
  "isError": false
}

The most important part here is toolCallId:

assistant toolCall.id      = call_abc123
tool result toolCallId     = call_abc123

This lets the model, Pi, the session log, and the UI all know: this result belongs to that tool call.

Tool results are also appended to the JSONL session, so they can later be restored, exported, replayed, and reviewed.

Step 10: Pi Sends the Result Back to the Model

Pi now makes another model request. The relevant part of the model's context looks like this:

User: Show me what files are on my desktop

Assistant: I'll use the bash tool to inspect the directory.

Assistant tool call:
find ~/Desktop -maxdepth 1 -type f -print

Tool result:
/Users/bw/Desktop/Project Notes.md
/Users/bw/Desktop/Screenshot.png
/Users/bw/Desktop/Test Data.xlsx

Note: the model did not read your hard drive directly.

The model only reads the tool result Pi gives it.
A tool result is the model's "sensory input" from the real outside world.

Step 11: The Model Generates the Final Answer

Now that the model has real data, it can generate:

There are currently 3 files on your desktop:

- Project Notes.md
- Screenshot.png
- Test Data.xlsx

If the model does not make another tool call, the provider marks this turn as complete with a finish reason such as stop.

Pi then:

1. Receives the final AssistantMessage;
2. Writes it to the session JSONL;
3. Updates token, cache-hit, and cost statistics;
4. Emits an agent-finished/settled event;
5. Has the TUI display the final text to you.

5. The Same Flow as a Sequence Diagram

You            Pi TUI        AgentSession       LLM API        bash tool        Shell / file system
│                 │                │                │                │                  │
│ Ask question     │                │                │                │                  │
├────────────────>│                │                │                │                  │
│                 │ prompt()       │                │                │                  │
│                 ├───────────────>│                │                │                  │
│                 │                │ Save user msg  │                │                  │
│                 │                │ Build context  │                │                  │
│                 │                ├───────────────>│                │                  │
│                 │                │                │ Analyze; decide to call bash      │
│                 │                │<───────────────┤ toolCall        │                  │
│                 │ Show tool call  │                │                │                  │
│                 │<───────────────┤                │                │                  │
│                 │                │ execute()      │                │                  │
│                 │                ├───────────────────────────────>│                  │
│                 │                │                │                │ spawn zsh/find   │
│                 │                │                │                ├─────────────────>│
│                 │                │                │                │ Dir contents     │
│                 │                │                │                │<─────────────────┤
│                 │                │                │ ToolResult     │                  │
│                 │                │<───────────────────────────────┤                  │
│                 │ Show result     │                │                │                  │
│                 │<───────────────┤                │                │                  │
│                 │                │ Put ToolResult in the next request │                  │
│                 │                ├───────────────>│                │                  │
│                 │                │                │ Generate final natural-language answer │
│                 │                │<───────────────┤                │                  │
│ Final answer     │                │ Save assistant msg              │                  │
│<────────────────┤<───────────────┤                │                │                  │

6. What Is the Agent's Core Loop?

The simplest agent loop can be written in pseudocode:

messages.push(userMessage)

while (true) {
  const response = await llm.generate({
    systemPrompt,
    tools,
    messages,
  })

  messages.push(response.assistantMessage)

  if (response.toolCalls.length === 0) {
    showToUser(response.text)
    break
  }

  for (const toolCall of response.toolCalls) {
    const result = await tools.execute(toolCall)
    messages.push(result)
  }
}

For "list my desktop files," the loop usually runs twice:

Loop 1: user question → model requests the bash tool
Loop 2: tool result → model writes a natural-language answer

For a complex task such as "fix a bug in this project," the loop may run many times:

Read files → search code → edit files → run tests → read errors → edit again → test again → final answer

This is the fundamental mechanism that lets a coding agent keep working through complex tasks.

7. Why Is a Tool Result the Agent's "Senses"?

A model on its own only has input context and the ability to output text or structured calls. It has no direct access to the real world.

Different tools give it different "senses":

Tool	What the model can perceive through it
`read`	File contents, code, images
`find` / `ls`	File and directory structure
`grep`	Where a keyword appears in files
`bash`	Command results, Git status, test results, system information
Browser tool	Web pages and DOM content
Database tool	Business data and query results
HTTP/API tool	Data returned by external services
Sub-agent	A report from another specialized execution unit

So an agent is not a model that has suddenly become all-powerful. It is:

The model's planning ability
+ A structured tool-call protocol
+ Facts brought back by external tools
+ A loop and state managed by the runtime
= An agent that can take action

8. Why Do Sessions Matter in Pi?

Pi stores sessions as JSONL by default and uses id and parentId to form a tree, rather than a single chat log that cannot branch.

This supports:

/resume  Resume an older session
/tree    Continue from a past node while keeping branches
/fork    Create a new session from a user message
/clone   Copy the current branch
/compact Compact context in a long session
/export  Export HTML or JSONL

A session can record:

- User messages
- Assistant messages
- Tool calls and tool results
- Model switches
- Thinking-level switches
- Context-compaction summaries
- Custom state or custom messages written by extensions

This means Pi is not just a "terminal chat window"; it is an agent runtime with history, state, branches, and recovery.

9. Where Do Extensions and Skills Fit In?

Pi's core design stays minimal while allowing extensions.

Skill

A Skill is usually an on-demand Markdown instruction file. It can tell the model:

For this kind of task:
1. Which files to read first;
2. Which tools to use;
3. Which limits to follow;
4. How to verify the result.

It mainly changes the model's "working method" and context; it does not execute code directly.

Extension

An Extension is TypeScript code that can hook more deeply into the runtime, for example:

- Register custom tools
- Validate or approve before tool_call
- Replace tool implementations
- Add commands and keyboard shortcuts
- Listen to agent events
- Customize the TUI
- Connect to remote shells, sandboxes, MCP, or business APIs
- Implement advanced workflows such as sub-agents and Plan Mode

So an extension can take the original:

bash → local shell

and replace it with:

bash → approval layer → Docker sandbox / SSH server / cloud executor

The model is still just calling an abstract tool named bash; where it actually runs can change.

10. Security Boundaries: Why Can't the Model Take Over Your Computer Directly?

The model's output is only text or a structured tool call; it cannot execute directly on your system.

The tools registered by Pi are what actually perform actions. That means permission boundaries belong here:

Model
  ↓ requests
Tool Schema / Tool Registry
  ↓ dispatches
Permission and approval policy (can be implemented by an extension)
  ↓
Concrete tool implementation
  ↓
Shell / file system / network / database

Pi's philosophy is to stay minimal by default. It does not build in fixed permission prompts, a sandbox, or a sub-agent system. You can run it in a container, or use extensions to add approval and security policies suited to your environment.

This reminds us:

"Will the model try a dangerous command?" is a model-behavior question.
"Can a dangerous command actually run?" must be controlled by the runtime and tool layer.

11. Three Key Lessons for Anyone Learning About Agents

11.1 The LLM Is the Brain, Not the Whole Agent

The LLM is responsible for:

Understanding intent, choosing tools, planning the next step, and interpreting results.

But it is not responsible for:

Accessing files, running commands, saving sessions, terminating processes, rendering the UI, or enforcing permissions.

Those are the harness/runtime's responsibilities.

11.2 Tool Output Must Become the Model's Next Context

After a tool runs, the model does not "automatically know the result." The runtime must package the result as a tool result and send it back to the model.

Model requests a tool
→ Runtime executes
→ Runtime creates ToolResult
→ ToolResult is added to the next model context
→ Model reasons from it

This is the agent's observe-act loop.

11.3 The Hard Part in Production Is the Control System, Not the Tool Call Format

A demo is easy:

User → model → tool → model answer

A production system must also handle:

Multi-turn loops, concurrent tools, timeouts, cancellation, retries, permissions, session recovery,
context compaction, output truncation, cost tracking, observability, auditing, and extensibility.

This "engineering beyond the model" determines whether an agent is truly usable.

12. Final Summary

When you ask Pi to "look at the files on my desktop," a natural-language sentence does not simply turn into a system command. What actually happens is a closed loop:

User intent
  → Pi assembles context and tool descriptions
  → The model decides to call a tool
  → Pi dispatches the tool
  → Shell / operating system reads the real world
  → Pi records the result and wraps it as a ToolResult
  → The model answers based on real results
  → Pi saves the session and shows it to the user

In the shortest form:

The model decides what to look at and how to ask; tools do the actual looking; Pi makes the two work together reliably.

References: Pi's Local Implementation and Documentation

Pi overview: /opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/README.md
Session format: /opt/homebrew/lib/node_modules/@earendil-works/pi-coding-agent/docs/session-format.md
AgentSession type: dist/core/agent-session.d.ts
bash tool implementation: dist/core/tools/bash.js
read tool implementation: dist/core/tools/read.js

Pi Agent: How One Sentence Becomes a Look at Your Desktop Files

Pi Agent: How One Sentence Becomes a Look at Your Desktop Files

1. Start with the Right Mental Model

2. The Big Picture

3. The Participants: What Does Each Component Do?

4. Step by Step: What Actually Happens?

Step 0: What Has Pi Prepared at Startup?

0.1 Determine the Current Working Directory

0.2 Load Rules and Context Files

0.3 Register Tools and Describe Them to the Model

Step 1: You Enter a Request in the Terminal

Step 2: The User Message Enters the Session

Step 3: Pi Builds the Model Request

Step 4: Pi Sends the Request to the Model Through a Provider

Step 5: The Model Decides, "I Need to Look at the Real Desktop"

Important: Who Decides Which Command to Use?

Step 6: Pi Receives the Tool Call and Dispatches the Tool

Step 7: How Does the bash Tool Interact with macOS?

Step 8: Live Output, Truncation, Cancellation, and Errors During Tool Execution

8.1 Live Output

8.2 Output Truncation

8.3 User Cancellation

8.4 Timeouts

8.5 Missing Files or Insufficient Permissions

Step 9: stdout Becomes a Structured Tool Result

Step 10: Pi Sends the Result Back to the Model

Step 11: The Model Generates the Final Answer

5. The Same Flow as a Sequence Diagram

6. What Is the Agent's Core Loop?

7. Why Is a Tool Result the Agent's "Senses"?

8. Why Do Sessions Matter in Pi?

9. Where Do Extensions and Skills Fit In?

Skill

Extension

10. Security Boundaries: Why Can't the Model Take Over Your Computer Directly?

11. Three Key Lessons for Anyone Learning About Agents

11.1 The LLM Is the Brain, Not the Whole Agent

11.2 Tool Output Must Become the Model's Next Context

11.3 The Hard Part in Production Is the Control System, Not the Tool Call Format

12. Final Summary

References: Pi's Local Implementation and Documentation

Step 7: How Does the `bash` Tool Interact with macOS?