-
Notifications
You must be signed in to change notification settings - Fork 952
feat: update tasks docs #18659
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
feat: update tasks docs #18659
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
00e8387
feat: draft tasks docs
bpmct 9132992
expand description
bpmct b5b5862
expand
bpmct fb29612
fix clunky sentence
bpmct 3e8b2a1
docs: restructure AI agents documentation with updated content and ex…
bpmct e853b03
format
bpmct 39da22b
fmt
bpmct e555146
Merge branch 'main' into tasks-docs
bpmct bdfaf60
fix typos
bpmct e4a73ff
fix broken links
bpmct 02e6b9c
fix more broken links
bpmct b36c2dd
fmt
bpmct 8e5b2df
chore: make suggestions to the Tasks documentation (#18714)
hugodutka 0dae4e1
apply feedback from hugo review
bpmct 84060eb
Update docs/ai-coder/custom-agents.md
bpmct da8873f
use proper URL
bpmct 58d7c80
add MCP server docs
bpmct d3ea0ae
fixup
bpmct 394180c
fix again
bpmct 0762280
fix typo
bpmct 04b77a4
Merge branch 'main' into tasks-docs
bpmct f8ab168
fix trailing comma
bpmct 362bc55
fmt
bpmct d817640
fmt again
bpmct File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,71 +1,53 @@ | ||
# Model Context Protocols (MCP) and adding AI tools | ||
# Best Practices | ||
|
||
> [!NOTE] | ||
> | ||
> This functionality is in beta and is evolving rapidly. | ||
> | ||
> When using any AI tool for development, exercise a level of caution appropriate to your use case and environment. | ||
> Always review AI-generated content before using it in critical systems. | ||
> | ||
> Join our [Discord channel](https://discord.gg/coder) or | ||
> [contact us](https://coder.com/contact) to get help or share feedback. | ||
This document includes a mix of cultural and technical best practices and guidelines for introducing AI agents into your organization. | ||
|
||
## Overview | ||
## Identify Use Cases | ||
|
||
Coder templates should be pre-equipped with the tools and dependencies needed | ||
for development. With AI Agents, this is no exception. | ||
To successfully implement AI coding agents, identify 3-5 practical use cases where AI tools can deliver real value. Additionally, find a target group of developers and projects that are the best candidates for each specific use case. | ||
|
||
## Prerequisites | ||
Below are common scenarios where AI coding agents provide the most impact, along with the right tools for each use case: | ||
|
||
- A Coder deployment with v2.21 or later | ||
- A [template configured for AI agents](./create-template.md) | ||
| Scenario | Description | Examples | Tools | | ||
|------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------| | ||
| **Automating actions in the IDE** | Supplement tedious development with agents | Small refactors, generating unit tests, writing inline documentation, code search and navigation | [IDE Agents](./ide-agents.md) in Workspaces | | ||
| **Developer-led investigation and setup** | Developers delegate research and initial implementation to AI, then take over in their preferred IDE to complete the work | Bug triage and analysis, exploring technical approaches, understanding legacy code, creating starter implementations | [Tasks](./tasks.md), to a full IDE with [Workspaces](../user-guides/workspace-access/index.md) | | ||
| **Prototyping & Business Applications** | User-friendly interface for engineers and non-technical users to build and prototype within new or existing codebases | Creating dashboards, building simple web apps, data analysis workflows, proof-of-concept development | [Tasks](./tasks.md) | | ||
| **Full background jobs & long-running agents** | Agents that run independently without user interaction for extended periods of time | Automated code reviews, scheduled data processing, continuous integration tasks, monitoring and alerting | [Tasks](./tasks.md) API *(in development)* | | ||
| **External agents and chat clients** | External AI agents and chat clients that need access to Coder workspaces for development environments and code sandboxing | ChatGPT, Claude Desktop, custom enterprise agents running tests, performing development tasks, code analysis | [MCP Server](./mcp-server.md) | | ||
|
||
## Best Practices | ||
## Provide Agents with Proper Context | ||
|
||
- Use the most capable ML models you have access to in order to evaluate Agent | ||
performance. | ||
- Set a system prompt with the `AI_SYSTEM_PROMPT` environment in your template | ||
- Within your repositories, write a `.cursorrules`, `CLAUDE.md` or similar file | ||
to guide the agent's behavior. | ||
- To read issue descriptions or pull request comments, install the proper CLI | ||
(e.g. `gh`) in your image/template. | ||
- Ensure your [template](./create-template.md) is truly pre-configured for | ||
development without manual intervention (e.g. repos are cloned, dependencies | ||
are built, secrets are added/mocked, etc.). | ||
While LLMs are trained on general knowledge, it's important to provide additional context to help agents understand your codebase and organization. | ||
|
||
> Note: [External authentication](../admin/external-auth/index.md) can be helpful | ||
> to authenticate with third-party services such as GitHub or JFrog. | ||
### Memory | ||
|
||
- Give your agent the proper tools via MCP to interact with your codebase and | ||
related services. | ||
- Read our recommendations on [securing agents](./securing.md) to avoid | ||
surprises. | ||
Coding Agents like Claude Code often refer to a [memory file](https://docs.anthropic.com/en/docs/claude-code/memory) in order to gain context about your repository or organization. | ||
|
||
## Adding Tools via MCP | ||
Look up the docs for the specific agent you're using to learn more about how to provide context to your agents. | ||
|
||
Model Context Protocol (MCP) is an emerging standard for adding tools to your | ||
agents. | ||
### Tools (Model Context Protocol) | ||
|
||
Follow the documentation for your [agent](./agents.md) to learn how to configure | ||
MCP servers. See | ||
[modelcontextprotocol/servers](https://github.com/modelcontextprotocol/servers) | ||
to browse open source MCP servers. | ||
Agents can also use tools, often via [Model Context Protocol](https://modelcontextprotocol.io/introduction) to look up information or perform actions. A common example would be fetching style guidelines from an internal wiki, or looking up the documentation for a service within your catalog. | ||
|
||
### Our Favorite MCP Servers | ||
Look up the docs for the specific agent you're using to learn more about how to provide tools to your agents. | ||
|
||
In internal testing, we have seen significant improvements in agent performance | ||
when these tools are added via MCP. | ||
#### Our Favorite MCP Servers | ||
|
||
In internal testing, we have seen significant improvements in agent performance when these tools are added via MCP. | ||
|
||
- [Playwright](https://github.com/microsoft/playwright-mcp): Instruct your agent | ||
to open a browser, and check its work by viewing output and taking | ||
screenshots. | ||
- [desktop-commander](https://github.com/wonderwhy-er/DesktopCommanderMCP): | ||
Instruct your agent to run long-running tasks (e.g. `npm run dev`) in the | ||
background instead of blocking the main thread. | ||
Instruct your agent to run long-running tasks (e.g. `npm run dev`) in the background instead of blocking the main thread. | ||
|
||
## Security & Permissions | ||
|
||
LLMs and agents can be dangerous if not run with proper boundaries. Be sure not to give agents full permissions on behalf of a user, and instead use separate identities with limited scope whenever interacting autonomously. | ||
|
||
[Learn more about securing agents with Coder Tasks](./security.md) | ||
|
||
## Next Steps | ||
## Keep it Simple | ||
|
||
- [Supervise Agents in the UI](./coder-dashboard.md) | ||
- [Supervise Agents in the IDE](./ide-integration.md) | ||
- [Supervise Agents Programmatically](./headless.md) | ||
- [Securing Agents](./securing.md) | ||
Today's LLMs and AI agents are not going to refactor entire codebases with production-grade code on their own! Using coding agents can be extremely fun and productive, but it is important to keep the scope of your use cases small and simple, and grow them over time. |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tell a lot of customers that background jobs should always have a clear outcome with tests. I think it's good to explain that long-running tasks should have well-defined final states.