Coordinating Multiple AI Research Agents Through Git - A Practical Guide
Coordinating Multiple AI Research Agents Through Git - A Practical Guide
When you have multiple AI agents researching different aspects of a problem simultaneously, they need a way to share findings without stepping on each other. The obvious approach is a message bus or shared memory system. The simpler approach is git.
Git is already a distributed coordination system designed for multiple contributors working on shared artifacts. AI agents are just fast, parallel contributors.
Why Git Beats a Message Bus for Research
Message buses (Redis, RabbitMQ, custom pub/sub) add infrastructure complexity. You need to define message formats, handle ordering, deal with delivery guarantees, and build retry logic. Git gives you all of this for free - ordering through commit history, delivery through push/pull, and a built-in conflict resolution mechanism.
The other advantage is persistence. Every finding is permanently recorded with full context. You can replay the entire research process by reading the commit log. The author, timestamp, and diff are all there. Try doing that with ephemeral Redis messages.
Teams running five agents in parallel on git worktrees report roughly 5x throughput improvements compared to sequential research, with none of the coordination overhead that message buses require.
Git Worktrees: The Key to Isolation
The critical component is git worktree, which allows multiple branches to be checked out simultaneously in separate directories from a single repository. Each agent gets its own directory with its own branch, but they all share the same .git history.
This is different from cloning the repo multiple times. Worktrees are lightweight - they share object storage, reflog, and configuration. Creating five worktrees takes about as long as creating five directories.
# Set up a research project with 5 parallel agent workspaces
git init research-project
cd research-project
# Create the main synthesis branch
git commit --allow-empty -m "init research project"
# Create isolated worktrees for each agent
git worktree add ../agent-1 -b agent/topic-1
git worktree add ../agent-2 -b agent/topic-2
git worktree add ../agent-3 -b agent/topic-3
git worktree add ../agent-4 -b agent/topic-4
git worktree add ../agent-5 -b agent/topic-5
Each agent now has a completely isolated working directory. Agent 1 writing to ../agent-1/findings.md cannot conflict with Agent 2 writing to ../agent-2/findings.md - they are separate filesystem paths on separate branches.
The Research Commit Format
Agents should commit findings in a structured format that makes them easy to synthesize later. Plain markdown works well:
<!-- findings.md committed by agent-1 -->
# Topic: Competitor Pricing Models
## Source: Company A Annual Report (2025)
- Base tier: $49/month, 10k API calls
- Pro tier: $199/month, 100k API calls
- Enterprise: custom pricing, confirmed by sales call transcript
## Source: TechCrunch Interview (March 2025)
- CEO mentioned "usage-based pricing shift in Q3"
- No public announcement yet as of commit date
## Confidence: High
## Contradicts: agent-3/findings.md claim that "all major players use flat pricing"
The Contradicts field is intentional. When agents commit findings that explicitly flag conflicts with other branches, the coordinator script can surface these for human review before synthesis.
The Coordinator Script
A simple coordinator runs on a loop, merging branches and flagging conflicts:
#!/usr/bin/env python3
"""coordinator.py - merges agent branches and reports conflicts"""
import subprocess
import sys
from pathlib import Path
AGENT_BRANCHES = [f"agent/topic-{i}" for i in range(1, 6)]
SYNTHESIS_BRANCH = "main"
def run(cmd: list[str]) -> tuple[int, str]:
result = subprocess.run(cmd, capture_output=True, text=True)
return result.returncode, result.stdout + result.stderr
def check_for_updates() -> list[str]:
updated = []
for branch in AGENT_BRANCHES:
code, out = run(["git", "log", f"{SYNTHESIS_BRANCH}..{branch}", "--oneline"])
if code == 0 and out.strip():
updated.append(branch)
return updated
def merge_branch(branch: str) -> tuple[bool, str]:
code, out = run(["git", "merge", "--no-ff", branch,
"-m", f"Merge findings from {branch}"])
return code == 0, out
def main():
run(["git", "checkout", SYNTHESIS_BRANCH])
updated = check_for_updates()
if not updated:
print("No new findings to merge")
return
conflicts = []
for branch in updated:
success, output = merge_branch(branch)
if not success:
conflicts.append((branch, output))
run(["git", "merge", "--abort"])
print(f"CONFLICT in {branch} - requires human review")
else:
print(f"Merged {branch}")
if conflicts:
print("\nBranches with conflicts (review before merging):")
for branch, details in conflicts:
print(f" {branch}: {details[:200]}")
sys.exit(1)
if __name__ == "__main__":
main()
Merge conflicts in this system are meaningful signals. When two agents produce conflicting findings about the same topic, that is exactly the kind of contradiction you want surfaced before synthesis. A merge conflict is not a failure state - it is a quality signal.
What the Commit Log Becomes
After a research session, the commit log is a complete audit trail:
a3f8c12 Merge findings from agent/topic-3
d91bc44 [agent-3] Add pricing data from 5 additional competitors
7e22a01 Merge findings from agent/topic-1
4b8a39d [agent-1] Correct confidence level on Q3 pricing claim - source unclear
2c7f91e [agent-1] Initial pricing analysis for top 10 competitors
e5d1820 [agent-4] Find contradictory claim in agent-3 findings (flagged)
...
You can reconstruct exactly what each agent found, when it found it, and how findings evolved as agents read each other's commits and updated their assessments.
When Git Coordination Breaks Down
This approach works for research and analysis tasks where agents produce documents. It has real limits:
Real-time coordination - agents need sub-second response times to each other's findings. Git commit-pull cycles take seconds at minimum. For real-time interaction, you still need in-memory communication.
Binary or large files - git handles text diffs well. It handles large binary files poorly. Research that involves screenshots, audio, or video cannot be efficiently coordinated through commits.
More than ~20 agents - the merge coordinator becomes a bottleneck. Managing 20 branches and resolving conflicts at scale requires more sophisticated tooling, closer to what Composio's agent-orchestrator or ccswarm provide.
For the "go research five topics and bring me a synthesis in 30 minutes" pattern, git commits beat custom infrastructure every time.
Connecting to Claude Code
Claude Code's --worktree flag (or manual worktree setup with CLAUDE.md per branch) integrates naturally with this pattern. Each Claude session gets its own branch and directory. The session's memory of what it has found lives in the commit history, survives context window resets, and is readable by both the agent and the human overseeing the research.
# Launch Claude in an isolated agent worktree
cd ../agent-1
claude --print "Research topic: competitor pricing models.
Commit your findings to findings.md as you discover them.
Flag any contradictions with other agents' branches."
The agent commits, the coordinator merges, the human reads the synthesis. Three roles, clean separation, no custom infrastructure required.
Fazm is an open source macOS AI agent. Open source on GitHub.